OpenAI authoring¶

Status: Current | Last reviewed: 2026-09-17

Select OpenAI Agents API in Settings → User Experience → AI Generation to use the managed Codex harness for new Generate with AI jobs, editor assistant conversations, and conversational designs.

Connection and model¶

Choose an active OpenAI integration. Its API key needs api.agents.read, api.agents.write, and api.responses.write. The configured base URL must implement /v1/agents; an endpoint that only implements Chat Completions is insufficient. The engine is openai_agents, separate from the unimplemented codex CLI authoring option.

GENERATION_MODEL_OPENAI pins the authoring model (default gpt-6-astra). GENERATION_REASONING_EFFORT_OPENAI pins reasoning effort (default low, selected from the authoring benchmark). Both defaults apply to new generation jobs, Studio designs and editor assistant sessions. Trusted per-job benchmark overrides still take precedence. Existing managed sessions retain their model and effort; start a new design or editor chat to use the updated defaults. GENERATION_SMALL_MODEL_OPENAI selects the model for Activity Log summaries and public research (default gpt-4.1-mini). Procedure derivation uses the main model through Responses. These choices do not change the generated agent's stage models.

The selected provider and integration are saved on each authoring session. Changing the tenant default affects new conversations and new generation jobs. Start a new editor chat or design to use the new engine. Retrying a failed generation creates a new job using the selected engine.

The Agents API retains authoring state in the United States and does not support Zero Data Retention. This applies even when execution is self-hosted. See the OpenAI overview.

Execution¶

The existing workspace-agent-runner provides a function bridge implemented in openai_authoring.py. It creates a session with environment.type=none and delegation disabled. There is no model-accessible filesystem, shell, or remote MCP credential.

The bridge:

Authenticates to /api/mcp/generation with the existing scoped token.
Reads that owner's tool catalog and converts it to function definitions.
Supplies the existing generation/editor/designer protocol and target.
Adds bounded list_authoring_documents, search_authoring_documents and read_authoring_document functions over GENERATION_DOCS_DIR. Traversal and symlinks outside the documentation root are rejected.
Handles pending function calls through the same MCP dispatcher.
Polls saved session/turn state and emits the existing NDJSON activity, final text, error and completion envelopes.

Polling is deliberate: the provider's event stream does not replay missed events. The bridge checks the root turn's result, not merely idle, and paginates its saved items and filters each item by its turn ID locally (the session-items endpoint does not support a turn filter). Business-facing replies contain final text; commentary and internal working notes are not displayed.

MCP tools retain all existing tenant, target, access-mode, revision, Build draft authorization and draft-only restrictions. Model completion cannot substitute for complete_generation or complete_design_build. Lifecycle calls count only after the tool returns success.

Research and capability configuration¶

New sessions include search_public_web. It makes a separate Responses request through the selected OpenAI integration using hosted web_search. Only the explicit query is supplied: no draft, history, private Library excerpts, scoped MCP tools or MCP token. The call uses store:false, a 90-second limit, at most three hosted tool calls and a bounded sourced summary. The endpoint/model must support Responses web search. Unavailable research is reported as a tool failure; the authoring protocol continues with verified local references. Returned research usage belongs to this separate model call, not the managed turn's token total.

The shared MCP catalog also exposes same-tenant example agents, capability packs, published visual catalogs, focused get_configuration_schema lookups and optional TrueServe Library reads. The schema lookup combines universal stage fields with the chosen service's generated schema and includes referenced definitions. It distinguishes saved settings from runtime callable arguments; neither schema presence nor a demo proves channel availability. See generation research for configuration and the bundled capability handbooks.

Managed Agents sessions snapshot their tools and instructions at creation. Restarting the runner preserves an existing provider session, including its old catalog. Start a new design, generation job or editor New chat to pick up new research tools and protocol guidance; existing drafts and conversations remain available. See the provider's session configuration reference.

Persistence and recovery¶

Migration 061_openai_authoring.sql adds pinned runner/integration fields, provider cleanup timestamps and authoring_tool_calls. The existing cli_session_id column/envelope carries the provider session ID for OpenAI; it remains a Claude session ID for Claude. These fields never enter executable configuration.

Each bridged MCP call includes a receipt keyed by provider session, turn and call ID, bound to one tenant and generation/assistant owner. A completed receipt returns the saved result. An admitted call with an uncertain outcome is not executed again; the model must inspect saved state before another action. This prevents blind mutation retries after an interrupted response. Only metadata containing receipt identity fields opts into this check. Ordinary MCP _meta entries, including Claude Code's progress metadata, must remain valid. Partial receipt identities are rejected.

An idle OpenAI editor or designer conversation survives runner workspace loss: ai-manager recreates the bridge with a fresh scoped token while retaining the provider session ID. It does not replace the conversation with Claude's recap fallback. If a prior OpenAI turn is still active or failed, the adapter reports that condition instead of silently submitting another message.

Generation answers refresh the scoped token. If the runner lost a paused OpenAI job, the answer path redispatches it with its saved provider session. Cancelled/timed-out turns request provider cancellation. Terminal generations request provider-session deletion; closed/expired assistant sessions do too. The application's hourly session sweeper retries pending provider deletions from database records, including after runner loss. Failed deletions remain pending. Provider physical deletion can complete asynchronously.

Studio retains its own draft, history and tool receipts under its existing application storage lifecycle. Provider deletion does not delete these records.

Vanguard verification¶

Apply migrations 061 and 062 before restarting the updated application. Rebuild the runner after its Python sources change:

overmind restart workspace-agent-runner
overmind restart ai-manager

Use the settings page to select OpenAI and its integration. Existing designs retain their original engine; create a new design for the comparison.

To reuse an existing generation brief and its answered questions in a separate draft, run the explicit local test entry point:

cd apps/ai-manager
OPENAI_AUTHORING_RUN=1 \
OPENAI_AUTHORING_SOURCE_JOB=<generation-uuid> \
OPENAI_AUTHORING_INTEGRATION_ID=<openai-integration-uuid> \
bash ../../.devcontainer/scripts/with-env.sh \
  node --import tsx scripts/test-openai-authoring-local.ts

The script changes the source tenant's default engine to OpenAI, creates a new draft, prints its IDs and progress, and stops at completion, failure, or a question requiring the author. It keeps the source untouched and retains the comparison draft. It does not publish or run the resulting agent.

Compare the saved configuration, prompts, tools and any generated test artifacts. A completed generation proves that authoring validation passed; it does not prove the generated agent passed a runtime evaluation.

Repeatable model benchmarks¶

See the 11 September 2026 pilot results for measured model, reasoning and two-stage comparisons.

benchmark-authoring.ts creates separate draft generations from a real source brief. It adds the same explicit business decisions and order fixtures to every candidate, pins the provider per job, and preserves the tenant's default. The matrix includes GPT-5.6 Sol, GPT-5.6 Terra, GPT-6 Astra, Claude Sonnet 5 and Claude Opus 5. The main matrix requests medium reasoning effort, with a separate astra-low candidate requesting low (the API name for lighter reasoning). Equal labels do not imply equal compute budgets across providers.

Migration 062_authoring_execution.sql stores optional authoring_execution model, effort and transport settings on the generation job. They are trusted server-side experiment options, not fields accepted from a public generation request body. Saved options are used when resuming a job. They never enter the generated runtime configuration. Reviewer experiments may seed a new draft from another draft in the same tenant; the source is preserved.

The managed Agents API is measured separately (sol-agents, terra-agents, and astra-low-agents). Its session and turn usage is best-effort and can be null, so a successful run may have no cost estimate. See official usage guidance. The explicit responses experiment uses the same scoped MCP tools and reference documents through a local Responses loop. It records every response's model, effort, usage and time. Responses use store:false, carry encrypted reasoning items in memory, and expose no hosted shell or delegation. This loop has an 80-response limit plus the job wall-time limit; it does not offer the managed API's durable conversation recovery or automatic compaction. A resumed Responses experiment starts a new conversation over the saved draft. Each adapter receives its own document-tool definitions. The Responses-only strict field must never enter an Agents catalog; shared mutable definitions previously caused a later managed session to fail with HTTP 400 after a Responses experiment ran in the same worker.

The runner's internal generation-status response includes metrics and settled. Claude usage comes from the CLI's cumulative final result, including helper-model usage; partial assistant messages are not added again. OpenAI Responses metering is updated after every call. Managed Agents usage is captured before deletion when available. Metrics are in runner memory until expiry/restart; the benchmark controller checkpoints them into its output files. A terminal database status can precede the provider's final reply, so the controller waits for settled before taking its final snapshot.

Run from apps/ai-manager, using the devcontainer environment wrapper:

export AUTHORING_BENCHMARK_RUN=1
export AUTHORING_BENCHMARK_TENANT='tenant-uuid'
export AUTHORING_BENCHMARK_SOURCE='source-generation-uuid'
export AUTHORING_BENCHMARK_OPENAI='openai-integration-uuid'
export AUTHORING_BENCHMARK_BEDROCK='aws-integration-uuid'
export AUTHORING_BENCHMARK_OUTPUT='/tmp/tsf-authoring-benchmark'
AUTHORING_BENCHMARK_CANDIDATE=sol-agents \
  bash ../../.devcontainer/scripts/with-env.sh \
  node --import tsx scripts/benchmark-authoring.ts

Candidate names are sol-agents, terra-agents, astra-low-agents, sol, terra, astra, astra-low, sonnet, and opus. Set AUTHORING_BENCHMARK_REPEAT=2 for a second run; the controller refuses to overwrite an existing candidate/repeat file. A terra-sol or sonnet-opus run copies the completed terra or sonnet draft with the same repeat number, then asks the stronger model to review and repair it. The report adds both stages' measured cost and active time. It excludes the manual gap between launching those stages.

AUTHORING_BENCHMARK_COLLECT=1 reattaches to an existing dispatched run without starting another paid generation. Queue waiting is bounded separately (60 minutes) from active generation (31 minutes). The controller writes snapshots atomically; the provider's own job wall-time limit still applies.

Evaluate and report without changing the generated drafts:

bash ../../.devcontainer/scripts/with-env.sh \
  node --import tsx scripts/evaluate-authoring-benchmark.ts
AUTHORING_BENCHMARK_JUDGE=1 \
  bash ../../.devcontainer/scripts/with-env.sh \
  node --import tsx scripts/judge-authoring-benchmark.ts
node --import tsx scripts/report-authoring-benchmark.ts

The evaluator runs 21 independent order-service cases in code-executor with mock integration mode and no integration grants, including cross-order postal codes, absent confirmation and missing return details. Nineteen check allow/deny outcomes; two follow-up cases combine an incorrect item with wrong verification and assert that the result does not disclose the valid fixture SKU. These two cases were added after a blind review identified an untested combination and are applied to every unchanged artifact. They do not assert every response field. The evaluator records raw arguments/results and separately checks any saved canonical mock test suite. The report counts intact, verified sample specifications separately: altered source, parameters or input schemas can invalidate a prepared tool even when its independent cases pass. Rubric revisions retain the earlier evaluation files; refusal mappings include already_shipped, item_not_available and not_completed without treating unknown statuses as successful denials. Handwritten sample Python is tested only in this explicit benchmark sandbox, not by broadening the Design completion gate.

The optional paid judge sees the artifact without its authoring-model label and scores eight prompt criteria using Sol/high. Treat this as one model's review, separately from executed tests and authoring costs. Evaluation files are keyed by artifact hash so unchanged drafts are not graded again.

Outputs include per-run JSON snapshots, test results, judge results and report.json, report.csv, and an interactive report.html. Files are restricted to the local user and contain sample drafts and usage, never integration secrets. Set a run's excluded_reason when a confirmed setup failure invalidates its timing; the report retains but excludes it from its chart. Do not discard genuine model failures this way.

OpenAI cost ranges use the dated standard price snapshot and account for cached input and long contexts per response. The range reflects unavailable cache-write counts; reasoning is already included in output tokens. Claude costs are the CLI's list-price estimates, not an AWS invoice. Reverify the snapshot before later comparisons, especially Sol's promotional price. Missing usage means unknown cost. Local infrastructure, paid evaluation and live voice usage are outside these authoring estimates.

Use multiple repeats and additional briefs before selecting a general default. This pilot measures draft correctness and prompt quality; it does not establish live caller experience, merchant integration correctness or production tail latency.