Scenario generation skill and shared authoring services¶

Status: Current foundation; extension plan below | Last reviewed: 2026-09-29

The Allianz annuity build demonstrated that a good scenario needs more than a prompt: executable tools, trusted data, native authentication, distinct Copilot/voice/supervisor roles, real components, and evidence from the experience the caller actually hears or sees. The reusable TrueServe scenario builder skill captures that process without baking in Allianz identities, branding, models or business constants.

The skill guides task decomposition, prompt ownership and diagnosis. Shared services own contracts, draft concurrency and test planning; existing native services continue to own execution and authorization.

What is implemented¶

A portable SKILL.md with four focused references: native authoring, interaction/prompt patterns, validation/repair, and bounded Allianz evidence.
Generation, editor and designer protocols route scenario work to the skill. Claude workspaces receive it in the docs bundle; managed OpenAI sessions and external clients can read it over MCP without a filesystem.
get_agent_build_skill returns one allowlisted document, its reference index and the full Markdown package content hash.
get_scenario_test_plan derives relevant cases and explicit latency event pairs from declared components/features. This is deterministic code, with no model, database or runtime execution. Every case returns not_run.
External authoring exposes the existing builtin/configuration schema, capability, published multimodal catalog and Copilot card discovery tools. These remain read operations under agents:read.
Skill and test-plan management REST routes use the same handlers. MCP and A2A share the existing management registry.
The CLI's existing TrueServe skill points to the server's maintained skill. There is no second copy of scenario guidance to maintain in the CLI.

This change does not expose new external project-build, component-publish, human-decision or test-execution operations. It does not make a generated plan an executed test, or an assembled draft a validated deployment.

Distribution and execution¶

Scenario skill source ── generation-docs build ──┬─ runner docs/skills/
                                                └─ bundled MCP document registry

UI generation / designer ─┐
External Codex / Claude ──┼─ scoped adapters ── shared services ── native artifacts
REST / A2A automation ────┘

Source lives in shared/generation-docs/src/skills/trueserve-scenario-builder/. shared/generation-docs/build.ts copies it to the runner bundle and emits apps/ai-manager/lib/generation-mcp/scenario-skill.generated.json. Remote reads never accept a filesystem path or arbitrary URL. Editing a reference changes the digest; the parity test rejects a stale generated copy.

The runner protocol reads the entry point only for relevant scenario work, then loads references as needed. Existing managed provider sessions retain their instruction/tool snapshot; create a new job/chat to receive updated instructions. An existing external MCP client may also need to refresh its tool catalog.

The folder can be distributed as a normal skill, including its references. For clients without local skill files, MCP initialization points to get_agent_build_skill; the core instructions are available through tools. Installing a skill does not authenticate a client or widen the MCP grant.

API contracts available now¶

Operation	REST	Scope	Effect
`get_agent_build_skill`	`GET /api/v1/management/authoring/skill?document=SKILL.md`	`agents:read`	Bounded documentation read
`get_scenario_test_plan`	`POST /api/v1/management/authoring/test-plan`	`agents:read`	Pure planning; no execution

MCP calls use /api/mcp/authoring; A2A uses the same operation names in a management DataPart. Internal generation/session tokens retain their existing target boundaries. Preferred external authentication is the existing resource-bound OAuth/SSO flow. The management OpenAPI describes both REST routes.

Example test-plan request:

{
  "components": [
    {
      "key": "producer",
      "kind": "voice",
      "features": [
        "gpt_live", "protected_data", "tool_lookup", "auth_link",
        "human_supervision", "multimodal", "branding"
      ]
    },
    {
      "key": "escalation",
      "kind": "workflow",
      "features": ["human_supervision"]
    },
    { "key": "specialist", "kind": "supervisor" },
    {
      "key": "beneficiary-assist",
      "kind": "copilot",
      "features": ["protected_data", "tool_lookup"]
    }
  ],
  "latency_targets": [
    {
      "component": "beneficiary-assist",
      "metric": "copilot_card_painted",
      "target_ms": 1000
    },
    {
      "component": "specialist",
      "metric": "supervisor_first_text",
      "target_ms": 1000
    }
  ]
}

The response includes stable component/case IDs, layer, trigger, assertions, required evidence, and status:"not_run". For the Copilot target it specifies acoustic_speech_end → correct_card_browser_paint, separate cold/warm cohorts and n/p50/p95/max/failed/missing. It neither promises a one-second result nor accepts a caller-provided pass claim. Duplicate component/metric keys, out-of-plan references, incompatible metrics and unknown fields are rejected.

The plan derives tests from declarations, not saved configuration. The authoring model must tailor exact expected facts to the source; a future readiness service must derive actual enabled behavior from pinned artifacts and join real evidence.

Reusable process and prompts¶

Use the skill references for the detailed working instructions:

Extract actors, business tasks, trust boundaries, facts/qualifications, confirmations/corrections, latency boundaries and alternative endings.
Map each requirement to a native owner and concrete artifact. Keep Copilot coaching, customer-facing voice, supervisor chat and asynchronous escalation distinct. Scope one-target generation separately from a requested project.
Discover current templates, schemas, capabilities, integrations and components. Select current compatible models empirically; no hardcoded “fastest model” recommendation belongs in this skill.
Build executable business fixtures when the task is a demo. Keep native authentication real; enforce record authorization in tools. Bind exact facts directly to cards rather than asking a model to reproduce arithmetic.
Author backend business rules and voice delivery separately. Preserve confirmed assumptions; answer the latest question and return to the original goal. Require an audible recap with actual pending/rejected/submitted state.
Save with native revision checks and collab refresh; re-read effective state.
Run relevant fixture, simulation, Preview auth, workflow, browser and channel checks. Inspect durable events independently of an LLM judge. Fix the first incorrect transition and rerun affected cases.
Handoff actual drafts, sample inputs, human actions, revision-pinned evidence, measured boundaries and untested connections.

For prompt examples see the skill's interaction patterns. For the real consent, summary, collaborative-state and teardown failures see Allianz lessons. Runtime-specific lessons remain conditional; one insurance script or end-tool sequence must not govern every model/channel.

Where the substantive logic should live next¶

Extend existing services and their scoped adapters. Do not create another generation database, parallel workflow engine or general-purpose “execute code” MCP endpoint.

Capability	Existing implementation to reuse	Extension needed
Source intake	`studio/design-sources.ts`, existing attachment readers	Bounded DOCX/PDF extraction with paragraph/table provenance, source hash, size limits and tenant ownership; current session reader is text/image
Scenario/project specification	`studio/live-schema.ts`, `demo-specification.ts`, `demo-store.ts`	External revisioned project/specification API; requirement-to-artifact/test links and unresolved decisions
Materialize linked drafts	`studio/demo-build.ts`, `design-build.ts`, `management/drafts.ts`	Principal-based entry point for the same approved component membership; dry-run diff and resumable operation receipts
Native capability composition	Capability registry, expander, schema/logic validators	Checked prepare/apply operation returning concrete wiring diff, prerequisites and provenance
Business tools/fixtures	`generation-mcp/mock-tools.ts`, native tool/capability tests	External scoped fixture preparation/checks; stateful fixtures through supported state/workflow contracts
Brand/surface/components	Native surface/branding configuration, Catalog Studio, catalog-builder	Tenant-scoped asset/catalog draft APIs, typed props/actions, compilation/render checks; publication separate
Supervisor assistance	Native Human Supervision, tenant Salesforce action, related-thread resolver and chat injection	Reusable wiring and two-path tests; no custom review desk or legacy Human Review node for ordinary guidance
Saved tests and runs	`design-build.ts`, `performance/simulation/`, durable evaluation workers	Revision-pinned create/run/status/cancel/evidence APIs with bounded runtime/cost and explicit delivery mode
Audio/auth/browser probes	Allianz Preview scripts and native Preview/auth services	Managed runner adapters instead of customer-machine gRPC/browser scripts; correct playout waits and durable cleanup
Evidence and latency	Durable conversation/workflow events and simulation evidence	Server-derived readiness and aligned audio/card/browser timing; revision coverage and missing evidence visible

Proposed external operations¶

These names and routes are proposed, not advertised tools:

MCP operation	Proposed REST route	Contract
`prepare_scenario_build`	`POST /authoring/builds/prepare`	Validates source/spec refs; resolves dependencies; returns immutable plan revision, artifact diff, gaps and required scopes
`apply_scenario_build`	`POST /authoring/builds/{id}/apply`	Exact prepared revision + expected target revisions + idempotency key; creates only authorized members/drafts; returns durable job
`get_scenario_build`	`GET /authoring/builds/{id}`	Artifact IDs/revisions, phase, per-step receipt, issues and remaining work
`prepare_scenario_assets`	`POST /authoring/builds/{id}/assets`	Brand sources and catalog props/actions mapped to real asset drafts; validation/render evidence; no implicit publication
`run_scenario_checks`	`POST /authoring/builds/{id}/checks`	Pins artifact/test revisions; selected suites with approved fixtures/channel, delivery mode, duration and spend limits
`get_scenario_evidence`	`GET /authoring/builds/{id}/evidence`	Bounded redacted server-owned facts, failures, latency distributions, coverage and source links

All proposed REST paths are relative to /api/v1/management. Reuse existing read/write/debug/run permissions where they fit; add narrowly scoped catalog or test-execution authority only where the existing permission model requires it. agents:read test planning is not authority to launch paid/live tests.

Prepared plans need a schema version, source hashes, component keys, dependency graph, concrete patches, expected revisions, required integrations, simulation labels and a requirements/test map. Resolve stable member IDs before wiring. A changed plan needs a new revision; stale apply fails before mutation.

Use transactions for database-only changes. For catalog compilation, model calls and external actions, use durable jobs with phase receipts, idempotency and bounded retries. Report partial completion and ambiguous remote outcomes. One database transaction cannot roll back a remote publication or customer message. Refresh collab only after committed draft changes.

Readiness must fetch evidence by authorized run IDs and compare its artifact, test and runtime fingerprints with the build. It must not accept passed:true, “approved” chat text, or model-supplied latency summaries as proof. Persist not_run, failed, cancelled, stale and inconclusive separately from passed. Tests across revisions are not one full-suite pass on the final build.

Delivery order and acceptance¶

This foundation: portable skill, shared planning service, discovery parity, MCP/A2A/REST contracts, bounded reads and tests.
External build parity: wrap project/draft/fixture/asset services. Accept when an external client can assemble both scenarios with real draft links, preserve concurrent edits, and resume after interruption without duplicate artifacts or premature publication.
Managed validation: saved-case execution, actual human review and Preview auth/browser/audio adapters. Accept when the original failure cases are detected, evidence survives teardown, and timing traces to actual events.
Client qualification: exercise OAuth, discovery, skill loading, read-only grants, stale saves, job resumption and evidence retrieval in each supported Codex and Claude workplace client. Unit tests are not client certification.

Heavy operations stay server-side. A workplace client should need the management endpoint and SSO, not a shell on the city host, database/provider credentials or local audio scripts. Hosted clients also need network reachability; access from a developer container does not prove a hosted connector can reach the endpoint.

Codex's official MCP documentation describes Streamable HTTP, OAuth and server initialization instructions; this implementation uses those existing integration points. Reference checked September 29, 2026: https://developers.openai.com/codex/mcp. For Claude workplace clients, qualify the exact client and organization connector settings during phase 4. This change does not claim a new Claude connector installation or a live client test.

September 29 supervision correction¶

The Allianz follow-up exposed an incorrect generated pattern: a hidden legacy Human Review node and separate review desk. The maintained skill now teaches single-action asynchronous Salesforce escalation and a separate connected-chat response. Its planner distinguishes approval from completed business action, checks the runtime-bound related call and tests caller-role claims. A separate transaction-approval workflow remains conditional on the requested business process; Human Supervision does not imply one. The generation validator also accepts native START/END graph boundaries while rejecting dangling/reversed edges. Regenerate the skill bundle after changing these references.

Verification¶

Run the skill validator, generation-docs build, scenario-authoring tests, management registry/scope/OpenAPI checks and runner protocol tests. The planner tests both a complex Allianz-shaped brief and a small public chat that must not acquire unrelated authentication/supervision work.

An independent offline skill exercise used an external-client request with authoring/discovery tools but no project, asset-write or test-run tools. It produced a bounded build/test handoff without invented native results. The review led to explicit guidance for external completion, failed note saves, unavailable fast coaching paths and post-call approval policy. That exercise evaluates skill behavior; it is not a runtime agent rehearsal.

September 29 validation: 21 focused TypeScript tests and 16 runner tests passed; the skill validator, AI Manager typecheck, touched-file ESLint and route-auth audit passed. The API-spec build succeeded and its management output contains both new routes. Existing FastAPI lifecycle and SDK-reference warnings remain. Authenticated Codex/Claude client qualification and generated-agent voice rehearsals were not run for this foundation change.