Scripted subagents¶

Status: Built end to end (2026-09-02) — runtime (injection route, aux ladder, controller + script STT lane, step machine, scripted transition_to, deep pause on Nova + OpenAI Realtime, engine-tagged transcripts, voice→surface reflection), the step-card editor (incl. the per-step component picker + answer mapping) + publish validation, the aux authoring surfaces (stage-inspector Auxiliary services section + tenant Settings page), the SLM matcher tier + eval harness, cost recording, and the preflight guided gate. Remaining: live E2E verification on vanguard (the checklist at the end) | Last reviewed: 2026-09-24

The design for scripted subagents: subagents whose engine is a deterministic step machine — STT + TTS + authored logic, pattern matching, or a small language model — instead of the primary (usually S2S) model. For flows that are a predictable, required set of steps (collect a claim number, confirm an address, walk a returns checklist), a scripted subagent runs at a fraction of the S2S cost, behaves deterministically, and hands back to the smart agent the moment the flow completes or the caller falls off the rails. It builds directly on the unified subagent model (agent-logic.md), the payment-capture handoff machinery (payments.md), and the auxiliary-service derivation that machinery already performs. Originally a design doc; everything below is built (2026-09-02) except where a section explicitly says otherwise — the per-section Built markers record what shipped and which deliberate deviations were made.

Three directional decisions were made 2026-09-02:

Deep pause is a launch requirement. Gating the S2S provider while feeding it silence (the payment-capture approach) keeps the provider billing — Nova Sonic charges for silence audio tokens like any other audio. A scripted flow of minutes must stop sending data to the provider entirely (suspend the stream) and resume at handback.
Multimodal capture is first-class, not a bolt-on. A step can render an A2UI component and gather its inputs from DTMF, speech, or the form — any mix — with all surfaces kept in sync.
Auxiliary tech (script STT, script/handoff TTS, SLM) resolves stack-derived → per-agent override → tenant-level default cascade.

Why¶

Cost. An S2S model bills per audio token in both directions for the whole session. Transcribe + Polly + a pattern matcher (or one Nova Micro completion per utterance) is one to two orders of magnitude cheaper for the minutes a predictable flow occupies — and those flows are often the longest part of a call.
Determinism. A required sequence ("read the disclosure, capture consent, confirm the date") should not depend on a frontier model's mood. Scripted steps execute; they don't improvise.
The machinery mostly exists. The payment handoff already pauses S2S, drives the call with private scripted TTS, captures DTMF, and hands back with a context summary — and SecureFieldCaptureController proves the controller subclasses cleanly. Topics already provide entry/exit, routing, and the current_topic variable. Rules already provide a deterministic action vocabulary. The step machine is a composition, not an invention.

Concept: a topic with a different engine¶

A scripted subagent is an ordinary entry in config.topics.topics[] with an engine discriminator. The two storage keys are unchanged (the standing subagent-unification decision) and everything about entry/exit is inherited:

// config.pipeline[N].config.topics.topics[]
{
  "id": "collect_claim",
  "label": "Collect claim details",
  "description": "…",                    // rides the transition_topic tool schema, unchanged
  "entry_when": "…", "return_when": "…", // unchanged
  "control": "refer",                     // refer | transfer, unchanged semantics
  "engine": "scripted",                   // NEW — default "llm" (today's behavior, absent = llm)
  "script": {                             // NEW — scripted engine only
    "entry_step": "greet",
    "defaults": { "matcher": "patterns", "retries": 2 },
    "escape": { "phrases": ["agent", "operator"], "dtmf": "*", "on_exhausted": "return" },
    "handback": { "summary_template": "Collected claim {{claim_number}} for {{incident_date}}." },
    "steps": [ /* below */ ]
  }
}

Inherited for free: the model routes in via the enumerated transition_topic tool (never a free-text id); rules can transition deterministically; entry_when gates entry (fail closed); __return__ pops; refer pushes return history, transfer clears it; current_topic remains the session variable #when sections and variable_changed rules compose with. The one divergence: entering a scripted topic does not perform a prompt takeover — transition_to's scripted branch arms the GuidedTopicController (below) with a StepMachine runner and deep-pauses the primary model instead; the tool result carries a stand-down instruction, a refused script leaves NO topic state, and the handback pops the topic history via TopicManager.on_script_ended before the provider resumes.

The step model¶

{
  "id": "ask_incident_date",
  "say": {
    "text": "What date did the incident occur?",     // prompt-token text, PromptResolver-resolved
    "component": {                                    // optional — multimodal, below
      "catalog": "standard", "name": "DateField",
      "props": { "label": "Incident date", "value": "{{incident_date}}" },
      "requires_surface": false                       // false = voice fallback when no surface attached
    }
  },
  "listen": {
    "inputs": ["speech", "dtmf", "form"],             // any subset
    "capture": { "variable": "incident_date", "kind": "entity", "entity": "date" },
    "matcher": "slm",                                  // logic | patterns | slm (script default otherwise)
    "retry": { "max": 2, "reprompt": "Sorry — what date was that?" }
  },
  "do": [ /* the existing rule-action vocabulary: run_tool, set_variable, transition, escalate, end_session */ ],
  "branch": [ { "when": "incident_date != null", "to": "ask_location" } ]  // else → next step in order
}

No third language. Every field reuses an existing primitive: say.text is prompt text with the same tokens and pills, resolved by PromptResolver with live variables; branch.when and logic matchers are trueserve_logic expressions (both implementations, fixtures-first — the standing drift-guard discipline); do is exactly the LogicRulesProcessor action vocabulary, executed through the same registered (gated, healed) tool handlers. The step machine is authored rules with a program counter.

Built (2026-09-02): the executor is step_machine.py (StepMachine, implementing the controller's ScriptRunner protocol — tests/test_step_machine.py); the engine/script shapes are typed in the generated config schema (the GuidedScript def in generate-config-schemas.ts, every field described) and their collab round-trip is pinned in BOTH pipeline-shape test copies. do actions execute through the LIVE rules processor's own _execute_action (never a second executor); timeouts and no-matches share one per-step retry budget (the payments rule); an authored step cycle ends in the escape ladder, not a busy loop; escape phrases/DTMF hand back with the captured variables named. The slm matcher tier (slm_matcher.py) and the say.component/form input path are built too — see Multimodal capture below.

Matcher tiers, cheapest first, per-step with a script default:

Tier	Mechanism	Cost	Use
`logic`	`trueserve_logic` expression over the raw capture (DTMF string, transcript)	zero	menu digits, yes/no keywords
`patterns`	authored keyword/regex lists per choice, compiled offline	zero	intent choice among enumerated options
`slm`	ONE constrained-output completion per utterance against the stack-derived small model	~10⁻² of an S2S turn	entity extraction (dates, addresses), fuzzy intent

The SLM call enumerates the step's declared choice ids / entity schema in its prompt and demands strict JSON — a matcher must never be asked to invent an id (the consult_agent_id lesson).

Runtime: the third controller¶

GuidedTopicController (guided_topic_controller.py) joins PaymentModeController and SecureFieldCaptureController as the third sibling on the same substrate — the substrate landed 2026-09-02 (controller subclass + builder glue guided_topics.py + placement at every capture-controller site, pinned by tests/test_guided_topic_controller.py); the step machine that drives it is M4. The step semantics live in a pluggable ScriptRunner protocol the controller wraps fault-tolerantly (a raising runner ends the script through the handback path, never a teardown). What is reused verbatim vs. new:

Piece	Status
Input gate	not needed (built finding) — a guided flow is not secret, so caller audio is never silenced or VAD-proxied; deep pause (M4) is what stops the provider hearing it, and during a live payment/secure capture the capture controllers sit UPSTREAM and starve the script automatically (the three-controller composition rule)
Output gate (drop model output while active, pass `TTSSpeakFrame`)	built (2026-09-02) — `PaymentS2SOutputGate` was already controller-generic; the S2S chain binds a third instance to the guided controller
Private TTS lane	built (2026-09-02) — `handoff_tts_stage_for_owner` keeps its signature/reuse rule, its per-provider voice-continuity table moved from code to data behind the aux resolution ladder (byte-identical fallback, pinned by `tests/test_aux_resolution.py`), and the S2S chain shares ONE lane across all three controllers (guided derives it from its own owner stage only when payments/secure didn't already)
Handback (`LLMMessagesAppendFrame(run_llm=True)` via `S2SUserAggregator`)	built (2026-09-02) — the inherited `_exit` path, blank `transaction_id` no-ops the reporters; the structured `handback.summary_template` rendering is M4
Timers, retry budgets, DTMF barge-in, prompt-suppression	built (2026-09-02) — inherited from the controller base (the secure-field precedent); timeouts route to the runner's escape ladder
STT in the private lane	built (2026-09-02) — guided_script_stt_lane.py: a `ParallelPipeline` tee (the `channel_split_stt` precedent) in the input chain AFTER the payment/secure speech gates (spoken card digits can never become a script transcript), whose `ScriptAudioTap` starves the STT of audio while no script runs (streamed STT bills per audio second) and whose `ScriptTranscriptGate` drops in-flight finals after the script ends (a late final must not become a model turn). `script_stt_stage_for_owner` consumes the ladder's `script_stt` slot; reuse-if-present when the pipeline is already cascaded
Step executor + matcher tiers	new (small — see above); M4, implementing the controller's `ScriptRunner` protocol
External-write dispatch (M1 seam)	built (2026-09-02) — `register_guided_capability_handler` routes a trusted `guided_topics` injection (`{kind: 'variable_write', name, value}`) to a live script's `on_external_variable`; dormant until M6's form sync publishes it
External-write completion (form fills a pending capture)	built (2026-09-02) — the capability-handler route, see Multimodal

Ordering constraints carry over verbatim: gate immediately after transport input, controller before the user aggregator, one active controller per session, parallel pipelines fail closed at publish. On a cascaded (non-S2S) agent the controller simply gates the LLM turn and reuses the authored STT/TTS — cheaper still, and no pause problem exists (per-turn HTTP bills nothing while unused).

Deep pause (required) — built (2026-09-02)¶

While a scripted topic is active, the primary provider must receive no data. The dispatch seam is deep_pause.py (deep_pause_provider/deep_resume_provider over the stashed provider service); the per-provider mechanics are tsf_deep_pause()/tsf_deep_resume() on the wrappers (tests/test_deep_pause.py). Layering: the guided output gate is the correctness layer (model output suppressed from the moment the script arms) and the pause is the cost layer, landing on a ~1 s grace after entry so the transition_topic tool result finishes its round trip.

Provider	Suspend	Resume
Nova Sonic	`tsf_deep_pause()`: SAVE the `LLMContext` reference, then `await self._disconnect()` — the stock method does the whole suspend (contentEnd→promptEnd→sessionEnd close sequence, receive task cancelled under `_disconnecting`, rotation monitor stopped, pre-created next session cleaned). The context save is load-bearing: `_disconnect()` CLEARS `self._context`, so an unsaved pause makes every resume a silent no-op. A context update arriving mid-pause is ADOPTED for the resume, never connected on	`tsf_deep_resume()`: `_start_connecting()` + `_handle_context(saved)` — the full-context replay whose LAST user message is sent interactive. That is the handback trigger: the summary was appended BEFORE the resume (the S2SUserAggregator writes appends into the saved context object even with no live session), so the replay delivers it and the model responds to it. Resuming FIRST would have Nova re-answer the pre-script last user turn
OpenAI Realtime	`tsf_deep_pause()`: flag set BEFORE `_disconnect()` so the receive loop's exit path (and any in-flight reconnect loop) reads a deliberate pause, never a drop — pinned in `test_openai_realtime_reconnect.py`	`tsf_deep_resume(run_llm=True)`: arms `_run_llm_when_api_session_ready`, then the stock `reset_conversation()` — the session-ready handler re-sends the whole context (summary included) and creates the response
Google ADK Live / Deepgram Voice Agent	no pause support: one warning, the script runs gated-alive (correct but billed) — matching the publish-time "guided unsupported on this provider" warning	—
Cascaded LLM	nothing to suspend (no stashed provider → dispatch answers False, gated-alive is free per-turn HTTP)	nothing

Resume is masked, not raced. Resume starts when the script reaches a terminal step — the handback summary's TTS playout (seconds, in the agent's own voice) covers the reconnect. The controller's end path is ordered: summary append → topic-history pop (so the refresher has swapped back before the model speaks) → resume, and a fast script (shorter than the pause grace) cancels the pending pause instead of paying a pointless pause/resume cycle. An escape mid-script ("agent", *) takes the same masked path.
Resume failure is the profile ladder, never silence: shared backoff → tier-B endpoint swap → the mid-call failover ladder's ending (FatalErrorFrame + breaker record), with the caller told — a scripted flow must not strand a call the smart agent can no longer rejoin.
Recording survives the pause. The standing invariant — the S2S service re-pushes caller InputAudioRawFrame downstream or the recording loses the caller track — holds through a _disconnect()-style pause with no extra work (spike-verified): the service object stays in the chain and its process_frame pushes every frame after handling, connected or not, so the recorder keeps the real caller track. Scripted TTS output already reaches the recorder via the private lane, and the guided controller (unlike the capture controllers) never zeroes audio.
Idle machinery is suspended while the controller is active (it owns its own field timeouts), resumed at handback — the same suspend/resume discipline as tool-call idle suspension, with the same bounded cap.
Context integrity: the persistent LLMContext is untouched during the pause; the handback summary is the only thing the primary model learns about the scripted interval (plus the captured variables in the store). refer/transfer history semantics apply to it exactly as to LLM subagents.

Cost accounting — built (2026-09-02), one deliberate deviation¶

Immutable quantities, never money (the standing cost invariant), with the shapes falling out of existing seams rather than new meters:

Script STT + TTS lanes are real services created through create_service, so the usage shim attaches automatically and their seconds/characters land in cost_usage_events as ordinary STT/TTS rows (stage ids __script_stt / __payment_handoff_tts) — no bespoke meters needed.
SLM matcher tokens are written directly to the shared costing ledger by slm_matcher.py (events_from_usage/persist_usage_events, product llm, stage __script_slm, fire-and-forget off the latency path) — their OWN provider/model rows, never folded into the primary provider's billing product.
The pause needs no meter — the saving is automatic: a deep-paused provider reports NO usage (the stream is closed), so observed S2S cost already reflects the suspension; inventing "paused seconds" rows would put quantities in the ledger no provider reported. The pause WINDOW rides the provider's Service Usage Summary span as an attribute (guided.deep_pause_seconds, accumulated) — the deviation from the original "pause start/end as meters" sketch, deliberate.

Pricing for the aux-lane rows rides ai-manager's existing price books (they are ordinary provider/product/model rows); the estimator's scripted-vs-prompted delta remains follow-up polish.

Multimodal capture: variables are the sync¶

The step machine treats the session variable store as the single source of truth, which makes multi-surface sync a routing problem rather than a state-machine problem. Built (2026-09-02) except the last bullet:

Every input path writes captures through the SessionVariableStore: speech/DTMF via the controller's matcher, form submits via the existing A2UI action paths (gRPC + Redis — both, the recurring two-path trap) through sanitize_ui_action_payload and the action_capture machinery.
The glue subscribes ONE store listener at script start (_subscribe_variable_writes in guided_topics.py, lazy + once, inert unless a script is live); any write — whichever surface produced it — reaches StepMachine.on_variable, which advances a step whose pending capture it satisfies. A form submit that fills several fields fast-forwards field by field as each write lands.
A step's say.component renders through the REGISTERED send_a2ui_json_to_client tool (StepMachine._render_component → invoke_registered_tool — the same gated, consent-checked, shape-normalized path a model call takes; a render failure or missing surface degrades to voice, never breaks the step).
Voice→surface reflection (built): a VOICE/DTMF answer re-renders the step's component with the freshly-captured value (props re-resolve against live variables — {{appointment_date}} shows what the caller said). Only that path re-renders: a FORM answer came from the surface, which already shows it. No SLM is involved beyond what the matcher already did — the value is structured by the time it reflects; reflection is just a re-render.

The prerequisite payments gap is CLOSED (2026-09-02) and the seam is generic. ExternalInjectionProcessor.register_capability_handler gives a registered controller first claim on a trusted capability injection (consumed = the controller tells the model itself; declined/fault = ordinary LLM dispatch), and register_payments_capability_handler routes a payment_outcome event to a live keypad capture's complete_externally — the fix that ended the burn-timeouts-then-announce-cancellation failure (payments.md — the capability-handler route). The guided controller registers on the SAME seam for its external variable writes.

Rules that carry over from the payments experience:

Consent is ONE shared fact. Rendering a component requires an attached surface via the existing multimodal handshake; a completed texted verification is consent; never re-ask, never send before it. A step with requires_surface: false degrades to voice-only; requires_surface: true with no surface skips the component and voice-prompts (entry can be guarded with entry_when if the whole flow demands a surface).
Per-surface DTMF transport fragmentation — keypad capture is functional on three surfaces and inert on Genesys AudioHook/Five9 with no error anywhere. Publish validation mirrors payments' warnings: a dtmf input on a transport with no DTMF ingestion, a form input with no multimodal surface configured.
Sensitive fields are out of scope by design. A scripted step must not collect PAN/IBAN/SSN-class data — that is the payments and Secure Data Capture capabilities' job (vault, masking, SensitiveValue discipline). Publish validation warns when a capture variable name/entity smells sensitive; the capabilities remain the sanctioned route (a scripted step's do can invoke collect_payment — the controllers compose by handoff, never by nesting).
A2UI rendering rides the existing dispatch paths with shape normalization and catalog-ownership rules unchanged; scripted turns publish through the _publish_transcript chokepoint tagged with their engine, and synthetic-turn suppression applies as today.

Auxiliary service resolution — built (2026-09-02)¶

The generalization of what payments already does invisibly. Three named slots per owning stage — handoff_tts (shared by payments, secure capture, and scripts), script_stt, script_slm — resolved by a ladder in aux_resolution.py (resolve_aux_stage), first hit wins:

Explicit stage config.aux.<slot> override — authored, applied as a sparse overlay over whatever the lower tiers derive (provider/service/integration_id replace; config keys merge over), so pinning one field never loses the incident-earned defaults that came with the derived entry. This tier subsumes the earlier design's separate stack.author_overrides tier.
The stack manifest's aux section for the family (shared/service-definitions/stacks/<family>.json — each currently pins a script_slm: Nova Micro / gpt-4o-mini / Gemini Flash-Lite)
Tenant default — tenant_settings key aux_services, edited via PUT /api/settings/aux-services (lib/aux-settings.ts, admin-only Tenant Settings surface; no secrets in this key), read by the runtimes through the shared libs/aux-services resolver (make_sync_aux_resolver/make_async_aux_resolver, ~60 s TTL — the libs/knowledge pattern)
The built-in per-provider table — shared/service-definitions/aux-services.json, the former _payment_handoff_tts_config if/elif chain migrated verbatim, plus the new script_stt/script_slm tables

Tiers 2–4 share one small token vocabulary resolved against the owner stage at build time ($owner, $owner_voice, $rate, $owner_provider, $alias, requires_owner — documented in the JSON's $comment), so a tenant/manifest entry gets the same owner-voice continuity the built-ins have. Every failure degrades to the next tier, and an exhausted ladder returns None for the caller to report — a bad aux default never fails a build harder than the pre-ladder code did.

Two disciplines, both now enforced:

The tables are data, living beside the service definitions and stack manifests, CI-validated against SERVICE_LOOKUP_V2 and SUPPORTED_PROVIDERS (test_every_aux_row_resolves_in_service_lookup_v2 in test_aux_resolution.py — a manifest naming an unregistered (provider, type, service) fails there, not the first caller; the seven-place-registration lesson). One consuming implementation: the builder, at build time, with the payments-style provenance log line ([Aux] Resolved <slot> … (source=…)). ai-manager calls the runtime resolver to display concrete defaults; it does not maintain a TypeScript resolver.
Voice continuity is a hard constraint (the neural vs generative Matthew lesson): the handoff_tts entries pin voices matched to the family's S2S voices, and the byte-identity of the migrated table is pinned test-by-test against a verbatim replica of the deleted chain.

The stage inspector's Properties → Auxiliary Services embeds ServiceConfigForm for all four slots (TTS, STT, guided matcher, logic checker), with fixed stage types and no nested HTML forms. POST /api/aux-services/resolve requires ManageAgents, reads defaults for the authenticated tenant, and delegates to the orchestrator's read-only /aux-services/resolve endpoint. That endpoint uses resolve_aux_stage with server-supplied tenant defaults; it performs no model/tool execution. Owner tokens resolve to actual values. The BFF filters default integration IDs to active profiles in the current tenant or platform scope, even for platform admins. Execution applies the same scope, capability and explicit-default/single-profile rule through aux_integrations.py; an ambiguous or unavailable profile leaves the slot unresolved instead of falling through to an unscoped database credential. Failure shows a retry state rather than pretending built-in values are the tenant defaults.

The editor displays Default selected and writes sparse config.aux.<slot> overlays. Changing provider/service drops incompatible inherited config and credentials; a complete override can also create a previously unresolved slot. The regular provider lookup and template/clone auto-selection are tenant scoped, and browser option caches include user, tenant, provider, and capability. Tenant-wide authoring remains Settings → Tenant Settings → Auxiliary Services and its existing admin-only API.

config.aux.script_slm.config.prompt_overrides accepts choice and entity instruction text ({entity} is substituted by the matcher). logic_slm.config.prompt_overrides accepts adherence and extraction. aux_prompts.py supplies the same defaults to execution and preview. Overrides merge per prompt, preserving untouched tenant/stack defaults. The runtime always supplies dynamic choice/evidence/utterance content and JSON schema instructions, and rejects invented choice IDs. AI judge criteria stay per condition. SLM model selection is restricted to the supported Bedrock/OpenAI/Gemini completion services; selected model, temperature, token limit, top-p, and explicit integration credentials are honored. A missing explicitly selected API key never falls through to an unrelated database default.

Authoring UI — built (2026-09-02)¶

Same tab strip (Main | subagents | + in SubagentTabs.tsx). The "+" is a two-option menu — Smart subagent (prompt) or Guided flow (steps, seeded with escape defaults + a first step); the tab and panel meta bar show a teal Guided badge, and the panel hides Return when for guided topics (a flow hands back by itself — dead config otherwise). Quick-create from an @name token stays Smart. Guided topics also refuse the topic↔delegate conversion (topicToDelegate throws — there is no prompt to convert).
Selecting a guided tab swaps the editor body from PromptTemplateEditor to GuidedStepsEditor: vertical step cards (say / wait-for-a-reply / branch pills, up-down reorder), with the full-prompt header, actions strip, and injected chips deliberately ABSENT — a guided flow has no prompt takeover, so that chrome would be fiction. Deliberately not a graph canvas — a flow that genuinely needs a graph is a workflow.
Each card reuses existing chrome: say-text and the handback template are AgentPromptEditor in miniature (variable pills, live {{var}}); branch/accept conditions use the shared WhenConditionBuilder + NewVariableDialog (writing through the page's config path, the standing rule).
Script-level settings (escape phrases/key/on-exhausted, retry budget, listen timeout, handback) are one collapsible Flow settings row at the top of the list.
Publish validation (agent-logic-validation.ts): structural script breaks BLOCK (no steps, duplicate/missing step ids, unknown branch/entry targets, choice captures with no choices, unparseable expressions, junk on_exhausted); tolerated gaps WARN (capture without a variable, form input with Multimodal UI off, sensitive-smelling capture names → pointed at Payments/Secure Data Capture, return_when on a guided flow, a script on an llm topic).
The preflight guided gate (_apply_guided_gate in preflight.py) resolves the aux lanes without connecting: unresolvable required lanes (script STT / private TTS on an S2S owner) disable the test buttons; unsupported deep-pause providers warn gated-alive; parallel pipelines fail closed.
Preview works unchanged (same pipeline, same transports); a later increment can highlight the current step in the tester via the debug event stream. User guide: Guided flows.

Failure policy¶

Situation	Behavior	Rationale
Matcher fault (SLM unreachable, expression error)	Fail open per attempt: count as no-match, consume a retry, follow the escape ladder	Never trap a caller — the smart agent is the safety net
Escape ladder exhausted	`on_exhausted`: `return` (default — hand back with a "caller stuck at step X" summary) / `transfer:<topic>` / `end`	Authored, but the default is the smart agent
Deep-pause resume failure	Backoff → tier-B swap → failover-ladder ending, caller informed	Never a silent hang (the OpenAI Realtime lesson)
`requires_surface` step, no surface	Skip component, voice-prompt; `entry_when` guards flows that demand a surface	Degrade, don't refuse
Scripted topic on a parallel pipeline	Publish fails closed	Same input-chain limitation as payments/rules
Sensitive-smelling capture variable	Publish warning naming the payments/secure-capture route	Compliance posture is never invented by default

Phasing¶

Phase	Scope	Notes
0	DONE 2026-09-02. The generalized controller substrate (`GuidedTopicController` + builder glue + placement pins); the private-lane STT derivation + gated tee; the aux tables migrated to data (the ladder, the JSON table, the tenant key + `libs/aux-services`, the ai-manager settings route, the drift test); the capability-sourced-injection → controller route	Touches payments/secure-capture shared machinery — their regression suites gate it; the injection route fixed the live payments voice↔form gap
1	DONE 2026-09-02: `engine` discriminator (+ generated `GuidedScript` schema, collab pins in both shape-test copies), step machine (`logic`/`patterns` tiers), scripted `transition_to` (no takeover — arms the controller; entry gate/refer/transfer/current_topic unchanged; handback pops via `on_script_ended`), deep pause (Nova + OpenAI Realtime), escape hatches, engine-tagged transcripts (`source="guided_script"`, `metadata.engine="scripted"`), the step-card editor + Smart/Guided create picker, cost recording (see Cost accounting)	Voice-only scripted flows usable end to end
2	DONE 2026-09-02 except one item: `slm` tier (slm_matcher.py + the live eval harness `tests/test_slm_eval.py`, `SLM_EVAL=1`), multimodal capture (component render through the registered A2UI tool, form sync + fast-forward via the store subscription), publish + preflight validation. Open: the voice→surface `ui_state` reflection	Launch scope is the end of phase 2 — multimodal is first-class, not a follow-up
3	Aux-slot authoring UI in the stage inspector + tenant `aux_services` settings PAGE (the API route exists) + scripted-script library sharing via capability packs	The authoring polish and reuse story — not started

Implementation checklist (the incident-earned traps): new nested topic keys must survive both Y.Doc shape converters + the round-trip test; the engine/script shapes go through generate-config-schemas.ts as a discriminated union with every field described (drift CI); preflight gains a buildability arm that a scripted topic's STT/TTS/SLM lanes resolve; the transition_topic schema is unchanged; new meters follow the cost-events idempotency discipline.

E2E verification checklist (vanguard) — the remaining launch gate¶

Author a 3-step Guided flow (greet → capture appointment_date → confirm) on a Nova S2S agent via the "+ → Guided flow" picker; publish (validation green).
Preview voice call → say the routing phrase → verify in orchestrator logs: [Topics] guided script armed, the private Transcribe/Polly lanes created, [SafeAWSNovaSonic] deep pause ON ~1 s after entry.
Answer the date question → confirm → verify the handback: closing line in the agent's own voice, deep pause OFF (resumed=yes), the model speaks the summary content, current_topic popped, #when sections see the captured variable.
Escape test: say "agent" mid-flow → immediate handback with the stuck-at-step summary; the model resumes without re-greeting.
Repeat 2–4 on an OpenAI Realtime agent.
Multimodal: emulator with an attached surface → a say.component step renders; submit the form → the flow fast-forwards; answer the NEXT component step by voice → the component re-renders showing the spoken value (voice→surface reflection).
Cost: cost_usage_events shows __script_stt/__payment_handoff_tts rows and (for an slm step) __script_slm rows; the S2S usage gap matches the pause window on the Service Usage Summary span (guided.deep_pause_seconds).

Open questions¶

Entity extraction depth — dates/phones via the SLM tier only (built), or dedicated NLU service definitions as a fourth tier?
Scripted-first agents — can a session start in a scripted topic (IVR-replacement shape) and escalate to the LLM, inverting today's direction? The mechanism supports it; the wizard story doesn't yet (user decision: topic-routed only in v1).
Google ADK Live / Deepgram Voice Agent deep-pause semantics — both run gated-alive today (warned at preflight); stream suspend/resume needs per-provider investigation.
Step-level analytics — funnel events (entered/completed/escaped per step) as conversation_events, for the "where do callers fall off" question.
Interaction with mid-call tier-C migration — a migration during an active script should be deferred until handback (the script doesn't need the primary provider at all); confirm the gate ordering.
Phase 3 polish, remaining — script library sharing via capability packs; current-step highlighting in the Preview tester (the aux authoring UI and tenant settings page are built).