A2UI rendering

Status: Current | Last reviewed: 2026-10-15

Representative catalogue buttons can send screens into a linked customer conversation through the shared customer-components API. The API records both histories, validates replies and emits a Copilot application event. Conversation renders the saved screen data as expandable read-only snapshots.

Composed Copilot views

layout.region: "context" marks data-only Copilot output. Surface partitioning excludes it from canvas, rail and workspace. A composed view receives authorized events for the same conversation through A2UISurfaceManager.relatedEvents. The shared renderer folds these into A2UIRenderContext.relatedSurfaces and passes conversationId and readOnly to catalog renderers. Multiple views can consume one extraction without fetching another conversation.

The optional surface flag copilot_composed_layout removes generic workspace and rail card chrome. A catalog owns its layout and representative controls. Shared-view controls remain read-only; server mutations must independently authorize the session. Takeda uses a record-bound Salesforce action route. BroadcastChannel messages contain only an invalidation and conversation ID, never patient data or authority to perform a write.

UI-action ordering and the visual-completion watchdog honor the owning stage's auto_injections.capabilities.multimodality policy in the active subagent. With generic guidance off, the action still reaches the agent, captured variables are updated and duplicate event IDs are ignored, but the runtime does not demand another render. Custom guidance replaces the default ordering and does not arm the generic render retry. Default guidance retains the existing bounded watchdog. Queued retries recheck scope before delivery. This prevents a completed photo attachment from forcing an unnecessary screen update and a second copay question when the authored procedure has already moved on.

An authored multimodality.post_send_instruction replaces the default post-render follow-up in both local callbacks and the frame handler. The runtime does not append generic instructions to render another component to a custom instruction that asks the agent to wait. Surface delivery and validation are unchanged.

Generic A2UI guidance uses the stage's existing auto_injections.capabilities.multimodality settings. main: false omits it from the router while preserving the exact segment for enabled subagents; mode: "custom" replaces it, and mode: "off" removes the guidance. The rendering tool remains registered. Runtime and prompt preview use the same setting. When replacing the generic guidance, retain explicit catalog, root component and action contracts in the relevant specialist prompt. For fixed buttons and capture requirements, use multimodality.component_props to bind the action name, required-photo list and photo limit rather than asking the model to recreate them each time.

How an agent emits structured UI components and a surface renders them in real time.

What A2UI is

Agent-to-UI: a protocol where the agent produces component descriptors (a header, a button, a form, a chart) instead of plain text, and the user's surface renders them inline. Built on the Model Context Protocol — A2UI rides on the same transport as MCP tools.

A2UI is always-on for text_chat agents (no permission prompt). For voice agents, it activates when the surface signals capability via the audio handshake — and on a phone call that handshake means texting the caller a link, so it is gated on their consent (see below).

Experience fragments (AXL-style spike)

The AXL-style spike adds an internal experience fragment layer above A2UI. This is not a Salesforce runtime dependency. It is a small, surface-neutral intent shape that can describe a card, form, choice, custom component intent, data model seed, actions, surface hints, and fallback text before compiling to a concrete surface format.

The first compiler target lives in shared/a2ui-react/src/experience-fragment.ts. It emits the current TrueServe A2UIRenderEvent shape (createSurface, updateComponents, optional updateDataModel) so surface-app, catalog bundles, user-action handling, and Redis-backed UI state keep working unchanged. The companion markdown adapter produces plain text for channels that cannot render rich UI and ensures unknown fragment intents become readable Text components instead of unregistered blank UI.

Layer	Primary role	Runtime owner	Notes
TrueServe A2UI today	Widget protocol and trusted renderer path	surface-app + @trueserve/a2ui-react	Uses catalog manifests, signed bundles, component registry, per-surface state, and existing action events.
AXL-style experience fragment	Surface-neutral intent before rendering	Shared compiler module	Lets one agent/UI intent target A2UI first and later target markdown, Slack/Teams blocks, or other adapters.
External A2UI spec	Open protocol family for declarative agent UI	External standard and libraries	Track spec drift separately; v0.9.1 is the current production family, while v1.0 is a candidate with RPC/action changes.

AG-UI, A2UI, and RTVI

AG-UI is not the generated component system. In TrueServe terms, A2UI is the widget/rendering layer and AG-UI is an interoperability event protocol for agent/frontend interaction. We project sanitized conversation events into AG-UI only when an agent enables settings.agui.enabled; the existing A2UI renderer and surface-app remain the production UI path.

The AG-UI stream is authenticated SSE at GET /api/v1/conversations/{conversation_id}/events?format=agui. The event mapping is:

TrueServe event	AG-UI category
TURN_STARTED / TURN_ENDED	RUN_STARTED / RUN_FINISHED
LLM_STARTED / LLM_STOPPED	STEP_STARTED / STEP_FINISHED
Assistant transcript deltas/finals	TEXT_MESSAGE_*
Tool pending/completed/error entries	TOOL_CALL_*
A2UI ui_render payloads	STATE_SNAPSHOT then RFC 6902 STATE_DELTA under /a2ui/surfaces/...

RTVI is also not a replacement for surface-app. It is useful for bounded Pipecat interoperability experiments, especially around RTVIProcessor, RTVIObserver, Pipecat client-web, and possible UIWorker flows that synchronize realtime voice, transcriptions, LLM processing, and TTS. Keep RTVI behind a demo route/session until it proves value over the existing surface-app + conversation-api + A2UI stack. The current probe is GET /experimental/rtvi on ai-orchestrator with AI_ORCHESTRATOR_RTVI_PROTOTYPE=1; it reports installed Pipecat RTVI/UIWorker support and does not switch any client path.

Pieces

Piece	Where	Role
Components	TypeScript source compiled by catalog-builder	The actual UI
Catalog	a2ui_catalogs row + bundle in object storage	A versioned set of components
Tool definitions	a2ui_tools rows (one per component)	Schema the agent uses to emit
Experience fragment compiler	shared/a2ui-react/src/experience-fragment.ts	Optional intent-to-surface adapter; currently emits A2UI events and markdown fallback
Renderer	apps/multimodal-service	Resolves component descriptors → React
Surface	apps/surface-app	Hosts the renderer in an iframe
Builder	apps/catalog-builder	Compiles TypeScript components, publishes to object storage

End-to-end flow

1. Developer authors a component in TS source under a catalog directory.
2. Developer runs catalog-builder /publish.
   ─ Source archived to object storage.
   ─ ESBuild compiles to JS bundle.
   ─ Bundle uploaded.
   ─ a2ui_catalogs / a2ui_catalog_versions / a2ui_tools rows written.
3. Agent's pipeline gets the component as a callable tool via agent-host —
   the catalog's data tools are served as MCP tools at
   `agent-host-mcp:8003/{catalog-slug}/mcp`, and a stage can only call them
   when that MCP connection is in its `config.tools`. Note components do NOT
   need them: no data tool calls a real backend yet, so they are hidden from
   authoring and the wiring stays dormant — see [agent-logic.md](agent-logic.md),
   "Catalog DATA TOOLS are HIDDEN from authoring".
4. Agent emits a tool call: render_<component>({ props }).
5. Tool result → a2ui-event frame → conversation-event bridge.
6. surface-app's iframe renders via multimodal-service.
7. User interacts (clicks, fills form). Events flow back as user messages.

The experience fragment spike can sit between steps 3 and 5 for agents that want to emit surface-neutral intent first:

ExperienceFragment
  -> compileExperienceFragmentToA2UI()
  -> createSurface / updateComponents / updateDataModel
  -> existing A2UI renderer path

The voice handshake is consent-gated in code, not just in the prompt

On a voice call there is no surface until the caller opens a link we text them, so the handshake ({"enable_multimodal": {"permission": "granted"}}) is a send. Both A2UI execution paths — the A2UIToolHandler frame processor and the provider-local callback in a2ui_tool.py that Nova Sonic and the ADK bridge use — refuse to mint the share code until consent is recorded, and hand the model back the permission question to ask instead. Previously the SMS branch sent on any enable_multimodal call and the "ask first" rule lived only in prompt text, which a model is free to skip.

The node and standalone Multimodal editors discover delivery profiles through /api/integrations?capability=sms, not an amazon_web_services/aws string match. Integration sources declare capabilities independently of their type name, so an AWS profile appears when it advertises SMS and future SMS sources work without another UI filter change. All results are scoped to the selected tenant.

Consent is one fact per conversation, shared with Customer Authentication, because that capability texts a link too. Whoever needs it first asks; the other sends silently. A completed texted verification (passkey link, SMS/email OTP) counts as consent, so a card after verification uses the same open customer page. The handshake returns sms_status=connected without another SMS. If that page is no longer available, a replacement link uses the existing consent and the tool explains the actual send. A text channel auto-grants (the chat window is the surface). The full state table, the auth-side consent argument, and the fail-open rules live in customer-authentication.md.

Link suppression is not consent, and it is not a channel-wide skip. set_sms_suppressed marks a conversation whose visual surface is already attached (Agent Studio's Preview), so the A2UI link has nothing to send — and both paths short-circuit on state.suppress_sms before the SMS branch. It must not auto-grant consent: authenticate_customer on the same conversation still texts a real (or simulated) secure link, so the auto-grant deleted the question in the one place an author rehearses the flow. Nor may Preview keep suppressing: with the link suppressed, a card requested after verification never minted its own second share code, so the tester saw no message for the visual session. Preview therefore sends X-Multimodal-Sms-Delivery: demo_notification and not X-Suppress-Multimodal-Sms — the full handshake runs, with only the carrier simulated (the message lands in the test phone's tray). The optional, environment-gated full-screen SMS sends real_sms through that same header and requires an explicit ANI; because Customer Authentication reads the same session setting, every capability switches together. The two shared/phone-emulator hook defaults still suppress on purpose: those surfaces set no demo delivery, so unsuppressing them would put a real carrier SMS on the wire.

Unchanged: while verification is pending, the handshake stays out of the way entirely (customer_auth_is_pending) rather than putting a second, non-authenticating link on the same phone.

A rendered surface also counts as caller activity for idle timing: ConversationAPIPublisher's A2UIRenderFrame branch calls mark_visual_engagement on the idle processor (one hook, both A2UI paths), so "are you still there?" doesn't land while the caller reads a card. It is a timer reset rather than a suspension because nothing reports when they finish reading.

The catalog

A catalog is a tenant-scoped, versioned bundle. Tenant directories: catalog-tenant-<tenant_slug>-<catalog_slug>.

One surface has one catalog

A purpose-built component must be rendered on a surface created with its owning catalog, and a standalone single component must be the id="root" tree entry. Reusing a healthcare appointment surface for an insurance PhotoCaptureWidget, for example, produces a valid updateComponents event and a successful tool result but no visible widget: the new photo_widget is an unreferenced sibling and the appointment component remains the surface root. Catalog bundles currently register component types into one client registry, but that implementation detail does not make a cross-catalog surface or an orphan component tree valid.

build_component_descriptions_map preserves the owning slug returned by a2ui_components and names the exact selected catalog URI even when a stage enables several catalogs. Its prompt guidance tells the model to create a new surface when the component's catalog differs. Both runtime A2UI paths share component_catalogs as a backstop: for an unambiguous one-component cross-catalog update, they recreate the referenced surface with the owning catalog, promote the component to root, and then emit the update. This is deliberately narrow; the runtime does not guess the structure of a larger mixed component graph. Nova Sonic uses the provider-local callback in a2ui_tool.py, while other providers may use A2UIToolHandler, so the repair must remain in both paths.

A UI action must finish visually in the same turn

An authored transition such as Done on an appointment card followed by PhotoCaptureWidget is incomplete if the model only says it will show the next screen—or if it merely disables Done and treats that cosmetic update as the whole workflow. Component-token guidance and the synthetic [User Action: ...] turn both require the next component before speech. Every successful A2UI tool result repeats the same continuation rule before inviting a spoken summary, because the tool result is the most recent instruction a streaming model sees; it must finish any additional component/tool steps required by the current caller turn before it speaks or waits. ConversationAPIPublisher tracks each surface's root component and arms a completion guard for every UI action. For terminal navigation actions (Done, Next, Continue), a same-surface update of the same root is only an acknowledgement and does not clear the guard; a new/different component or surface deletion does. If the response ends before that transition, the publisher injects one tool-only repair turn automatically. The retry is limited to one so a provider that repeatedly refuses the tool cannot create an autonomous loop.

Each catalog publishes:

• Source archive (source.tar.gz) — for /restore to reverse a bad publish.
• Compiled bundle (bundle.js) — what multimodal-service serves.
• Manifest (manifest.json) — component list, props schema, default values.

Source archiving was added 2026-05-04 after a healthcare incident: a PVC wipe destroyed a customer's catalog source. /publish now always uploads source; /restore reverses.

Renderer

apps/multimodal-service serves the catalog bundle and resolves component descriptors. Surface-app embeds it in an iframe; the iframe origin is allowlisted via allowed_origins rows (see "CSP gotcha" below).

Catalog manifest resolution (why a surface can render as blank <div>s)

Surface-app's runtime catalog loader (catalog-loader.ts) fetches a per-conversation manifest from conversation-api (GET /api/conversations/{id}/catalog-manifest, built in apps/conversation-api/catalog_manifest.py), downloads each bundle, and calls its register() to populate the component registry. A component type with no registered renderer falls back to an empty <div> (A2UIComponentRegistry.ts) — so an empty manifest renders the whole surface as nested blank divs (<div class="a2ui-surface">…<div></div>…) even though the A2UI data arrived and the surface mounted.

The surface-app browser fetch for that manifest must bypass HTTP cache (cache: "no-store"), and it retries with a cache-busted URL if a bodyless 304 Not Modified still appears. The component registry is in-memory JavaScript state, so a refreshed iframe may have no registered catalog bundles even when the browser has a cached manifest validator. Treating a 304 as an empty manifest causes exactly the "A2UI event arrived, UI-state persisted, but the multimodal tab is blank" failure.

The manifest builder resolves catalogs from three places: (1) the agent_versions.catalogs pin column (pin-picker UI); (2) legacy top-level config.catalogs; (3) per-node config.multimodality.catalogs — the trueserve://catalogs/...@<pin> URIs the agent editor's per-node A2UI/multimodality control actually writes. Source (3) was added 2026-06-04 for agents configured purely through the node control (which leaves (1)/(2) empty); the node-URI harvest strips the @<pin> tail and live-resolves each slug to its catalog's current_version_id via _entries_from_catalog_uris.

Source (3) is UNIONED with (1)/(2), not an either/or (fixed 2026-06-25). The pin column is frequently NON-empty because new agents auto-pin the two baselines (standard-primitives, common-extended). If the node's multimodality control then selects a different catalog (e.g. copilot-support), an either/or manifest serves only the baselines and silently drops the selected catalog — so its component (e.g. ApprovalRequest) has no registered renderer and the surface renders blank, even though the orchestrator correctly emitted that component on the right catalogId. The builder now always harvests node URIs and merges their resolved entries into the final list (deduped by catalogId). When you see a mounted-but-blank A2UI surface, compare the manifest endpoint's catalogIds against the agent's config.multimodality.catalogs: if the agent's selected catalog is missing (but baselines present), it's this union bug — verify the pin column directly with SELECT catalogs FROM agent_versions WHERE agent_id=… AND status='draft'.

Catalog URIs come in two forms: trueserve://catalogs/v1/<slug> (shared) and trueserve://catalogs/t/<tenant_id>/<slug> (tenant). _entries_from_catalog_uris resolves the v1/ form against scope='shared' rows — but migration 183 (2026-06-04) rescoped all built-in catalogs to the TrueServe tenant, so no shared rows remain and a stale v1/<slug> URI now resolves to nothing → blank surface. That migration rewrites every TrueServe agent config's v1/<slug> URIs to the tenant t/<tsf>/<slug> form; if you hand-author a config or import an old one, use the tenant URI. By-id catalog pins (source 1) are scope-agnostic and were unaffected. The orchestrator's prompt-enrichment registry (a2ui_catalogs.py) is keyed on the old v1/ URIs but now resolves by trailing slug, so it keeps working regardless of a catalog's scope/tenant — it's metadata only, never a runtime gate.

The primitives catalog is ALWAYS appended to the manifest — the standard primitives are not built into the renderer (fixed 2026-08-04). Column, Card, Text, Button, TextField, CheckBox, ChoicePicker, Row, Image, Slider, DateTimeInput live in the standard-primitives catalog bundle like any other component; nothing is compiled into A2UIRenderer. They are also the structural scaffolding of nearly every hand-built surface (root is usually a Column/Card), and the A2UI tool description advertises the standard catalog to the model — so an LLM reaches for them constantly, whichever domain catalog is attached. An agent that attaches only domain catalogs (e.g. insurance + common-extended) therefore produced a manifest with no primitives, the root component had no renderer, and renderA2UIComponent returned null for it — the whole surface mounted and painted nothing. This is the nastiest variant of the family because it is completely silent: the model says it rendered, the orchestrator logs Successfully rendered 1 UI surface(s), tool_call events show status: completed, and the browser logs no error (the fallback is a null return, not a throw). _with_baseline_catalogs in catalog_manifest.py now appends BASELINE_CATALOG_SLUGS to every return path (agent, node-only, standalone multimodal, and the no-agent-found fallback), deduped by trailing slug so the two URI forms can't double-register, scoped to the agent version's tenant with scope='shared' as a legacy fallback and no cross-tenant borrowing (a tenant missing the baseline logs a warning; provision it with npx tsx apps/ai-manager/scripts/backfill-tenant-baselines.ts). Note the emitted catalogId is irrelevant to renderer lookup — the registry is a flat global keyed on the component type, so a surface created with catalogId: "https://a2ui.dev/specification/v0_9/standard_catalog.json" renders fine provided some loaded bundle registered those types; the missing bundle is the bug, not the URI mismatch.

The agent-stage Multimodal UI picker filters its /api/catalogs response down to renderable catalogs (status='published' with a current_version_id) and prefers tenant rows when duplicate slugs exist. Do not offer draft shared placeholder rows in that picker: they produce v1/<slug> URIs, cannot load bundles, and make valid A2UI events render as blank wrappers.

Malformed a2ui_json: repair at the publisher, not just at the renderer

An LLM sometimes emits structurally invalid JSON in the A2UI tool's a2ui_json argument — a duplicated closing bracket (…}}]]), or a ] where a } was expected. Nova Sonic does this regularly.

The render path has always tolerated it: _process_a2ui_call (a2ui_tool.py) catches json.JSONDecodeError and retries through json_repair, so the surface draws correctly. But ConversationAPIPublisher._publish_tool_call published the raw, unrepaired arguments, so every SDK re-parsed the broken original and logged a pair of console errors ([A2UI] Failed to parse a2ui_json + [A2UI] Problematic JSON string: …). Symptom: the component renders fine and the agent behaves correctly, yet the browser console shows a SyntaxError per render — because the two paths are independent (the ui_render event draws the surface; the tool_call event is what the SDK re-parses).

Fixed 2026-07-27 in two layers:

• Server (the real fix). repair_a2ui_json_argument / repair_a2ui_tool_arguments in a2ui_tool.py normalise the argument, called from _publish_tool_call in conversation_api_publisher.py — the single chokepoint all seven publish call sites funnel through, so no branch can leak an unrepaired payload. Valid JSON passes through byte-identical (no re-encoding churn); an unrepairable payload is published as-is rather than dropped; the caller's arguments dict is never mutated (the tool handler still reads it).
• Client (defence in depth). repairJsonBrackets in shared/trueserve-sdk/src/sdk.ts does a string-literal-aware bracket-balance repair and retries the parse once before logging, so an older orchestrator degrades to one warn instead of a dropped render plus two errors. surface-app's in-tree SDK copy already had this; the packaged @trueserve/sdk (what ai-manager's agent tester loads) did not.

Keep the SDK's existing streaming-fragment heuristic in mind: a payload that doesn't end in ]/} is treated as a mid-stream fragment and skipped quietly. These payloads did end correctly — they were malformed, not truncated — which is why that heuristic never suppressed them.

Valid JSON is not the same as usable JSON: every element must be a mapping

Repair only answers broken JSON. The other failure is JSON that parses perfectly into the wrong shape. Every reader downstream does "<op>" in message and then message[op], so an element that is not a mapping is either a crash or — worse — a silent miss. Two live models have produced two different versions of it:

• A double-encoded element. The model wraps its own objects in quotes: ["{\"enable_multimodal\": {…}}"]. Nothing raises at the parse. The handler's first act is a membership test, "enable_multimodal" in message, and on a string that is a substring test that happily returns true, so the very next line raised TypeError: string indices must be integers. The tool returned an error and the caller heard the agent apologise for technical difficulties on a card that was otherwise correct.
• A flat alternating list. The operation name arrives as its own element with its arguments following: ["enable_multimodal", {"permission": "granted"}, "createSurface", {…}]. This one raises nothing at all — the bare string is unparseable and gets dropped, and the argument dict that follows matches no operation, so the whole call becomes a no-op. On agent 3032 (Nova Sonic) that produced a loop: the handler answered "Cannot send A2UI yet: the visual session has not been opened", the model re-sent the same flat shape, and POST /conversations/{id}/multimodal-sms was never called on those turns — so it read as "multimodal links aren't being sent" rather than as a payload-shape problem.

normalize_a2ui_message_shape in a2ui_tool.py is the one gate, and it runs before anything reads an operation off an element. A string that names a known operation (_A2UI_OP_KEYS) becomes a pending op paired with the next mapping; any other string is json.loads-ed (falling back to json_repair) and logged as an unwrapped double-encoding; a nested list is flattened, with a pending op taking its first mapping; a non-object is dropped with a warning; and a trailing operation name with no arguments is dropped rather than paired with an invented empty object — guessing there would dispatch an operation the model never specified. An argument that yields no mappings at all returns a model-facing error naming the shape it should have sent ([{"createSurface": {…}}, …], "each operation name must be a KEY of an object") instead of a Python type error. A well-formed payload passes through with the same dicts by identity, so this is a genuine no-op on the happy path.

Same both-paths rule as the repair above and the fold below: it is called in _process_a2ui_call (the provider-local route Nova Sonic and the ADK bridge take) and in A2UIToolHandler._handle_a2ui_tool_call. The frame-processor path had no element normalisation whatsoever before this — a bare "enable_multimodal" string would have satisfied its in test by substring and then raised. Which path a call takes is a property of the provider, not of the payload, so fixing one leaves the other broken on exactly the providers you didn't test. Regression: the flat-shape group in test_a2ui_shape_normalization.py, which drives the verbatim 3032 payload through both dispatch paths.

Mis-nested component definitions: fold them back, and refuse a tree that still can't render

There is a third, deeper "parses fine, wrong shape" failure, and it is the one that produces a genuinely blank surface on a call that reports success. The renderer requires that updateComponents.components be an array holding every component definition, that exactly one carry id: "root", and that every id named by a child/children reference be defined in that same array (unregistered/undefined types render nothing). A model — Nova Sonic especially, whose token stream is under no schema enforcement — sometimes streams a components array holding only root while the remaining definitions ride as loose sibling keys: some beside updateComponents at the message level ({"travel": {…}, "cash-back": {…}, "updateComponents": {…}}), some inside it beside components ({"intro": {…}, "title": {…}, "components": [{"id": "root", …}]}). Nothing raised. The render loop read components verbatim, registered an empty root whose children pointed at ids it had never seen, and the surface painted nothing — while the model reported success and the caller heard "I've sent you a visual page."

normalize_a2ui_components in a2ui_tool.py is the one shape gate for this. For each updateComponents message it folds every loose component-shaped key (a dict carrying component or id) — at both the message level and the updateComponents level — into components[], deletes the mis-nested keys, then validates the folded tree: missing root, or any child/children reference that resolves to no definition, is reported. A caller with problems tells the model what to fix ("put EVERY component object inside the updateComponents.components array, keep exactly one id='root', define every child you reference") rather than dispatching a blank surface — the model can resend a complete array in the same turn. A well-formed payload has neither loose keys nor dangling refs, so it passes through byte-identical (the no-op guarantee), and non-updateComponents ops are left exactly as authored.

Two kinds of failure fall out cleanly. If the loose defs were complete — every referenced child is somewhere in the payload, just mis-placed — folding alone recovers a renderable tree and there are no problems. If the payload also names children that were never defined anywhere (the field case that motivated this carried both tips and content as dangling refs), folding cannot invent them, so the tree is reported unrenderable and the model is asked to resend rather than drawing a partial surface.

Like the malformed-JSON repair above, this must run on both A2UI dispatch paths or it silently regresses on the provider it was not tested against: it is called inside _process_a2ui_call (the provider-local route Nova Sonic and the ADK bridge take) and inside A2UIToolHandler._handle_a2ui_tool_call (a2ui_tool_handler.py, the frame-processor route), each returning the same corrective message to the model — as a tool result, never an isError (which would abort the turn instead of letting the model self-correct). Regression: test_a2ui_shape_normalization.py, which drives the exact blank-surface payload from the field and asserts both dispatch paths refuse it.

Operation-envelope shape: the A2UI wire form, which matches no operation and renders nothing

The fourth "parses fine, wrong shape" failure is the whole message being in the A2UI wire form rather than the op-keyed form TrueServe's dispatcher reads. Both dispatchers key off the operation name — present_ops = [op for op in _OP_ORDER if op in message], where _OP_ORDER is ("createSurface", "updateComponents", "updateDataModel", "deleteSurface") — so a message must have its operation as a top-level key: {"createSurface": {"surfaceId": …, "catalogId": …}}. Nova Sonic (QA agent 4066, conversation 139defc6) instead emitted the operation-envelope form the A2UI spec uses on the wire: {"operation": "createSurface", "surface": {"id": …, "catalogId": …}} and {"operation": "updateComponents", "components": […]}. The keys operation/surface/components match none of _OP_ORDER, so present_ops is empty, the dispatcher logs Unknown A2UI message type and renders 0 surfaces — then reports success. The caller sat at "Waiting for the agent to send a multimodal component" on an agent whose multimodal config, catalog bundle, presign, fetch and CORS were all verified healthy; the only trace was rendered 0 UI surface(s) in the orchestrator log.

normalize_a2ui_operation_envelopes in a2ui_tool.py rewrites each {"operation": <op>, …} message to {<op>: payload}: the sibling keys become the payload, a surface/dataModel/data wrapper is unwrapped and folded up (mapping its id → surfaceId), and everything else is kept verbatim. Two subtleties are load-bearing. (1) In the wire form a createSurface establishes the active surface and the following updateComponents omits surfaceId and implicitly targets it — but both dispatchers resolve each message independently (no cross-message "current surface"), so a rewritten updateComponents with no surfaceId would default to a fresh surface (surface-<tool_call_id[:8]}) and its components would orphan off the one just created. The rewrite therefore carries the last surfaceId forward onto a following op that lacks one, so create + populate land on a single surface. (2) It runs first, before anything keys off an operation — including the enable_multimodal handling and normalize_a2ui_components — and messages already in op-keyed form (and enable_multimodal) pass through untouched, the no-op guarantee.

Same both-paths trap as the three failures above: wired into _process_a2ui_call (Nova/ADK) and A2UIToolHandler._handle_a2ui_tool_call, since a fix to one silently regresses the other. Regression: test_a2ui_shape_normalization.py drives the verbatim 4066 envelope payload as pure-function tests and renders it end-to-end through both dispatch paths.

max_tokens is an A2UI budget: a truncated argument makes the agent go silent

send_a2ui_json_to_client takes the whole A2UI document as one string argument, so the payload is spent out of the stage's completion budget alongside the reply text. A per-line table is easily kilobytes: a 16-row OrderLineTable is ~3.5 KB of function-call arguments.

Overrun config.max_tokens and the provider truncates mid-JSON. This is not the malformed case above — json_repair cannot recover a document that stops halfway through an array — and the failure signature is much worse than a blank surface:

WARNING | pipecat.services.openai.base_llm:_process_context - Failed to parse function
call arguments: {"a2ui_json":"[{\"updateComponents\": {\"surfaceId\": \"order-lines\" …

The tool never runs, the surface never draws, and the turn produces no text at all — the agent simply stops mid-conversation. Nothing reaches the transcript, nothing reaches the user, every service stays green, and the only evidence is that one provider-level warning. It surfaces late, too: it fires on whichever beat first renders a full table, so an agent can pass every earlier exception-only render and die on the summary.

A stage that renders per-line or per-row tables therefore needs real headroom — 4096 rather than the usual 1024 — and the budget must cover the payload plus the sentence the agent says about it. When a table is genuinely large, prefer a component prop that filters server-side (showCleanLines, hiddenCleanCount) over sending every row.

Worked example: medtech-po-exceptions-concierge (135_medtech_journey_c.sql, header note 6).

Component props are camelCase; status values are the tool's own words

The standard prompt instruction — tools speak snake_case, components speak camelCase — applies to keys only, and models reliably over-apply it to values as well. Two distinct render bugs come out of this, both silent:

• A key the component doesn't recognise is not an error — it renders as a dash. Copying a tool's po_price / contract_price / line_number straight into PriceDiffTable produces a table of em-dashes where the prices should be, with no warning anywhere.
• A prettified status value breaks logic, not just the label. A component that owns its own label mapping (STATUS_LABEL) prints an unmapped value raw, and any conditional keyed on the literal value stops firing: OrderLineTable draws the per-row ETA only for lineStatus === 'backorder', so a model sending "Backorder" silently deletes "ETA 21 days" from the one beat that exists to show it.

So: convert every key, including keys inside nested line objects; copy status values through byte-for-byte. When a component's status/lineStatus is a closed enum, say so in the prompt and list the members — a tool's own status word is frequently not a member (a PO the OMS calls received is processing to OrderStatusCard).

Layout and treatment over the gRPC hop

A render event can carry two optional presentation hints beside its payload:

Hint	Shape	Meaning
layout	{region: hero\|main\|rail\|full, order: int, span: 1\|2, title: str}	Where the surface belongs on the canvas.
treatment	quiet \| focus \| reframe \| unfold	How loudly to present it; stamped as data-treatment on the surface by A2UIRenderer.

Both are latest-wins per surface and only-when-present: an update that omits layout keeps the surface's previous placement, and a producer that sets neither is unaffected by any of this.

Copilot cards required layout and treatment to cross the gRPC boundary. UIRenderEntry (proto/conversation.api.proto) carries them in fields 6 and 7:

string layout_json = 6;   // {region, order, span, title} canvas layout hint
string treatment  = 7;    // quiet|focus|reframe|unfold

layout crosses as a JSON string because protobuf has no bare-dict type. Both readers accept a stored dict or that string: _ui_render_entry in event_transformer.py handles replay, while the live UI_RENDER arm of process_sse_outgoing_entry (http_api.py) decodes layout_json and keeps only dictionary values. The wire form is the string declared by the proto.

One rule protects the only-when-present contract:

• The publisher sends "", never "{}". _publish_ui_render (conversation_api_publisher.py) emits json.dumps(layout) if layout else "" — an empty JSON object would decode to a truthy-shaped hint that names no region, i.e. present-but-meaningless. Dropping a hint in transit is not a cosmetic loss. For a copilot card the hint is {"region": "rail"}, which is the only thing that marks the surface as an Agent Assist card — so losing it doesn't degrade the card, it moves it, onto the multimodal canvas where a rail card has no business being.

Replace-in-place is the stable surface id, not a dedup

A copilot card refreshes on every turn, and it must swap rather than stack. That is achieved entirely on the client: _publish_copilot_card uses message_type: "updateComponents" on every publish (never createSurface) against a stable surface_id — copilot:<worker id> — with the same two component ids each time, and the client merges components by id and creates the surface when it's absent. There is no server-side dedup anywhere in this path. If you find yourself adding one, the bug is a surface id that isn't stable.

The summary card's order is likewise derived, not configured: _copilot_card_layout stamps COPILOT_SUMMARY_LAYOUT_ORDER (-100, in suggestion_frames.py) when the frame's suggestion_type/worker_id says summary, so the summary pins above the per-turn cards. A layout_order config key would have had to thread through the type defs, the planner, the wrapper, the processor config, the frame, and the JSON Schema to say something two fields already say.

One pending approval decision has one surface

An ApprovalRequest is different from the append-only cards above: a second card for the same pending decision leaves two actionable controls in the supervisor chat. Models sometimes refine the request in another A2UI call and invent a new surface id. coalesce_approval_request_surfaces in a2ui_tool.py retains the first pending approval surface and rewrites later ApprovalRequest-bearing createSurface/updateComponents operations onto it. The prompt also tells the model to update the pending request rather than create another one.

This guard runs in both A2UI execution paths: the provider-local callback used by Nova/ADK and A2UIToolHandler. An explicit deleteSurface releases the id for a future decision. Regression coverage drives two differently named approval surfaces through both paths in test_a2ui_shape_normalization.py.

The copilot rail is a partition at ingest, not a filter at each consumer

surface-app splits render events once, on arrival: partitionCopilotSurfaces (copilot-surfaces.ts) returns {railEvents, canvasEvents}, and uiRenderEvents downstream is the canvas half. This is deliberate, because that array feeds eight consumers in page.tsx — the multimodal tab, the unified timeline's surfaceGroups, the card view, the a2ui_only page, the auto-scroll area, the preview bridge, and both transcript-view render arms. Suppressing rail cards at the three transcript sites would still have leaked them onto the multimodal canvas and into a2ui_only. Partitioning instead means every existing consumer keeps its behaviour with no edit, and the rail is the only thing that opts IN.

Consequences worth knowing:

• There is no agent-type gate. Nothing in surface-app tests for agent_assistant or a copilot subtype; the rail is purely region-driven. Any producer that sets region: "rail" lands there.
• Classification is by the LAST event carrying a layout, not the first — matching latest-wins. So a hint-less createSurface followed by an updateComponents carrying region: "rail" is a rail surface, and both events must travel together, or the rail renders an update for a surface it never created.
• Grouping is per surface id, not per time window. groupCopilotRailCards sorts on layout.order with first-seen as the tiebreak and deliberately does not reuse the 5 s suggestionBatches grouping: batching exists to keep an append-only stream readable, and this stream doesn't append.
• The rail auto-opens when a copilot card exists (showSuggestions = showSuggestionsParam || copilotRailCards.length > 0), which is why the CRM wrapup preset must not also set showSuggestions — see CRM embed components.

An inline copilot translation is not a card at all and never reaches this path: it publishes as a transcript with source: "translation", additively beside the original line, because surface-app's transcript already selects between the two on preferredLanguage. See Copilots.

Component preview (ai-manager → surface-app bridge)

The Multimodal → Components tab in ai-manager's Tools & Templates page (and the catalog detail page) previews a single component via ComponentPreviewSheet.tsx: it embeds surface-app with ?a2uiPreview=1&embed=true, includes the synthetic preview token subject as agentId so /api/session/me can pick the right cookie entry when the browser already has other surface-app tokens, then drives a postMessage handshake — tsf:preview-bridge-ready (iframe is listening) → tsf:preview-bundle (parent pushes the presigned bundle URL) → tsf:preview-bundle-ready (iframe loaded + registered it) → tsf:preview-render (component + sample props). The iframe side is installPreviewBridge. Its preview-<uuid> conversation id is likewise a synthetic session/token binding, not a conversations.id; /api/embed/mint-internal must preserve it in the token without querying the UUID-backed conversations table.

The TrueServe Studio preview panel inside code-server uses the same a2uiPreview=1 surface shell, but it must mint a signed preview embed token through studio-api (/api/studio/mint-embed-token) first. The webview iframe is pointed at surface-app's /api/session/exchange?t=<JWT>&return=/en?...a2uiPreview=1 endpoint so surface-app sets the sfa_session cookie before rendering. For HTTPS browser-facing requests, the exchange stamps the cookie with SameSite=None; Secure even if the underlying Next.js server is running in dev mode; without those attributes, browser iframe cookie rules can redirect back to the preview shell and then trip surface-app's "Embed session required" guard.

The overall Tools & Templates Components list is fed by GET /api/catalogs/components. That endpoint dedupes components by (catalog_slug, component_name) and prefers rows whose current catalog version can be joined to an actual bundle. Do not enable preview from a2ui_catalogs.current_version_id alone: stale or duplicate baseline rows can point the sheet at a catalog whose /api/catalogs/:id/preview response is available=false, even though another copy of the same slug previews correctly from its detail page. As a final guard, ComponentPreviewSheet retries by catalog slug and switches the preview bridge to the resolved catalog id when the row's original id has no bundle.

The parent MUST pin its own origin on the iframe URL (&parentOrigin=<studio origin>); the document.referrer fallback is a last resort that the session-exchange redirect silently corrupts. resolveAllowedParentOrigin in catalog-loader.ts resolves the trusted parent origin in priority order: (1) the explicit options.allowedOrigin the caller passes, (2) NEXT_PUBLIC_A2UI_PREVIEW_ORIGIN / NEXT_PUBLIC_AI_MANAGER_ORIGIN, (3) document.referrer. installPreviewBridge (and the live-theme listener in BrandingProvider.tsx) refuses to run without an allowedOrigin, and drops every postMessage whose ev.origin !== allowedOrigin. ai-manager's origin is dynamic across environments (localhost, Codespaces *-3001.app.github.dev, ai-manager.<domain>), so the static env vars are usually unset — a security hardening (commit 433e30eb, 2026-05-28) that only read those vars once left the bridge disabled, so it never posted tsf:preview-bridge-ready and the preview hung forever.

The referrer fallback was added to cover that, but it is not reliable for the component preview, because ComponentPreviewSheet's iframe URL always carries the embed token ?t=. surface-app's proxy.ts sees ?t= and 302s through /api/session/exchange?t=…&return=… and back to the return URL — both hops on surface-app's own origin. Under the default strict-origin-when-cross-origin referrer policy, that same-origin final redirect rewrites the iframe's document.referrer to surface-app's own origin, not the Studio parent's. So in any environment where the env vars are unset (dev and QA both), the bridge's referrer fallback resolved allowedOrigin to surface-app.<domain> ≠ studio.<domain>, dropped every message from the Studio parent, and posted tsf:preview-bridge-ready back to the wrong origin — every component preview hung ("Loading latest bundle…" on the Studio side, "Waiting for component…" in the iframe), env-independently (so prod would hit it too). The fix pins the origin explicitly: ComponentPreviewSheet.tsx appends parentOrigin=${window.location.origin} to the iframe URL, and page.tsx reads searchParams.get("parentOrigin") and passes it as installPreviewBridge(…, { allowedOrigin }). The param survives the exchange redirect because proxy.ts strips only t. This is safe because every message is additionally gated on ev.source === window.parent, so pinning the origin only names the window already framing us. Keep the referrer fallback (it still covers callers that don't pin), but never rely on it for a ?t=-carrying iframe, and don't revert to env-var-only resolution.

Relatedly, preview bundle/css URLs are MinIO/S3 presigned URLs on a different origin than the embedder, so they can't be pinned to the parent origin. The bridge validates them with a scheme allowlist (http(s) only — blocks javascript:/data:/blob:); the sha384 integrity check in fetchAndVerify is what guarantees the bundle content wasn't tampered with. Don't re-add an origin-equals-parent check on the bundle URL — it rejects every real presigned URL.

A second, entirely separate cause produced the identical hang in QA: the presigned URL came back null, so the bridge rejected a URL-less bundle. The failure is upstream, in the MinIO client's region resolution. On an AWS/KMS EKS pod the S3 credentials come from IRSA (web-identity token), which requires AWS_REGION — but MINIO_REGION/S3_REGION are typically unset there, so getMinioClient() built a client with no region pointed at a regional endpoint (s3.us-west-2.amazonaws.com). minio-js presign then tries to auto-discover the bucket region by probing signed with its default us-east-1; AWS rejects with AuthorizationHeaderMalformed (expecting 'us-west-2'), and minio's retry keys on err.name (which is 'S3Error', not 'AuthorizationHeaderMalformed' — that string is on err.code), so it rethrows instead of retrying. presign() in preview/route.ts swallowed the throw to null, and the GET handler still reported { available: true, bundleUrl: null } — so the Studio pushed a null-URL bundle and previewBundleUrlsAreSafe rejected it (logging the misleading rejected preview bundle with unsafe URL scheme) while Studio hung on "Loading latest bundle…". Confirmed by reproduction: the object exists and the IRSA role can read it (aws-sdk HeadObject OK), STS/GetBucketLocation succeed, and minio presignedGetObject throws with region unset but succeeds the instant region is set. The fix is in minio-client.ts resolveRegion(): region now falls back to AWS_REGION/AWS_DEFAULT_REGION, which pins the region and skips the broken probe — restoring every presign on IRSA, not just catalog preview. Two defensive follow-ons in the route: presign() now logs the throw instead of swallowing it silently, and a version whose bundle can't be presigned reports available: false (reason: 'bundle_unavailable') so the preview shows an honest message instead of hanging. This is a cross-cutting MinIO-on-IRSA fact — see the root CLAUDE.md.

Surface integration

surface-app mounts the multimodal renderer in an iframe per conversation. The renderer subscribes to a SSE channel from conversation-api for incoming descriptors and posts message events back when the user interacts.

On a phone call, the two ends address the conversation differently

The surface arrives holding a share code, which resolves to conversations.id (the uuid). The runtime writes under conversations.session_id, which on telephony is the Asterisk channel id (1788193570.12). Every Redis key on this path is <prefix>:{conversation_id}, so both halves of the round trip missed each other on a real call:

• Renders: the orchestrator XADDs entries:1788193570.12; the surface's SSE replayed entries:<uuid>, which did not exist. The page opened, reported healthy, and sat on "Waiting for content from your session…" forever. SSE replays from "0", so it was never a timing race — the stream was the wrong one.
• Actions: a tap published to a2ui:action:<uuid> and the route answered no delivery channel available, because the gRPC callback registry is keyed on the session id too.

Browser and Preview sessions mint one uuid for both columns, so none of this is reachable without a phone call — which is how it survived. conversation-api resolves the pair and fans out (conversation_aliases.py); note that fanning out the channel is not sufficient for a2ui:action:/multimodal:connected:, because the orchestrator's listener and agent-host's queued-render flusher both route on the conversation_id in the body. Full detail, including what deliberately wasn't converted: conversation-api § the same split in Redis.

Awaiting custom component actions

Catalog components that need transport-aware feedback use the render context's dispatchAction function. It returns a promise that settles after the surface consumer finishes sending the UI action, so a component can show an in-flight state, confirm success, and unlock retry only after an error. The older handleAction function remains fire-and-forget for components that do not render action progress.

Guard duplicate submissions inside the custom component as well as in the renderer. The renderer's optimistic commit lock covers standard primitive buttons, but a catalog component owns its internal controls. PhotoCaptureWidget, for example, uses an immediate ref gate plus visible uploading, uploaded, and error states: repeated clicks cannot emit another upload while the first is in flight or after it succeeds. It offers separate Camera (explicit getUserMedia, with a live preview) and Upload file controls so desktop webcam support does not depend on the inconsistent capture attribute of a file input. A non-empty requiredPhotos array creates named required slots; otherwise maxPhotos creates generic optional slots and at least one selected photo is required. Treat an empty requiredPhotos array like omission so a model-generated payload cannot render a zero-slot submit form.

Photo bytes must not cross the UI-action channel. The surface host intercepts the widget's in-memory images, uploads each one through POST /conversations/{id}/images (or the share-code equivalent), and replaces every data URL with its object-storage key, bucket, internal URL, s3:// URI, S3 ARN, content type, and byte count before calling sendUIAction. The spinner remains active across both the attachment uploads and the action dispatch. conversation-api is also an enforcement point: if an older host submits a PhotoCaptureWidget data URL, post_ui_action persists it to the configured recordings bucket and replaces it before Redis or gRPC publication. The browser/model never chooses the bucket. The endpoint still rejects combined UI-action context/data-model JSON above 256 KiB with 413 after normalization and logs only action metadata and payload size rather than the context itself.

For voice previews, multimodal activation can arrive either as an ENABLE_MULTIMODAL SSE event or as the first send_a2ui_json_to_client payload item, for example {"enable_multimodal":{"permission":"granted"}}. The surface SDK treats permission, permissionStatus, and permission_status as the same field and accepts in-progress A2UI tool calls with running, executing, or completed status so the Preview panel can switch on the embedded multimodal surface before render messages follow.

The iframe's frame-ancestors CSP must include the surface's origin. There was a bug where getActiveOrigins() filtered tenant_id IS NULL, making the allowlist empty for tenant-scoped rows; surface-app fell back to 'self', killing the iframe. Fixed 2026-05-27.

Dynamic data binding

A2UI components can declare props that resolve from session variables, conversation state, or tool outputs. Resolution happens in agent-host before the descriptor reaches the surface.

User actions are sanitized before they reach the LLM — a widget's dataUrl never rides the context

A component action ([User Action: "…"] Surface: … Context: {…} Data: {…}) is serialized into a synthetic user turn by ConversationAPIPublisher (both the gRPC and Redis paths). sanitize_ui_action_payload (conversation_api_publisher.py) walks the context + data model first: a data URL longer than 256 chars becomes <image/jpeg attachment, ~213 KB — content omitted> and any other string over 2,000 chars is truncated with an explicit marker, with the surrounding shape kept intact. Without this, a PhotoCaptureWidget submit injected the full base64 of every photo as a user turn — the model stalled ~10 s on a six-figure-token input and then reacted to the noise unpredictably (a live session jumped straight to collect_payment, since its prompt's next MUST-step was payment). The model needs to know what arrived, never the bytes; the actual file handling belongs to the widget's own tool/plugin. Fixed 2026-08-13; regressions in test_ui_action_sanitize.py.

Chat events: UI actions render in the transcript as events, not bubbles

The synthetic [User Action: …] turn above is model-facing only — it (and the payment/platform-notice injections) must never render as a caller bubble. The one suppression chokepoint is _publish_transcript in conversation_api_publisher.py (SYNTHETIC_USER_TURN_PREFIXES): on an S2S provider (Nova) an injected user turn round-trips through the provider's own user-text events and comes back as an ordinary TranscriptionFrame, so a per-path check misses the next path — which is exactly how [User Action: "submit_headshot"] … Required response ordering … was stored as a caller message in the Salesforce transcript.

What the transcript shows instead is a chat event (CHAT_EVENT entry → conversation_events.event_type='chat_event', replayable post-call): a centered SLDS-style line — icon, short bold title, muted subtext, optional thumbnail. Producers:

• UI actions — both publisher handlers (gRPC TYPE_UI_ACTION + the Redis fallback) publish one per click. The wording comes from the catalog component's emits[].event_view ({title_template, subtext_template?, icon?, image_prop?}, {placeholder} dot-paths over the sanitized label/action/context.*/data.*; unresolved placeholders stay literal). The lookup keys on the component type (smuggled as context.__component, beside __label) because action names are author-renamable props; a component with exactly one view applies it to any renamed action, and a missing view falls back to User clicked "<label>". Resolution is producer-side so the stored event is self-contained. image_prop only ever carries a URL/storage ref — never a data: URL.
• Payment outcomes — record_outcome in conversation-api (payments/service.py), covering both capture surfaces; titles come from the STORED status words (the SEPA rule), only from the call that actually settled the row.
• Verification — _grant in customer_auth/service.py, the one choke point every successful ceremony funnels through.

conversation-api-side producers use the chat_events.py helper (persist + alias-fanned live XADD, never raises); orchestrator-side ones ride the gRPC ChatEventEntry, which grpc_api persists and streams in one hop. surface-app renders them via ChatEventDisplay (both SDK copies parse CHAT_EVENT). Regressions: test_synthetic_turn_suppression.py, test_ui_action_event_view.py (ai-orchestrator) and the chat-event cases in test_event_transformer.py (conversation-api).

Read-only viewers: persisted surfaces, live UI_STATE, and readOnly mode

A CRM transcript embed (hideInput=true) is a spectator of the caller's surfaces. Three pieces make that work:

• ui_render/enable_multimodal entries are now persisted as conversation_events rows (grpc_api _persist_debug_event), so the Postgres historical replay carries the surfaces — before this, a viewer opening the case after the Redis stream expired got a permanently blank multimodal panel.
• every PUT …/surfaces/{id}/ui-state write (the caller's debounced form fill) also XADDs a UI_STATE stream entry to both conversation addresses and persists a ui_state event row — one mechanism gives live viewer sync and post-call reconstruction. The Redis hash stays the fast initial hydrate.
• @trueserve/a2ui-react's readOnly mode (A2UISurfaceManager/ A2UIRenderer): pointer-events-none is the whole interaction gate, the debounced ui-state persist is SKIPPED (a viewer's stale copy must never clobber the caller's in-flight state), and incoming liveStates updates are applied value-compared (latest-wins per surface on updated_at). surface-app passes readOnly={hideInput} and feeds liveStates from the SDK's UI_STATE subscription; the camera controls are hidden entirely for read-only viewers.

Submitted photos need a server-side merge, because the widget only ever holds dataUrls. PhotoCaptureWidget keeps captured photos as component-local base64 — nothing a ui-state write may carry (inline bytes are exactly what the sanitizer exists to strip). The durable copies exist only server-side: post_ui_action's _persist_inline_ui_photos rewrites the submitted dataUrls into storage URLs, and _merge_submitted_photos_into_ui_state (http_api.py) then merges a compact __photos list ({label, url, contentType} only) into that surface's ui-state — hash + UI_STATE fan-out + persisted row, the same three-legged delivery as any state write, plus commit_locked so the widget reopens in its uploaded state. The widget hydrates thumbnails and the "Photos uploaded" lock from data_model.__photos (label-matched to slots, positional fallback), proxying /conversations/…/images/… refs through the BFF image route. Regressions: test_ui_state_photo_merge.py — including that a dataUrl (rewrite didn't happen) never reaches ui-state. Without this, a tab revisit, a read-only CRM viewer, and post-call replay all showed empty capture slots for a submission that plainly succeeded.

Authoring components

In a catalog directory:

// my-button.tsx
export const MyButton = ({ label, onClick }) => (
  <button onClick={onClick}>{label}</button>
);

export const meta = {
  name: "my_button",
  description: "A button the agent can render",
  props: {
    label: { type: "string", description: "Button text" },
    onClick: { type: "callback", description: "User click event" }
  }
};

catalog-builder compiles this and registers it as a callable tool the agent can emit.

For full authoring docs, see the catalog-builder README. The previous standalone page (docs/_archive/2026-05-pre-overhaul/a2ui/custom-catalogs.md) covers component patterns; pull what you need from there.

See also

• catalog-builder service doc
• multimodal-service service doc
• surface-app service doc
• Copilots (Agent Assist) — the producer behind the rail cards
• Local Actions / host_action
• A2UI spec convergence — draft analysis of this dialect vs the a2ui.org spec