Conversational authoring capability evidence¶
Status: Draft | Last reviewed: 2026-09-10
Use this matrix to decide what conversational authoring can reuse and what must be implemented or tested. Evidence is from the current working tree on the review date, including existing uncommitted work; it is not a claim about a released commit or deployed tenant.
Agent-type contracts¶
| Requested type | Existing representation | Design requirements |
|---|---|---|
| Voice Agent | subtype: voice_agents, agent_type: voice_ai; pipeline selected from supported voice services |
Entry channel, language, speech policy, interruption, latency, tool progress, handoff, visual fallback |
| Chat Agent | subtype: chat_agents, agent_type: text_chat; generation validator requires one LLM stage |
Knowledge scope, citations, messages/components, attachments, authentication, tool result handling |
| Translation Agent | subtype: translation, agent_type: voice_augmentation; directional translation flows |
Speakers, language per direction, transcription/translation/TTS or supported S2S translation, terminology, mixing, failure behavior |
| Background Agent | Existing workflow identity/version/config and workflow editor | Trigger, input/output contract, graph, execution mode, retries, idempotency, timeout, schedule/activation |
Evidence: pipeline types, workflow types, Studio integration, and translation runtime.
Keep Copilot/Agent Assist supported as an existing adjacent type. Do not rename it Background Agent or migrate existing workflow IDs into conversational-agent IDs.
Capability map¶
Reuse means an inspected implementation exists. Integrate means authoring needs an adapter or broader discovery. New means a proposed mechanism is missing in the inspected paths. Verify means runtime/tenant evidence is still required.
| Capability | Evidence | Authoring work |
|---|---|---|
| Agent/workflow generation | Generation process, runner | Reuse jobs/questions/validation; integrate pre-target brief |
| Existing-draft assistant | Editor assistant | Reuse target-bound tools; new revision protection |
| Templates and prompts | Generation MCP | Reuse searches; add compatibility/provenance facets |
| Saved and inline tools | Tool types, generation MCP | Reuse real definitions; new unresolved-intent lifecycle |
| Tenant integrations | list_tenant_integrations in generation MCP |
Reuse metadata; add operation eligibility and health checks |
| Provider services | Service registry | Reuse runtime definitions; join tenant/model availability |
| Smart/Guided behavior | Universal pipeline builder, config types | Retrieve exact supported structure; verify combinations |
| Procedures and supervision | Procedure supervision | Reuse authored mechanisms; test prompt/runtime enforcement |
| Variables and guards | Agent Logic, validator | Reuse declarations, availability/bind/capture; prefer enforced rules |
| Capability packs | Capability packs, workspace adapter | Reuse reusable prompt/tools/variables/logic units; add discovery |
| Knowledge | Tool types and configuration contract | Reuse sources; verify indexing, visibility, retrieval, citations |
| Workflow nodes/triggers | Workflow triggers, workflow types | Reuse manifests; test activation, duplicate events, outputs |
| Customer authentication | Customer authentication | Reuse platform ceremony; verify tenant and stage gates |
| Payments | Payments | Reuse only verified channels; surface current limitations |
| Transfer and supervision | Warm handoff, human supervision | Discover destinations and prerequisites; test unavailable recipient |
| Branded surface | A2UI rendering, Studio API | Reuse surface configuration; snapshot intended identity |
| Catalog source/build/preview | Code-server, builder | Reuse workspaces; integrate scoped durable worker |
| Catalog publication | Publish route | Reuse versioning; add preflight impact and operation reconciliation |
| Declarative UI intent | Experience-fragment compiler | Reuse supported forms/cards/selection/custom components; inspect fallback result |
| UI action and attachment transport | A2UI rendering process | Reuse capture/upload transport; verify business completion separately |
| Multimodal authoring input | No complete attachment-to-brief contract in inspected generation path | New normalized attachment adapter and evidence extraction |
| Test/evaluation evidence | Existing tests and runtime test interfaces | Integrate version-linked scenarios; new authoring benchmark |
| Cross-artifact resume | Generation and builder each have local lifecycle | New persisted operation/dependency journal |
No row establishes a universal provider × channel × capability combination. The capability response must include the exact supported combination and evidence level.
Behavior selection and concrete configuration¶
Use existing deterministic mechanisms when the requested experience depends on required steps or business rules. “Conversational design” describes how the author works; it does not require every generated behavior to be model-directed.
In this table, stage.config means a service stage's configuration. Resolve the current schema and runtime support before writing these fields.
| Need | Existing configuration seam | Required design/test evidence |
|---|---|---|
| Flexible specialist conversation | stage.config.topics.topics[], engine: llm, routing description/instruction |
Topic entry, refer versus transfer, correct return behavior |
| Fixed collection/confirmation sequence | Topic engine: scripted and script.steps |
Auxiliary services, capture mapping, retries/escape, handback |
| Supervised flexible procedure | Main/topic procedure, authored step blocks and per-step settings |
Completion evidence, dependencies, unavailable judge, irreversible-action gate |
| Typed customer facts | Root variables[] |
Mutable versus linked source, description, validation, capture source |
| Conditional prompt text | Existing #when and template token mechanisms |
Real variable names and effective-prompt rendering |
| Deterministic rules/guards | stage.config.logic.rules / response_guards |
Trigger/condition/action, supported enforcement, fail behavior |
| Tool availability and data mapping | stage.config.tool_logic[resolvedToolName] |
available_when, input bind, output capture, error response |
| Progress during a tool call | Tool logic progress_message |
Emitted after the gate passes; no false success claim |
| Reusable capability | capability:<slug> tool reference and capability_configs |
Manifest/config compatibility, exact version provenance, expansion |
| Provider connection | Stage integration_id plus service definition |
Actual eligible integration and runtime credentials resolution |
| Business auth/payment action | Built-in reference plus builtin_tool_configs |
Tenant/stage gates and supported secure capture path |
Evidence: pipeline types, Agent Logic, scripted subagents, and procedure implementation notes.
Smart topics use the main model's conversation context; do not represent them as arbitrary nested saved-agent references. Guided topics use the existing step machine and can combine spoken, DTMF, and form capture where the runtime supports it. Check their auxiliary STT/TTS/matcher prerequisites and channel eligibility.
Procedure documentation contains an earlier design followed by implemented deviations. The implemented compiler keeps procedure_gate separate from authored available_when; generation authors procedure content through existing draft-save tools. Do not add a fictitious procedure-generation MCP tool based on the earlier design section.
For the support-intake vertical slice, compare a Guided collection step with a Smart topic plus explicit field/action gates before creating a new interaction engine. Keep the user's natural corrections and optional form synchronized through the existing capture model. Live guided/runtime verification remains part of the experiment.
Specific findings that affect implementation¶
Generation is a useful foundation with a narrow execution scope¶
The runner guidance already directs the model to read documentation, inspect the draft, search templates/tools, plan, ask questions, validate, and save a draft. Its generation CLI allowlist permits reading and MCP tools, not arbitrary file writing or shell execution.
Therefore, component source generation needs an explicitly scoped builder operation. Simply adding “create a component” to the existing generation prompt cannot provide that ability.
The documented generation runner currently implements claude_code, including configured authentication variants. OpenAI, Nova, Gemini, and other runtime-agent providers do not thereby become available authoring runners.
Tool guidance and validation disagree¶
The runner's generated guidance asks for def handler(...) and permits realistic sample data without a backend. The canonical python_sandboxed check in generation tools requires config.source containing async def run(args, context). Legacy Python has a different compatibility path.
Correct the generated instructions, bundled best practices, examples, and validator diagnostics together. Replace unconditional sample-data advice with the explicit simulation contract in tool design.
Validation is not reference resolution¶
validateAgentConfigPayload warns when a stage has no matching service definition and skips its subsequent per-service checks. For ordinary saved-tool strings, it warns that existence must be confirmed; that loop does not resolve the ID against an authorized tool record.
The offline probe confirms that an invented saved-tool string is accepted with a warning. Common validation still rejects the probe's unknown top-level service; the additional walker's warning does not mean all unknown services pass.
Add server-side reference resolution and channel/provider eligibility checks before a draft is considered test-ready. Keep an unresolved design intent outside executable config.tools. Warnings alone cannot establish readiness.
Translation needs an explicit traversal audit¶
The generation validator's additional stage walker starts at config.pipeline. Current translation types also contain config.translation.flows.incoming/outgoing.pipeline. The common pipeline validator inspected in this review traverses top-level sequences and legacy speaker sequences.
The offline probe confirms that unknown services inside these translation-flow fields pass generation validation in its synthetic configuration, while an unknown service in a top-level chat stage is rejected. This is not a claim that every translation rule is absent throughout the repository. Trace the remaining validators and runtime builder, and share a canonical stage iterator where their intended coverage is the same.
Search provides a baseline, not broad semantic discovery¶
Generation searches largely filter names/descriptions, with differing tag handling by asset type. Discovery for capability packs, editable catalog source, worked examples, brand identity, and exact compatibility needs a common result contract.
Measure simple lexical search with normalized names, descriptions, tags, and facets before adding embeddings. A relevant-sounding result with the wrong channel or no usable source is a retrieval failure even if its text similarity is high.
Catalog metadata is not a business integration¶
The packaged catalog-component guidance includes tool declarations. The runtime process explicitly documents dormant catalog data-tool wiring.
Audit and rewrite the guidance around the actual render and action contract. Do not equate a YAML tool declaration, a disabled submit button, or resolved action transport with a completed backend operation.
Existing source builders need durable orchestration¶
The build route validates a folder-scoped session, resolves paths, streams a child process, and terminates that process on connection close. That is useful workspace isolation, but not a persisted conversational job with restart/replay guarantees.
Wrap it or adapt its worker logic behind a durable job interface. Do not infer completed work from a closed SSE connection. Inspect source/output hashes and build results.
Publication can affect other agents¶
The catalog publish route returns agents whose published references float via a missing pin or latest. Concrete version pins are unaffected by that publish.
Generate into an isolated catalog or explicitly assess shared impact before publishing. Record catalog ID, concrete version ID, source hash, manifest, and agent binding. Do not use a display semver as a substitute for the runtime's resolved reference.
Payments require channel-specific eligibility¶
The current payments process documents secure capture mechanisms and a remaining typed-card-number gap. Do not advertise generic “payment support” from the presence of a built-in alone, or generate a normal A2UI form for card numbers.
Read the tenant configuration and current verified capture path. Keep unsupported paths unavailable to the authoring compiler until their acceptance checks pass.
Proposed capability descriptor¶
Return this information from a shared discovery layer, using actual IDs and schema versions:
identity: kind, id, version, tenant/scope, source_hash
purpose: experience benefit, example intents, evidence references
compatibility: agent kinds, channels, provider/API family, runtime versions
requirements: integrations, permissions, feature flags, variables, source assets
contract: input/output schema refs, config paths, failure modes
availability: platform_supported, tenant_configured, author_authorized, test_verified
readiness: reusable | adaptable | blocked | unknown, with reason codes
Keep unknown distinct from false. An inaccessible result should expose no confidential metadata. A missing permission should identify the permitted next action without revealing another tenant's asset.
Drift detection¶
Build the guidance manifest from schema/service definitions, node manifests, prompt injections, and curated capability descriptors. Hash these inputs along with examples and provider-guide versions.
On change, regenerate affected references, run contract fixtures, and mark behavioral evidence stale for impacted combinations. An unchanged model name does not guarantee an unchanged provider API or effective prompt. Require live service/tenant checks when binding, even if a retrieved template previously passed.