Conversational authoring research spike¶

Status: Draft | Last reviewed: 2026-09-10

Use this spike to plan an authoring assistant that helps a user design a working experience, discover suitable capabilities and examples, and produce coordinated agent configuration and branded multimodal components. The output is a researched implementation plan with feasibility evidence.

Scope and relationship to the redesign¶

The Studio redesign can proceed independently. Connecting its conversation UI to the existing generation APIs is necessary integration work, but does not establish the quality of the authoring guidance or support creation across agents and catalogs. Resolve those questions in this separate spike before committing to the full conversational authoring implementation.

“Skill” here means the product's authoring guidance, discovery resources and tool contracts. Its packaging is an open research question. The export-only designer guide helps a designer modify the prototype; it is not the runtime authoring skill.

Cover initial creation and subsequent changes, including a user moving between conversation and the real editor. Preserve current agent and background-agent semantics. The designer's nested saved-agent canvas is not a technical contract.

This document defines the investigation scope. It does not implement a new skill, change permissions, or establish that the integration works in a deployment.

Research update¶

The implementation research pack reviews this plan and adds a capability matrix, architecture recommendation, authoring protocol, provider-guidance sources, placeholder-tool contracts, multimodal design paths, evaluation fixtures, and a dependency-ordered backlog.

The review explicitly covers Voice, Chat, Translation, and Background Agents. Background Agents retain workflow semantics; translation retains directional audio flows. It separates multimodal design inputs from the generated agent's runtime modalities and customer-facing components.

Security/compliance discovery, cascade versus half-cascade versus S2S selection, rules/procedures/subagents, and generated agent-specific tests are explicit in the requirements and runtime-selection extension.

Repository inspection, 43 selected TypeScript/Python tests, and offline validator probes provide initial evidence. The feasibility record identifies observed validation gaps and the remaining live experiments. Model comparisons and the isolated-tenant catalog-to-agent vertical slice have not run; the full spike's completion criteria below remain open.

Starting evidence¶

These are repository findings as of the review date. Verify deployed availability and current contracts during the spike.

Existing building block	Evidence and implication
Generation protocol and reference bundle	Runner guidance already requires template/tool discovery, planning, questions, validation and draft saves. AI generation describes the generated reference bundle. Evaluate and extend these assets rather than assuming guidance starts from zero.
Authoring discovery and mutations	Generation tools search agent, workflow and prompt templates, discover tools/integrations and inspect service/node definitions. Management AI tools expose generation and target-bound assistant operations. These inspected toolsets do not provide the complete catalog authoring/publication sequence.
Editor assistant and durable work	Editor assistant and management implementation operate against existing targets. A durable pre-agent brief, cross-artifact revisions and coordinated builder jobs still need an explicit design.
Studio builder path	Code-server, its build route, and Studio API provide workspace authoring, preview and publication operations. Establish whether these can be reused by the conversational coordinator within their existing user, tenant and workspace boundaries.
Component guidance and branding	The packaged catalog-component skill covers registration, sample props, tool declarations and semantic branding variables. The Studio API exposes surface configurations for previews. Audit this guidance against current runtime contracts before reusing it.
Catalog publication and runtime	Catalog builder builds versioned bundles; the publish route reports affected published agents with floating references. A2UI rendering documents selection, manifests, interactions and channel behavior. Publication, binding and successful runtime use are separate steps to verify.

Research questions¶

1. How should the assistant guide experience design?¶

Define a question strategy that uses what the user has already supplied, searches before asking for information the system can discover, and asks the next question that materially changes the design. Avoid a fixed questionnaire for every agent.

Investigate how to establish:

The audience, task, desired outcome and evidence of success.
Entry points and channels, voice versus visual interaction, accessibility, language, tone and identity.
The happy path, correction, interruption, missing information, failed tools, human handoff and return from a visual interaction.
Required inputs and their sources: user answers, existing variables, customer context, tool results, uploaded knowledge or tenant configuration.
What is needed to sketch an experience, save a draft, run a meaningful test and publish it. Show unresolved dependencies at the appropriate stage.

Specify when the assistant should challenge a proposed design, explain the consequence and offer an implementable alternative. For example, a request to collect an address should trigger consideration of spoken confirmation versus a visual form, correction behavior and where the resulting address is used. Pushback should help the user decide without forcing routine implementation choices into approval loops.

Produce example conversations for vague requests, well-specified requests, changes to an existing agent and a request that available capabilities cannot satisfy. Record assumptions and decisions so users are not asked repeatedly.

2. How should discovery and reuse work?¶

Inventory searchable agent/workflow/prompt templates, working examples, tools, capability packs, catalog components and their editable source. Separate curated examples from tenant-authored material and examples that only demonstrate UI.

Compare current search with richer retrieval where evidence shows a gap. Evaluate intent, channel, provider, input/output requirements, brand adaptability and runtime compatibility when selecting an example. Do not prescribe a vector index before testing whether existing search is sufficient.

Require provenance: IDs, versions, source references, reasons for selection and what was adapted. Define how to handle an outdated example, incompatible tool, missing source archive, denied access and no suitable match. Retrieved material must remain within tenant/user access and must not override authoring rules.

3. How does it learn the full supported capability range?¶

Build a capability map from current schemas, service definitions, runtime code, editor behavior and verified examples. Cover at least:

Agent types and channels, providers, languages and multimodal availability.
Current Smart/Guided subagents, procedures, variables, prompt logic and guards.
Actions/tools, integrations, knowledge, background workflows and lifecycle hooks.
Customer authentication, payments, transfer/handoff and human supervision.
Branding/surfaces, component events, input capture, testing and versioning.

For each capability record its experience benefit, prerequisites, required inputs, supported combinations, configuration fields, discovery/validation tools and test evidence. Distinguish platform support from what the current tenant has configured and the current author may use.

Use the configuration contract as one input, not proof that a configuration will run. Investigate drift detection when runtime features, editor controls, generated references or packaged skills change. Never treat a prompt promise as implementation of the promised behavior.

4. How should branded multimodal generation work end to end?¶

Trace the complete candidate path:

Understand why a visual interaction improves the experience and its fallback.
Search existing components/catalogs and decide whether to reuse, adapt or create.
Resolve the agent's intended surface identity: tokens, typography and assets. Distinguish customer-facing branding from Studio's own application theme.
Generate or adapt editable component source, registration, props schemas, sample data and the current runtime's event/tool contracts.
Build and preview with the selected identity; exercise loading, error, empty, validation, keyboard and responsive states.
Publish a catalog version under an agreed publication policy.
Attach the correct catalog/component references to the intended agent draft, map props and captured events to variables/actions, and validate the result.
Run the experience through the actual conversation runtime and verify that visual actions lead to the intended business action and conversation response.

Determine which steps can use existing builder interfaces and which need scoped authoring tools or durable orchestration. Compare reuse of the current builder, delegation to a component authoring worker and a shared orchestration service. Keep these as alternatives until tested.

Generating a form does not create its data integration. The current A2UI process explicitly distinguishes components from dormant catalog data-tool wiring. Trace real action capture, authorized tool execution and result handling rather than relying on sample data. Inspect whether the existing experience-fragment compiler helps express intent; do not assume it covers every component or channel.

Verify concrete catalog versions versus floating references, reuse versus cloning, and whether publishing a shared catalog changes other agents. Catalog publication and agent publication have different effects; the current generation protocol saves drafts only. The spike must recommend the review boundaries and recovery behavior before adding cross-artifact publication.

5. What state, tooling and skill structure are needed?¶

Compare a versioned core authoring protocol plus retrieved capability guidance with specialized skills coordinated by a parent author. Evaluate both against the same tasks rather than selecting an architecture from prompt size alone.

Resolve:

Whether a draft exists from the first message or a standalone brief precedes it.
Ownership and revision links among the brief, conversation, agent draft, builder workspace, source commit, brand configuration and catalog version.
The boundary between planning notes and executable configuration.
How AI changes are reviewed and reconciled with manual editor/collaboration changes, including regeneration after a designer edits component source.
Job progress, durable resume, cancellation, retries and duplicate prevention.
Recovery when publication succeeds but binding/testing fails, or a builder job is interrupted. Preserve enough evidence to explain and repair partial work.
Scoped delegation across agent and catalog targets without broadening a target-bound authoring token into unrestricted workspace access.

Feasibility experiments¶

Use a separate test tenant/workspace and explicit test fixtures when executing the spike. Record real versus simulated dependencies in every result.

Experiment	Evidence to collect
Experience coaching	Run the current guidance and candidate skill against the same briefs. Assess missed requirements, useful challenges, unnecessary questions and whether users can explain the proposed experience.
Search and capability grounding	Include a strong reusable match, a misleading example, an unavailable integration and no match. Record retrieved sources, compatibility decisions and any invented IDs/capabilities.
Branded multimodal vertical slice	Design a support-intake agent that collects an issue and preferred contact details. Reuse or generate a branded form, preview it, publish a test catalog, bind it to an agent draft, capture input, call a test support-system action and show/speak the result. Verify a voice-only alternative and tool failure.
Revision and recovery	Change the brand and a required field after a manual edit; interrupt the builder and retry after publication. Verify preserved edits, correct references and recoverable state without duplicate artifacts or unintended changes to another agent.

Agree evaluation thresholds before comparing candidates. Measure task completion, critical requirement coverage, grounded reuse, configuration validity, real runtime success, brand consistency, interaction accessibility, recovery, question burden, latency and cost. A compelling transcript or successful build alone does not demonstrate a working experience.

Deliverables and completion criteria¶

The spike is complete when it produces:

Capability and integration matrix: existing support, verified reuse paths, integration work, net-new backend requirements and remaining unknowns, each linked to evidence.
Authoring skill proposal: question policy, capability discovery/retrieval strategy, source precedence, versioning, example conversations and evaluation fixtures. Include how guidance stays aligned with runtime changes.
Feasibility record: end-to-end traces and artifacts from the experiments, including the branded catalog-to-agent path and observed failures.
Architecture recommendation: compared options, state ownership, tool/API contracts, permissions, publication boundaries, collaboration and recovery.
Implementation backlog: dependency order, smallest useful release, separate frontend/backend/content/evaluation work and estimates justified by the experiments. List unresolved product decisions with their consequences.

Initial areas to investigate for net-new work are structured brief persistence, broader discovery contracts, catalog authoring delegation, coordinated job/revision tracking and evaluation orchestration. These are candidates, not a claim that every area needs a new service or that existing APIs already satisfy it.

Keep the immediate brand/layout rollout separate. Evolve the conversational designer in the current app through the prototype iteration track, using the spike's findings to define its behavior, release scope and acceptance checks. Clearly distinguish demonstrated interactions from working authoring capabilities.