Conversational authoring implementation research¶
Status: Draft | Last reviewed: 2026-09-10
Build conversational authoring on the existing generation and editor-assistant paths, with a persisted design brief, capability discovery, explicit tool readiness, and a scoped component-building worker. Read this research to turn the spike plan into implementation work for Voice, Chat, Translation, and Background Agents.
The typical author is a functional or business user. All skill packages inherit the plain-language policy: discuss business needs and practical outcomes, resolve implementation choices internally, and provide technical detail when requested.
This pack remains a research recommendation. The first live implementation now reuses the editor-assistant engine for four-type draft creation, durable conversations, structured output and revision-checked saves. Comparative model evaluations, isolated-tenant catalog publication, and real voice/runtime experiments remain unexecuted. The feasibility record states that boundary explicitly.
Research pack¶
| Document | Use it to |
|---|---|
| Capability and integration evidence | Distinguish implemented building blocks, integration gaps, and unknowns |
| Requirements, runtime selection, and generated evaluations | Discover security needs; select cascade/half-cascade/S2S, behavior controls, and agent-specific tests |
| Authoring protocol and guidance | Implement discovery, questions, provider guidance, and four agent-specific conversations |
| Tool and multimodal contracts | Design placeholders, system binding, attachments, branded components, and action completion |
| Designer prototype iterations | Review the conversation-led output panel and plan grounded decisions, four agent paths, connections and test evidence |
| Evaluation and delivery | Reproduce local checks, run the remaining experiments, and sequence the backlog |
| Primary-source research | Check the external evidence, applicability, and source-version policy |
| Evaluation fixtures | Seed repeatable authoring scenarios; these are inputs, not recorded results |
Review of the original plan¶
Keep its separation from the Studio redesign, evidence-first capability mapping, search baseline, complete catalog-to-runtime trace, and explicit recovery experiment. These address real gaps in the inspected code, rather than assuming a better chat interface fixes authoring.
The conversational designer itself now has an explicit prototype iteration track. Its conversation-led interface is implemented in the current app; subsequent demonstrations and backend work are listed separately. The broader visual rollout remains independent.
Extend the plan in these areas:
- Use four explicit authoring contracts. Background Agents retain workflow semantics. Translation retains directional audio flows. Neither should be compiled as a generic chat assistant with a different prompt.
- Represent incomplete integrations. An unresolved action needs a typed design artifact and an honest simulation path, rather than an invented saved-tool ID or sample-returning production tool.
- Separate four multimodal concerns. Design inputs, voice interaction with the designer, generated customer interfaces, and the generated agent's model modalities have different runtime and permission requirements.
- Validate references and behavior. A structurally valid configuration can still contain unavailable services, unresolved saved tools, or a form with no business action.
- Persist decisions independently of a model session. Questions, assumptions, artifact revisions, dependency readiness, and test evidence must survive runner restarts and manual edits.
- Evaluate the effective prompt. Templates and provider advice are only part of the prompt; tenant instructions and capability injections also affect behavior.
- Make runtime and control selection explicit. Apply security/compliance constraints before comparing latency and conversational quality; generate a requirement-linked test suite for each produced agent.
The original completion criteria remain appropriate. Do not mark the entire spike complete from this desk research or convert the estimates below into delivery commitments before the vertical slices run.
Recommended architecture¶
Use one authoring coordinator in ai-manager, backed by its existing PostgreSQL tenant boundaries. It owns the brief, dependency graph, and operation journal. Use the existing runner for reasoning and target-specific generation, with a narrow adapter to the existing builder for source work. Keep the component worker behind a catalog/workspace-scoped job contract.
The coordinator chooses the next useful operation; deterministic handlers enforce target type, permissions, revisions, validation, and publication policy. Load agent-type and provider guidance when relevant. Specialist guidance does not require a separate autonomous agent.
This recommendation combines repository fit with the composable-workflow approach described by Anthropic. It is an architectural judgment, not a measured claim that one coordinator always outperforms multiple agents.
| Option | Strength | Cost or unresolved issue | Decision |
|---|---|---|---|
| Extend the current runner prompt only | Smallest initial change | No durable pre-target brief, cross-artifact operation journal, or conflict handling | Use as evaluation baseline |
| Coordinator plus versioned guidance and scoped workers | Reuses present APIs and tenant boundaries; centralizes revisions | Adds state, discovery adapters, and job contracts | Recommended starting architecture |
| LangGraph-backed authoring coordinator | Explicit checkpoints and resumable graph steps | Adds an authoring state model; existing workflow-runtime usage does not provide this automatically | Reconsider if branching/resume complexity warrants it |
| Autonomous specialist agents for every concern | Can explore independent difficult subtasks | More latency, duplicated context, conflicting writes, and evaluation surface | Test selectively after the baseline works |
| Separate authoring platform/service | Independent scaling and provider adapters | Another deployment and authorization boundary | Defer until isolation or scale requires it |
LangGraph persistence is a useful reference for checkpoint semantics. It does not remove the need for idempotent external mutations or correct tenant authorization.
State ownership¶
The following entities are proposed, not current tables or APIs. Store design state outside executable agent/workflow JSON; unknown root fields are not a safe persistence mechanism through collaboration serializers.
| Entity | Minimum responsibility |
|---|---|
| Authoring session | Tenant, author/access policy, target kind or undecided, protocol version, status, current brief revision |
| Brief revision | Outcome, audience, channels, decisions, assumptions, unresolved questions, success examples, source references |
| Artifact reference | Kind, actual ID, base revision, produced revision, source hash, ownership, dependency edges |
| Tool intent | Business action contract, unknown system fields, readiness, fixtures, actual binding if resolved |
| Operation | Idempotency key, request hash, expected revisions, worker scope, attempts, checkpoints, result/error |
| Evidence | Exact artifact versions, environment, real/simulated dependencies, assertions, timestamps, trace references |
Use structured fields for decisions and concise explanations. Do not depend on a stored hidden reasoning transcript. Attachments remain access-controlled objects referenced by ID and hash; store extracted observations with page, image region, or audio time offsets.
Existing generation_jobs and assistant_sessions remain child execution records. Do not make a model CLI session the sole owner of the design. A resumed session reconstructs context from the brief, pending questions, selected evidence, and artifact revisions.
Execution and recovery¶
Progress from briefing through discovery, drafting, and validation to ready_for_test and tested. Suspend the relevant operation for an answer, access, connection, revision conflict, or recoverable failure. Track cancellation separately from successful completion.
Treat readiness per artifact and per scenario. A mocked support action may allow a complete UI demonstration while preventing a connected-service test. A tested agent draft still requires the existing publication action.
The coordinator:
- Captures the brief before a target exists, if the agent type is undecided.
- Resolves type and creates or reuses the existing target-bound generation job.
- Retrieves compatible assets and records why each is selected or rejected.
- Produces a typed change proposal against known draft/source revisions.
- Validates shape, references, tenant eligibility, and relevant behavior.
- Applies a revision-checked change, then records the resulting artifact revision.
- Runs the permitted test or displays the dependency blocking that test.
Give each write a stable operation ID and a request hash. Retrying the same operation returns its recorded result; a different payload with the same key is a conflict. A worker crash after an external write triggers reconciliation against that operation's artifact metadata before any retry.
Use a transactional operation record/outbox when dispatching work from a database change. A cross-service build/publish/bind sequence cannot be made atomic by wrapping only the agent save in a transaction. Persist each completed step and resume from it.
Cancellation stops queued work and requests cooperative termination. It does not imply rollback of an already completed publication. Preserve completed artifacts and show what remains unattached or untested.
Manual editing and collaboration¶
The current editor assistant saves through database mutations and resets the collaborative document; its documentation acknowledges the risk to unflushed edits. Resolve this before enabling simultaneous long-running authoring and unrestricted editor writes.
Introduce an expected-revision contract shared by the API and collaboration persistence:
- Read the base config and collaborative revision after outstanding local edits have been flushed.
- Propose changes by stable stage/node/tool identity, with field paths and old values.
- Apply only if the authoritative revision still matches.
- For disjoint changes, rebase against the current revision and revalidate.
- For overlapping changes, show the specific conflicting fields; preserve both versions until resolved.
A database updated_at check alone cannot protect keystrokes that have not reached the database. Until the shared revision contract exists, use an explicit temporary edit handoff with flush acknowledgement and a bounded lease.
Component source uses the same principle: retain the base source hash/commit, edit in an isolated workspace revision, and merge against the current source. Regeneration must preserve human changes or report a conflict.
Permissions and publication¶
Keep discovery, configuration writes, test execution, component source edits, catalog publication, and agent publication as separate capabilities. Access to an active integration does not imply permission to invoke every operation on it.
The coordinator can request a narrower child token containing tenant, author, allowed operation, concrete target, expiry, and parent operation ID. It cannot turn a generation target token into general workspace access. Recheck current access before resumed side effects; session history does not preserve revoked privileges.
Generation/editor workers continue to save drafts. The external management authoring surface already has distinct publish tools; preserve those authority boundaries instead of treating all MCP callers as equivalent.
For generated catalogs, prefer an isolated tenant catalog and a concrete version reference. Before publishing a reused catalog, evaluate existing floating consumers. An affectedAgents response returned after publication is useful evidence, but arrives too late to authorize its effect.
Present the exact operation and affected artifacts only when existing permission or product policy requires approval. Ordinary questions, discovery, validation, and permitted draft edits should continue without repetitive confirmation.
Smallest useful release¶
Deliver conversational brief capture, questions, grounded reuse, and validated drafts for all four requested agent types. Include effective-policy discovery, a justified runtime/control choice, and a generated test-suite draft. Include tool-intent cards with explicit unknowns, authoring attachments where supported, existing catalog selection, typed business-action mappings, and links into the real editors.
Then add deterministic simulations, scoped component generation, and durable cross-artifact recovery behind their own acceptance gates. A generated visual preview should state whether its data and actions are simulated, connected to a test system, or verified against the intended environment.
Do not make native voice input to the authoring conversation, arbitrary video understanding, a new vector database, or autonomous specialist agents prerequisites for this release. Add them when a measured authoring task benefits and the corresponding adapter exists.