Conversational tool and multimodal contracts¶
Status: Draft | Last reviewed: 2026-09-10
Use these proposed contracts when the designer needs an unconnected business action or a branded visual experience. They distinguish design intent, executable configuration, simulation, and verified business results. The proposed fields and operation names below are not existing API contracts.
Discover the system before implementing an action¶
Model an operation in business terms first: “create a support request,” “look up an order,” or “reserve a callback.” Search authorized tools, integrations, capability packs, and workflow tools before creating another implementation.
The current addable tool families are sandboxed Python, API/OpenAPI, MCP, A2A, and Knowledge. Prefer a verified saved tool or capability pack. Use API/OpenAPI for a known endpoint, MCP/A2A for an authorized remote contract, and sandboxed code for a real transformation or supported integration that needs it. Preserve legacy references when editing, but do not generate new legacy types. Host Actions have a separate host-application contract and are not generic LLM tools.
Ask only for missing information that affects its contract:
| Concern | Information to establish |
|---|---|
| System and ownership | Which system owns the record; who can establish its connection |
| Environment | Test versus production; intended tenant/account |
| Operation | Read/create/update/delete; synchronous versus accepted-for-processing |
| Inputs | Required fields, source variables, identifiers, validation |
| Output | What proves success; record ID, state, display fields |
| Authority | User/session identity, allowed scope, confirmation/business limits |
| Failure | Not found, invalid input, unavailable, unauthorized, ambiguous completion |
| Reliability | Timeout, rate limits, safe retry, idempotency, compensation |
A name such as “Salesforce” does not establish the object, fields, access scope, or permitted operation. An OpenAPI document may help derive them, but its existence does not establish authentication or successful execution.
Do not ask the user to paste credentials into the authoring conversation. Use existing secure integration setup and store references. MCP elicitation is an optional adapter for clients that negotiate the capability; it does not replace the product's existing ask_user flow.
Tool-intent lifecycle¶
Use a design-side record with explicit readiness:
| State | Meaning | Allowed behavior |
|---|---|---|
unbound |
Business contract exists; no executable binding | Explain/design; no business execution |
mock |
Deterministic fixtures implement the contract | Isolated simulation, clearly identified |
connected_unverified |
An authorized real tool/integration is bound | Permitted test execution; no claim of verified behavior |
verified |
Contract scenarios passed for a specified binding/version/environment | Eligible for the corresponding release policy |
blocked |
Access, incompatibility, missing source, or another dependency prevents progress | Preserve work and show the next concrete dependency |
A connection/config/schema change invalidates the relevant verification evidence. “Verified in a test account” must never imply “verified in production.”
Do not add type_id: mock, fake saved IDs, dummy API URLs, or unknown metadata fields to existing executable tool schemas. Introduce a dedicated design artifact first; implement an explicit test adapter before allowing simulations to execute.
Example placeholder contract¶
This example uses authoring-local identifiers and JSON Schema. It is intentionally not an agent configuration or an executable tool definition.
{
"schema_version": 1,
"intent_key": "support.create_request",
"state": "unbound",
"purpose": "Create a support request after the user confirms the details",
"system": {"name": null, "integration_id": null, "environment": "test"},
"side_effect": "create",
"input_schema": {
"type": "object",
"properties": {
"issue": {"type": "string", "minLength": 1},
"contact_email": {"type": "string", "format": "email"},
"request_key": {"type": "string", "minLength": 1}
},
"required": ["issue", "contact_email", "request_key"],
"additionalProperties": false
},
"success_schema": {
"type": "object",
"properties": {
"request_id": {"type": "string"},
"status": {"enum": ["created", "accepted"]}
},
"required": ["request_id", "status"],
"additionalProperties": false
},
"error_codes": ["invalid_input", "unauthorized", "unavailable", "unknown_outcome"],
"retry_policy": "reconcile_by_request_key_before_retry",
"binding": null,
"fixture_cases": ["created", "invalid_email", "timeout_before_write", "timeout_after_write"]
}
Validate format and business constraints server-side; JSON Schema format annotations are not automatically enforced by every validator. Adapt this schema to the selected model API's supported subset without weakening the authoritative contract.
Generate fixtures for success, empty/not found, validation error, authentication failure, timeout, duplicate request, and ambiguous completion as relevant. A created fixture and an accepted fixture must produce different agent language.
Simulation implementation¶
Prefer a deterministic fixture service or runner adapter over free-form model-generated fake results. The executor records trusted metadata separately from the business payload:
{
"execution": {"mode": "simulation", "fixture": "created", "side_effect_performed": false},
"result": {"request_id": "SIM-1042", "status": "created"}
}
This is a proposed adapter envelope. A tool cannot make itself “real” by changing a field in its returned JSON; execution provenance comes from the server-selected adapter.
If reusing sandboxed Python for fixtures, use the canonical async def run(args, context) contract and enforce test-only execution independently of the code. Limit the fixture worker's network, credentials, and writable scope. A disclaimer in the prompt is insufficient.
Block mock/unbound actions at production publication and at production execution, including when referenced through a capability pack, inline tool, workflow, or copied draft. These enforcement points are new work. Until they exist, keep placeholder contracts non-executable.
Display simulation status in the test transcript and action result. The agent should say a request “would be created in this demo,” not claim it has booked, charged, sent, or updated something.
Binding and promotion¶
Resolve an intent to an actual saved tool, inline tool, capability-pack tool, or workflow action. Validate:
- Same tenant and current author/runtime authority.
- Input, output, errors, and side-effect semantics match the intent.
- Credentials remain in integration/secret storage.
- Required variables and auth state are available at execution.
- The intended runtime supports the tool type and invocation pattern.
- Contract tests pass for the exact binding and environment.
If a backend cannot provide idempotency or reconciliation after an ambiguous write, do not blindly retry. Route that case to an unresolved-status response or human review. “Request accepted” is not proof of completed work.
Use MCP tool schemas and errors as protocol guidance. Treat external tool annotations as hints, not authorization or proof of harmlessness.
Four multimodal layers¶
| Layer | Example | Implementation implication |
|---|---|---|
| Design evidence | Screenshot, PDF process guide, logo, sample call | Extract observations into the brief with provenance |
| Authoring interaction | Speaking to the designer | Transcription/voice transport feeding the same session; optional separate adapter |
| Generated customer interface | Form, choices, photos, status card | A2UI/catalog/surface contract plus actual action mapping |
| Generated model input/output | Image understanding, native audio, video frames | Requires a compatible provider service and TrueServe runtime adapter |
Supporting one layer does not establish the others. A screenshot can inform branding even when the generated phone agent has no vision input.
Attachment-to-brief pipeline¶
Add tenant-scoped uploads with size/type limits, controlled storage, hash/version, and access checks. Treat documents and media as untrusted task data.
Use a provider-compatible extraction worker to produce observations, candidate requirements, uncertainty, and source locations. Retain document page references, image regions, or transcript timestamps so the author can correct an inference.
Examples:
- Screenshot: identify structure and visible branding; do not infer hidden backend behavior.
- OpenAPI/specification: extract candidate operations and schemas; do not execute embedded instructions or arbitrary endpoints.
- PDF policy: distinguish policy text from suggested agent behavior and preserve citations.
- Sample call: extract turn-taking and failure examples; make transcript uncertainty visible.
- Logo/brand guide: resolve an approved asset and tokens; do not treat approximate visual color extraction as canonical identity.
A model's ability to accept media is only one dependency. Verify transport, storage, renderer, tool, and runtime support before offering that modality to the generated agent.
Branded component path¶
Prefer existing primitives/components when they meet the task. Use the experience-fragment compiler for its supported intent shapes; inspect reported unsupported components and fallback text. Do not silently turn a required interactive form into a static markdown fallback.
For new or adapted components:
- Select the business interaction and voice/text fallback.
- Resolve the intended customer
surface_config, token revision, typography, and approved assets. - Discover a compatible catalog/component and editable source archive.
- Reuse a concrete version, or create an isolated source revision for adaptation.
- Produce registration, props schema, sample props, event contract, and loading/error/empty states.
- Build and preview through a narrowly scoped worker.
- Exercise validation, keyboard use, focus, error text, responsiveness, and repeated submissions.
- Publish the intended test catalog version under the applicable policy.
- Bind that concrete version to the draft and map captured inputs to variables and actions.
- Test the whole runtime interaction, including failure and voice fallback.
A2UI's primary repository distinguishes declarative UI descriptions from trusted client components. Generating component source adds a code-build boundary; do not present arbitrary generated code as safe merely because the runtime uses A2UI.
Action completion contract¶
Track rendering, input capture, validation, dispatch, authorization, execution/reconciliation, result capture, surface update, and the agent's response separately.
In the inspected runtime, custom-component dispatchAction completion acknowledges transport through the surface consumer. It is not by itself proof that a ticket exists in the external system.
Map a component event to declared fields, then to a specific business action. Prefer existing declarative variables/tool logic where supported; identify any missing deterministic transition explicitly. Do not rely on the model noticing an arbitrary button label.
Keep entered data after recoverable failure. Disable duplicate submissions while an operation is pending, but enforce idempotency on the server too. Only show business success after the action's success contract is satisfied.
The existing attachment path uploads photo bytes separately and passes storage references through actions. Preserve that separation; do not put large base64 payloads in prompt text or UI-action metadata.
Use text that identifies the invalid field and the correction, following W3C's error guidance. Test with keyboard and assistive technology; an automated accessibility scan alone is insufficient.
Proposed worker operations¶
| Operation | Inputs | Required result |
|---|---|---|
| Discover component | Intent, channel, available catalog scope | Compatible candidates, exact versions, source availability |
| Prepare source revision | Target catalog, base hash, brand revision | Isolated workspace/revision, allowed paths |
| Build/preview | Source revision, operation key | Build status, manifest/hash, preview reference, diagnostics |
| Test component | Build hash, fixture scenarios | Assertions, screenshots where useful, failures |
| Publish catalog | Exact build/source, policy decision, operation key | Actual version ID and impact result |
| Bind artifact | Agent draft ID, expected revision, exact catalog version | Updated revision or conflict |
All operation names are conceptual. Reuse existing API services behind adapters instead of inventing parallel build/publish implementations.
On publish-success/bind-failure, retain the published version and retry only the binding after revision reconciliation. Do not republish automatically or delete a shared version to “roll back.” Track orphaned artifacts for deliberate cleanup.