Director application API¶

Status: Current | Last reviewed: 2026-10-20

These authenticated ai-manager application endpoints use the active tenant. They are separate from the public /api/v1 integration and management APIs. FEATURE_SCENE_AGENTS must be enabled. Reads require AIManager:RunAgents; mutations require AIManager:ManageAgents, with matching API-token scopes.

Method and path	Contract
`POST /api/director/scenes`	Create, duplicate by `sourceAgentRef`, or promote by `projectId`; returns `{id,agentId,href}`
`GET /api/director/{id}`	Read `{id,agent_id,revision,document,updated_at}`
`PUT /api/director/{id}`	Save `{revision,document}`; stale revision returns 409
`POST /api/director/{id}/images`	Multipart `file`, up to 5 MB; returns 201 `{id,url,contentType}`
`GET /api/director/{id}/assets/{assetId}`	Download a project-owned audio/image/video asset; MP4 supports a single byte range
`GET /api/director/{id}/assist?sceneId={chapter}`	List `{agents,conversations,jobs}` for the selected chapter and Assist agent
`GET /api/director/{id}/assist?sceneId={chapter}&jobId={batch}`	Read `{job}`; a completed valid capture also returns `{moments,assist}`
`POST /api/director/{id}/assist`	Generate, capture or cancel an Assist take as described below
`POST /api/director/{id}/audio`	Generate a take for JSON `{momentId,revision}`, or upload multipart recorded audio
`GET /api/director/{id}/video`	List `{jobs}` for the project, including pending and completed generations
`POST /api/director/{id}/video`	Generate/refresh a video job, or upload raw video, as described below
`GET /api/director/catalogs`	Tenant/shared published Agent Assist and multimodal components, including `catalog_type`, pinned schema and `emits`; `?versionId={id}` returns a pinned renderer manifest
`POST /api/director/{id}/exports`	Snapshot and start an MP4 render; returns 202 with its export job
`GET /api/director/{id}/exports`	List export jobs and progress
`GET /api/director/{id}/exports/{exportId}`	Download the completed MP4

The bounded version-1 project schema lives in lib/director/model.ts and workspace-model.ts. A chapter has an optional stage: legacy/default voice_call, agent_assist with app: salesforce|headless, or a personal assistant stage. Documents may be up to 2 MB, with 40 chapters, 200 moments per chapter and 500 moments overall. Voice-call stages accept optional showActivity: boolean, overriding the project's activity setting for that chapter. Omission inherits the project value.

Each chapter also accepts optional transition: {type: "cut"|"crossfade"|"fade"|"slide", duration: 0..3}. Omitting it preserves a cut. The first chapter's incoming transition is ignored. A positive non-cut duration inserts a silent interval between complete chapters; moment at values remain relative to incoming content start. Transition seconds count toward the 600-second export limit. Compiled scenes expose transitionStart, transitionEnd, start and end; start=transitionEnd. The project also accepts optional closingFadeDuration: 0..3. A positive value appends a fade to black after all chapter content, followed by 0.25 seconds of black. Compiled closingFade: {start,end} bounds the fade; total duration includes the black tail. Omission or zero keeps the existing ending. Both intervals count toward the export limit. Uploaded image URLs must name an asset owned by the same project; duplication copies bytes and rewrites those references.

Personal assistant sessions and audio¶

The optional assistant configuration holds names, portrait, session ID, six-digit code, website, queue note and resolution cards. Chapters also accept personal_assistant, assistant_call, or assistant_browser stages. Salesforce stages can use preset: "ralph_lauren", personalAssistant: true, records.messagingSession and a conversation section. Moment kinds include assistant, browser, chat and dtmf, each with a same-named typed payload. See the session contract.

The browser payload {event:"connect"} accepts optional clickDelay and connectingDuration (each 0–30 seconds). Chat becomes connected after their sum; the moment's duration must cover that interval. Omitted values retain instant connection. Preview presents the enabled button, press and connection progress through the same reducer used by export.

POST /api/director/{id}/audio accepts either JSON {momentId,revision} for speech/preset/DTMF generation or multipart fields momentId, revision, and file for a sound with audioSource: "upload". Uploaded audio is bounded to 8 MB and 120 seconds and normalized to mono 24 kHz WAV. Both paths require the saved revision and return {take}; save the returned take association in the project before playback/export. Stale revisions return 409. Uploaded captions can change without invalidating the recording.

Speech voices can use provider: "polly", an active tenant Amazon integrationId, voiceId, region, language, and engine: "standard"|"neural". Generation validates that the requested voice supports the engine. Existing voices with no provider field continue using ElevenLabs. Sound presets include phone-ring-v1 and hold-music-v1; DTMF uses authored digits/press/gap timing. The optional project soundIntegrationId selects the ElevenLabs connection for generated effects independently of Polly speaker connections.

These application routes are documented here rather than in the generated public integration and management OpenAPI documents.

Guided service and video¶

The native service_journey stage contains organization, website, accent and product image and organization logo. A journey moment contains a complete typed screen/policy snapshot, including breadcrumbs, showChat and videoPhase. Assistant call/browser stages can display its participant/call/queue strip. These are authored presentation values, not runtime routing decisions. Each choices[] row optionally accepts an owned imageUrl. Image choices use visual product tiles, and their references participate in save validation and duplication just like the stage logo.

browser.event=open_chat starts an ordinary website support conversation without an IVR pairing code. chat.event=widget accepts a title, detail rows, selectable options, action label and pending/complete status. These values replay from the authored timeline.

assistant.event=message with sender=assistant,thinking=true displays a thinking bubble for the moment's duration and delivers the reply at its end. Omitted thinking preserves existing immediate messages. assistant.event=notification shows the native phone banner with the assistant's app identity without opening the app; an explicit phone.screen=assistant opens it. A personal-assistant stage can choose conversationScope=chapter for a fresh conversation, or all to continue its history. Narration can anchor to a visual moment's start, keeping actions moving while the voiceover plays.

customer_video reuses the journey stage's branding fields and the native phone browser shell. Its journey phases are portal, permission, preview, waiting, connected, rear, paused and ended. Paused customer video keeps audio connected and displays the contact profile. In workspaces with records.videoCall, both preview and export route call speech by these phases: connected, rear and paused are audible; pre-join and ended states are silent. Named video participants do not use telephone consultation-leg routing. Narration stays audible independently. Camera access is a presentation state, not a browser permission request. Personal-assistant stages optionally accept clockStart (HH:mm) and dateLabel for an authored time transition.

Agent workspaces support optional records.videoCall, tabs referencing that record, type=video center sections, and section action:{label,tab} buttons which select an existing record tab. Return forms still use normal fill_section and form_submit events. No live Salesforce operation occurs. An assist section's optional guidance:{kind,source} selects a native rich card (recommendation, coaching or completed) with supporting policy/context source, authored detail rows and the same record-navigation action.

A video moment contains generation settings, immutable owned takes, takeId, trim boundaries, viewport placement, display overlay and source-time event markers. aspectRatio selects 16:9 or 9:16; surface selects stage or video_call (customer camera) or representative (the specialist's camera). The two video surfaces play independently from the same Scene clock. loop repeats the trimmed clip while that camera is shown; the Alder camera check uses a short, non-looping trim and joins during playback. reviewedTakeId records which take's markers were reviewed.

Video POST accepts:

JSON {action:"generate",momentId,revision,operationId}. The operation ID is a UUID. A matching saved revision is required. Returns 202 {job}.
JSON {action:"refresh",jobId}. Returns {job} and collects completed provider output without creating another generation.
Raw video/* bytes with X-Director-Moment and X-Director-Revision headers. Returns {take}. Save the association in the project.

The route requires ManageAgents and a 12/minute limit. GET requires RunAgents. Generate uses the moment's active tenant ElevenLabs integration, never browser-provided credentials. References must be project-owned PNG, JPEG or WebP images. Duplicate active/source or operation requests reuse the saved job. Reopening the editor lists and resumes pending jobs. Completed jobs retain their take without changing the document's selected take.

Uploads are bounded to 100 MB, 120 seconds and 4K and normalized to silent, seekable H.264 MP4. This route bypasses Proxy body cloning and authenticates at its withAuth boundary. Invalid input returns 400, stale revisions 409, and oversized uploads 413. Asset GET returns 206 for a valid single range and 416 for an unsatisfiable range. Provider job metadata is not downloadable as a media asset. Duplicate Scenes remap video/reference/review IDs and omit generation jobs.

Manual cards and component events¶

Manual cards require no Assist agent or /assist request. Revision-save them as chapter catalogue sections or kind: "component" moments. Definitions contain catalogId, versionId, name, props, schema and optional dataModel, emits, interactions. Metadata comes from the published source_manifest. Timed cards accept componentMode: "append"|"replace"; omission retains replace. Phone card moments also accept componentEntrance: "none"|"slide". Omission preserves instant display. Slide fades and moves the card upward over 0.4 seconds, derived from the saved moment start so seeking and export reproduce the arrival.

Each interaction has id, trigger and response. Triggers are an exact {type:"action",name}, {type:"click",label} or {type:"proxy",capability,action}. Responses contain outcome: "success"|"error", delay: 0..10, error, and optional JSON result, props, dataModel, toast. Patches merge shallowly; the result resolves the component's simulated request.

A kind: "component_event" moment contains componentEvent:

{
  "type": "action",
  "target": { "type": "section", "id": "review-card" },
  "action": {
    "actionName": "copilot.form.complete",
    "context": { "confirmed_fields": [] },
    "dataModel": {}
  },
  "click": { "label": "Confirm details" }
}

Targets are typed section or moment IDs in the same chapter. Other event types are click, proxy, props, dataModel, fields, toast and reset. Each supplies its corresponding payload; reset needs only its target. A standalone toast omits the target and provides {variant: "info"|"success"|"warning"|"error", title, message, duration: 0.5..15}. Response toasts use the same shape. Schemas, size limits and deterministic projection live in lib/director/component-interactions.ts. Card events and responses replay without live agent, tool, proxy or conversation writes.

Optional Assist generation and capture¶

Save the chapter's selected Assist agent and context before generating:

{
  "action": "generate",
  "sceneId": "chapter-id",
  "agentId": "agent-id",
  "version": "published"
}

The response is 202 with {"job":{"id":"batch-uuid","status":"queued"}}. version may also be draft. Copilot replay uses the saved human dialogue, record context and mock host context. Narration is excluded; up to 30 human lines are accepted. Real Copilot model workers run with validated tool mocks. The request cannot supply a runtime target, credentials, or disable the sandbox. Generate/capture are rate limited. Repeating an identical active generation returns that run; different inputs while a run is active return 409.

Poll the chapter-bound job. Job statuses are queued, starting, active, completed, failed or cancelled; error and conversationId are optional. Only completed output returns importable moments. Replace that chapter's prior kind: assist moments with the returned moments, apply the returned assist stage configuration, and revision-save the project.

Each captured moment contains actual normalized A2UI events, pinned catalogue versions, source conversation ID and input fingerprint. Input changes while a job runs return 409 at completion. Changes after import mark the take stale and block export. Application restarts do not discard durable generation jobs.

Capture UI output from an existing conversation owned by the selected agent:

{
  "action": "capture",
  "sceneId": "chapter-id",
  "agentId": "agent-id",
  "sourceConversationId": "conversation-uuid"
}

The response is {moments,assist}. Existing-conversation capture preserves relative UI-event timing and pins accessible catalogue versions at capture time. It does not infer historical renderer versions that the source did not save. Missing output, malformed data, inaccessible catalogue dependencies and oversized captures are rejected.

Cancel with {"action":"cancel","sceneId":"chapter-id","jobId":"batch-uuid"}. Cancellation fences unfinished jobs and their runtime callbacks. Job IDs are checked against tenant, project and chapter before reading or cancelling.

See Director implementation for state projection, audio routing, rendering, source fidelity and local verification.

Cursor annotations accept optional fromTarget, moveDuration (0–10 seconds) and clickAt (0–120 seconds relative to cue start). Geometry uses the saved timeline and smooth interpolation; click feedback lasts 0.45 seconds. The cursor is presentation only: author the matching workspace/component event separately. Targets must remain visible for the full cue. Missing optional values preserve the original static cursor.

Other cues accept easeDuration (0–5 seconds) for entrance/exit easing. Spotlights additionally accept dimOpacity (0–0.8) and optional focusScale (1–2) for a synchronized zoom. Omitted values preserve existing presentation. Start/end timing anchors may reference any other moment in the chapter; word anchors require speech. References cannot be circular. The compiler resolves chained anchors in dependency order.

Salesforce stages support preset allianz. Header actions additionally accept complete (Mark Complete), which reveals the wrapup section. Sidebar verification is an optional bound string. A section may set presentation: "modal" to appear over the console; visibility uses the existing section_visibility event. Omission retains normal panel rendering.

A sound moment may set soundPreset: "phone-ring-v1". POST .../audio then prepares the standard recorded ringtone as a normal project-owned take without resolving an ElevenLabs connection. The preset version and authored duration identify that take; duration changes invalidate it. The optional field leaves existing generated sound fingerprints unchanged. New editor sound moments use this preset and a 4.5-second duration. The source repeats every 2.5 seconds, including a 0.5-second silence, and stops at the chosen duration.

Other sound generation requests a self-describing encoded asset and decodes its channels/sample rate to mono signed PCM at 24 kHz before saving the WAV take. Exports have a 45-minute render deadline to accommodate the supported 600-second desktop timeline at 24 fps.

Director application API¶

Personal assistant sessions and audio¶

Guided service and video¶

Manual cards and component events¶

Optional Assist generation and capture¶

Cursor motion and Salesforce modal presentation¶