Personal assistant Scene journeys¶

Status: Current | Last reviewed: 2026-10-20

Use this guide when extending or debugging Scene journeys that follow a personal AI from its owner's phone into a contact-center call, browser chat and human Messaging Session. The editable Ralph Lauren return starter contains nine chapters with fictional customer data and authored return options.

Stages and shared state¶

The version-1 Director document accepts an optional assistant configuration. It stores the assistant portrait and names, owner, contact center, session ID, six-digit code, service website, queue explanation and permitted resolutions. The existing agent_assist stage also accepts personalAssistant: true.

Stage	Presentation
`personal_assistant`	Native phone home screen and a dark personal-assistant chat
`assistant_call`	Assistant and contact-center identities, transcript and native keypad
`assistant_browser`	Standalone browser, session-code entry and service chat
`agent_assist`, Salesforce	Messaging Session, conversation, context sidebar and Agent Assist

The Salesforce ralph_lauren preset adds records.messagingSession, a conversation section and a Conversation Wrap Up form matching the DCC Call Wrap Up used by the Allianz Scene: case reason, status and intents on the left, a bulleted conversation summary and recommended actions on the right, and Add Task / Mark as Complete below. It uses the reference's 30% left column, 24-pixel gap, Arial typography, list spacing and footer controls. The recommended action is “Flag this conversation as initiated by a customer's AI assistant”; its checkbox is captured as wrapup.flagPersonalAssistant UI state. The form retains the existing fill_section and form_submit record bindings. Salesforce uses its own Enhanced Conversation presentation over the same history as browser chat: one Conversation header, Active/Ended state, End Chat, incoming/outgoing bubbles, author/timestamp/read metadata and a reply toolbar. Current human-agent messages use the SLDS blue outbound treatment; prior Service AI messages use the dark-gray “another agent” treatment. Incoming messages from the personal assistant are light gray. In the assistant's browser, Ari's outgoing messages are dark gray and customer-care replies are light gray. The shell shares the existing Business Plus/DCC spacing and sidebar styles. The Messaging Session header, icon, Conversation/Details tabs and rounded controls follow the authenticated devsdo Messaging Session reference captured on 2026-10-14. Status, Start Time, Origin and related records appear in Details. Profile and Agent Assist notices explicitly identify the owner's personal AI; editable resolution cards appear above the transfer summary. These are saved presentation records, not a Salesforce connection.

The assistant reducer reconstructs one session from the complete timeline. Phone messages, owner typing, thinking and working status use authored time. The owner's phone composer is a wrapping textarea that grows to six lines as either a recorded or authored message is typed, then scrolls longer drafts. Sending or seeking to a shorter draft restores its height. Enter sends during interaction recording; Shift+Enter adds a line break. The browser only connects after a complete matching code. Ending voice requires a connected chat; queue acceptance and chat end update the same history. Seeking backward reconstructs the previous state without retaining later messages or queue state. Authentication uses the existing native mocked SMS/browser flow. In the starter, the owner-consent chapter begins with a thinking bubble, then the question. The text arrives one second after the owner's reply. Authentication ends at the next call chapter, with a cut and no extra phone-chat acknowledgment.

The personal assistant greets {{customerFirstName}}; the full {{customer}} detail remains available for records. Actions appear only below the portrait. Launching from home grows the selected icon for 0.25 seconds, then opens the app over 0.4 seconds. This is driven by Scene time and disabled while recording a direct interaction so a paused playhead does not leave the app mid-launch. The Virtual Keypad retains synchronized presses and tones. Dialing and pressed digits also appear as transcript events. After Ari identifies itself, the shared card explicitly labeled AI Activity shows a 2.5-second order lookup before the Service AI requests permission to authenticate Alex. The starter expands this event's inputs and completed result. Browser chrome displays the configured site (default ralphlauren.com/chat/a2a) without a voice banner or session badge. The code-entry page omits an additional verified-identity disclaimer. Call and browser identity labels use the owner's first name, for example “Alex’s AI Assistant”. Owner-facing replies refer to “the representative”.

Starter AI chat messages type and send over 1.5 seconds. Ari's browser composer reveals words quickly; the Service AI shows a thinking bubble. The continuing browser-chat chapter uses a cut so the first reply does not fade in. Human replies pause for 1.5 seconds, then type at approximately 5.5 characters per second before sending. ChatComposer shows the draft in a growing multiline composer with a visible caret, holds the completed draft for 0.3 seconds and presses Send during the final 0.18 seconds. The reducer delivers the message only at the moment's end. Returning from the owner's phone shows Ari typing before its decision arrives. A cursor cue visibly clicks End Chat after the final goodbye. The queue explanation is optional and hidden when empty, including in the starter. Human escalation first shows an expanded Omni-Channel offer with name, personal-AI identity, order and reason for transfer for 4.5 seconds. The standard phone-ring-v1 incoming sound accompanies the offer. A saved cursor cue clicks Accept; accepted work remains visible for one second before the utility minimizes. offer_work, accept_work and chat accept remain distinct replay events.

The starter's browser-code moment leaves value empty to use the configured code. Editing assistant settings refreshes the sessionCode reusable detail used by the spoken line and matching Messaging Session numbers. An explicit code on an individual moment overrides that default and can demonstrate an incorrect code.

Browser connect events can set clickDelay and connectingDuration. The completed code enables the button; it presses after the delay, displays Connecting, then opens the linked chat. The reducer reconstructs these phases on seek and rejects chat before connection completes. Omitted timings preserve instant connections. The moment must cover both intervals. The stable assistant.browser.connect target supports a cursor cue whose clickAt matches the connection event's delay.

Preview and export share DirectorStage, the reducer and the saved assets. Assistant scenes render motion frames so typing, key presses and code entry remain visible in MP4. These stages do not place calls, navigate external websites, send texts or operate a real retailer account.

Authored events¶

Moment kind	Payload
`assistant`	`message` with `owner`/`assistant` sender, `thinking`, or `action` with icon
`browser`	`navigate`, `code`, `connect`, or `end_call`
`chat`	`message` with `assistant`/`service`/`human` sender and optional `typing`/`typingDelay`, `queue`, `accept`, or `end`
`dtmf`	Digits, press duration and gap between digits

Message and status text use the normal reusable details. Record interactions captures native app launch, phone messages, keypad clicks, browser code/connect and chat messages as moments at the current playhead. Replay remains authored; it does not invoke an LLM for a reply. For chat messages, Type before sending defers delivery until the moment's end. Pause before typing (seconds) reserves the initial part of its duration; the remainder controls composer typing speed. Messages without typing retain their original immediate-delivery behavior. Both drafts and typing indicators are reconstructed from the playhead, including backward seeks.

The starter covers owner request, IVR, authentication consent and secure text, voice-to-chat handoff, damaged-item review, an 8-human/2-AI queue, human escalation, owner choice and case closure. Adding it preserves existing chapters. A Scene with an existing browser connection cannot append a second complete journey; edit or duplicate that Scene to create another example.

Fictional-brand variations¶

Duplicate the Scene through POST /api/director/scenes with sourceAgentRef to copy the complete journey and give the new project its own audio/image assets. Update the duplicate's assistant organization, website, details, spoken script, authentication sender/link, Salesforce branding, records, sidebar, incoming-work context and wrap-up summary together. The Salesforce preset identifies the layout; its saved branding and record values remain editable. The call emblem uses the first two name initials, skipping standalone punctuation such as the ampersand in Alder & Branch. The wrap-up's optional case.customerIntents field supplies its Customer Intents label. Existing Scenes without that field retain the starter's wording. Regenerate rewritten speech, including lines affected by changed reusable details, and verify each selected take against takeSource.

The Alder & Branch variation uses a fictional clothing brand and an unworn Weekender Canvas Jacket that is too small. Delivery was 45 days ago, outside the example 30-day return window; original tags remain attached. Human review approves a store-credit exception. Its simulated website is https://alderandbranch.com/chat/a2a.

That variation inserts an explicit 1.5-second IVR-to-AI handoff pause after the final IVR transfer prompt. The prompt's trailing gap is zero, so nextAgentLine.start - lastIvrLine.end is exactly 1.5 seconds with prepared takes. No hold music plays. Verify this interval after generating replacement audio, alongside session-code matching and the source Scene remaining unchanged.

The variation also has an audience-only Narrator persona: an opening overview starts after 1.5 seconds of silence, and a policy explanation follows the human's review message. Narration reuses normal immutable voice takes and stays out of call transcripts and Agent Assist input. The policy explanation anchors a spotlight to assist.resolutions, with a gentle zoom and reduced background dimming. Two phone callouts explain owner verification and the owner's choice.

Visual cues accept easeDuration for symmetric easing, dimOpacity for spotlight strength, and optional focusScale to zoom a spotlight. All motion uses Scene time, without wall-clock CSS transitions. Start/end anchors can reference another moment in the same chapter, including browser, authentication and assistant-message events; word anchors still require speech. Validation rejects cycles, and compilation resolves reference dependencies regardless of script order.

Audio preparation¶

Voices accept the existing ElevenLabs configuration or provider: "polly" with an active tenant Amazon connection, region, language, voice ID and engine: "standard"|"neural". Standard is the starter IVR choice. Polly preparation checks the tenant's integration and supported engine before using the existing internal Polly service. It does not silently substitute a generative engine. The optional soundIntegrationId preserves the ElevenLabs connection for generated effects when speakers use Polly; existing projects still inherit their configured ElevenLabs voice connection.

DTMF uses paired frequencies from the shared phone package's server-safe dtmf-frequencies.json export. The PCM generator and pressed-key renderer use the same press/gap timing. The hold-music-v1 preset generates a short original instrumental phrase; neither preset needs a voice-provider connection. The return starter does not play hold music during the transfer to the Service AI.

A sound moment with audioSource: "upload" accepts a recording of up to 8 MB and 120 seconds. The revision-checked audio endpoint normalizes it with FFmpeg to mono 24 kHz PCM and stores an immutable take. Preview and export mix the same take and gain. Editing an uploaded recording's caption does not invalidate its audio; changing a generated tone's cadence or a generated preset's duration does. Uploads can supply recorded IVR prompts or hold music.

Shared phone extension¶

HomeScreenProps.extraApps accepts host-owned app IDs, labels, optional icon URLs, selection callbacks and optional launchProgress for the selected icon. PhoneKeypad.activeDigit gives authored playback the same visible pressed state as direct input, with aria-pressed on the key. Live phone callers that omit these optional props retain their normal behavior. Keep the personal app's CSS inside PhoneFrame's Shadow DOM.

After changing the shared package, rebuild and pack it, then install the tarball in ai-manager. The package's release script performs those steps.

Portrait provenance¶

The original Ari portrait was generated on 2026-10-14 using gpt-image-2 after explicit approval for the image-generation API. It is an original ceramic robot, not the mascot in the reference screenshots. The 512-pixel WebP is served locally and can be replaced through the assistant portrait field. Uploaded replacements remain project-owned and are copied when duplicating a Scene.

Generation prompt:

An original friendly personal AI assistant portrait for a phone app: a small matte ivory ceramic robot with a rounded square head, two warm black dot eyes, a tiny subtle smile, sage green collar and shoulders. Premium photographic 3D product render, tactile unglazed ceramic with subtle texture, soft daylight, centered bust portrait against a pale warm gray studio background. The head occupies most of the frame and reads clearly at 80 pixels. Calm and approachable. No fur, no ears, no hood, no resemblance to existing mascots, no text, no letters, no logo, no watermark.

Verification¶

The reducer tests cover backward seek, authentication, incorrect codes, session-setting changes, consent-to-text delay, visible Omni offer/acceptance and ring alignment, typed message delivery, backward seek, app launch, order lookup, queue acceptance, case closure, DTMF spectra/gaps and asset ownership. The browser tests capture each stage, record a phone interaction and exercise a compressed MP4 export with prepared DTMF, a preset ring and uploaded audio. For a Studio-only review, run only the replay test; it does not start a video export.

cd apps/ai-manager
npm run typecheck
node --import tsx --test lib/director/*.test.ts
bash ../../.devcontainer/scripts/with-env.sh node_modules/.bin/playwright test e2e/director-assistant.spec.ts --project=chromium -g 'personal assistant journey replays'

During local development, finish source edits before a full export. A Next.js development reload can unmount the render page and remove its seek callback. The script and takes remain saved; retry after the reload completes.