Skip to content

Preview Mock Caller

Status: Current | Last reviewed: 2026-10-07

Voice Preview's Mock Caller supports a configurable AI caller with ElevenLabs v4 speech, Nudge caller and Wrap up controls. TesterSidebarPanel owns useMockPreviewCall above the phone's ordinary and recording layouts so switching layout does not reset the caller selection. Settings live in versioned local storage scoped by tenant, user and agent. They are not written to agent configuration or the collaboration document.

An explicitly opened agent URL with mockCallerPacing=patient applies a 3-second wait and appends the bounded guidance from preview-mock/pacing.ts to an existing caller prompt. It preserves the scenario, voice, integration, time limit and trim options. Hydration waits for the tenant/user and matching agent storage key, saves locally, then consumes the URL parameter. It never starts a call or enables Mock Caller. The append is idempotent; subsequent manual edits are not overridden on reload. An empty scenario stays empty.

Header, settings and live timing

The Mock Caller toggle and settings icon appear only in the maximized Preview header. Recording uses that same header. Non-maximized Preview, including the embedded inspector header, shows Run tests and the maximize control, with no header toggles. PhoneCallTest gates its controls on fullscreen so a missing header portal cannot render a fallback caller toolbar in the sidebar. Minimizing stops the mock session and clears its enabled flag; saved caller settings remain. The page also clears Thinking, Recording, Telephony and SMS, ending active telephony tests. PhoneCallTest stops recording when the mode or full-screen state turns off. It portals Mock Caller controls into one persistent header slot, preserving recorder ownership and disabling settings during capture. All five switches use PreviewToggle for matching typography, label color, spacing and disabled treatment, ordered Show Thinking, Recording, Mock Caller, Telephony, SMS. Enabling Mock Caller opens the modal; the settings icon reopens it. DirectorCallerSettings uses the normal Combobox, useElevenLabsVoices and useTtsPreview for search and authenticated ElevenLabs samples. The player unmounts on modal close and resets on integration change, so audition audio cannot continue into a recording.

PhoneCallTest owns the caller settings dialog's open state. MockCallerControls renders the toggle, an inset pencil button and the shared dialog; the same settings button in the widget opens that dialog. Both buttons disable while a call or recording is active. PreviewToggle accepts an action slot so the pencil remains inside its border without nesting a button in the switch. The canvas widget title uses Lucide's decorative Drama icon; the header toggle keeps its text-only label. MockCallerDirections renders status, a persistent direction field, Nudge caller and Wrap up inside the canvas. PreviewControlWidget gives Mock Caller, Telephony and Recording the same compact shell and rounded-xl corners as Thinking. At a 900 px canvas breakpoint, the layout uses equal flexible side columns around the phone's scaled width. Controls fill the left column; Thinking fills the right and uses the phone's scaled height. The phone stays centered between them. Narrow layouts stack controls above the phone and Thinking below it; phone fitting reserves the measured controls height. The column can hold additional widgets without moving call controls into the handset. The shared canvas owns both ordinary Preview and Recording. Its recording composition spans the phone and Thinking columns using CSS subgrid. PreviewRecordingControls renders recording status, Test & Record, Record, Stop and all download actions in the left column. There is no separate recording header or layout branch, so switching Recording keeps the header and recorder mounted. Recording allocates the panel outside recordingCompositionRef, keeping its private text and buttons out of the MP4, including with Thinking enabled.

Mock Caller and Telephony enable PreviewBrainPanel's compact layout: a single timeline sorted by transcript speech time and action start time. Action rows show status without expanding payloads; opening one reveals inputs, results, errors and nested workflow steps. Stream updates retain action identity, so completion updates its existing row. Standard Preview retains its separate transcript and action sections. Both layouts use the same passive stream and lifecycle data.

The shared modal takes a custom caller prompt and has no client-specific example button. Existing saved prompts are retained.

Timing fields live in Advanced, collapsed each time the modal opens. Their explanations are keyboard-focusable help-icon tooltips beside each label. Recording Options stays visible with one switch, Shorten long silences in downloads, whose help icon explains that it also trims silence at the start and end. The modal omits the demo-defaults paragraph and provider/model footer; timing and export defaults are unchanged.

Telephony is disabled whenever Mock Caller is enabled, including while the caller is idle and in Recording. Its accessible help explains that IVR simulation is not supported. Turning Mock Caller off restores Telephony when no call is active. The existing Telephony layout does not expose Mock Caller.

maxSeconds is a 30–600 second elapsed deadline, including connection setup; the caller is asked to wrap up about 25 seconds before it or after eleven turns. pauseMs is the 800–5000 ms quiet guard before caller planning, shown in seconds in the modal (default 2). It is additional to model and TTS latency. These settings control the live interaction; recording trimming does not.

Audio and conversation flow

POST /api/preview/mock-call opens the existing internal VoiceAgent gRPC transport. The server resolves the requested agent version within the active tenant; the browser supplies a pinned :draft or published version reference. ANI and Preview variables travel in the normal session metadata. ANI comes from Preview variables, not the caller prompt. An empty value is an unknown caller; the Consumer Cellular mock account requires +12025550147.

The caller uses the tenant's default OpenAI integration, Responses streaming and structured output. PREVIEW_CALLER_MODEL defaults to gpt-6-luna. The caller's selected ElevenLabs integration is resolved server-side within tenant/platform visibility. No provider credentials reach the browser. One generated utterance preserves its vocal tags in a v4 HTTP streaming request; PCM consumption begins before the utterance finishes generating.

A 400 ms startup buffer feeds AudioClock's continuous 20 ms microphone frames, including silence between turns. ElevenLabs is requested at 24 kHz; monitoring and recording keep those original samples. Only the agent input copy is resampled when its configured input negotiates another rate. The model gets the publisher's canonical conversation transcript, avoiding duplicate generation/playback captions on the raw gRPC text channel. Its history query orders by audible start (falling back to event time), rather than the time a final transcript was published. The publisher applies reserved user speech-start time/sequence in its shared publication path and retains them across raw captions, aggregator updates and late finals. Both Preview transcript views sort these events by speech time and merge revisions by message identity. Audio energy and caption activity determine when the agent has paused. An agent continuation during caller planning discards that stale caller response before synthesis. Agent backchannels do not cut off caller speech already playing.

An NDJSON response carries connected, audio, transcript, status, ended and error events. Audio records have a role, PCM16 base64, negotiated sample rate, and elapsed arrival time. Both streams are consumed continuously. MockCallPlayback maintains a separate contiguous playback queue per role, monitors them locally and exposes original audio tracks to the phone recorder. No microphone, SIP, WebRTC codec, browser echo cancellation or Krisp is involved in the mock transport. Browser Web Audio performs playback/sample-rate conversion. The phone omits microphone mute, camera and keypad callbacks in mock mode; the shared phone treats an omitted mute callback as a disabled control.

usePhonePreviewRecorder(cleanCallerAudio=false) selects the direct recording graph. The default remains true for human callers. Recording includes the phone and optional Thinking panel, uses direct caller/agent tracks, drains queued playback before the call becomes idle, and follows the existing MP4 conversion and download flow.

Optional silence edits

Caller settings can carry recordingTrim: {enabled, thresholdSeconds, keepSeconds, trimEdges}. It is optional for existing saved setups; trimming defaults off, with a 1.8-second threshold, 0.9-second retained pause and edge trimming on. useMockPreviewCall normalizes trimEdges to true when loading or updating caller settings, so older saved setups also use the combined switch. The converter retains the field for compatibility with existing recording snapshots. The recorder snapshots settings at start alongside the original blob. Downloads offer separate trimmed and original MP4 actions; a later settings change does not alter the captured recording's edit configuration. downloadVariant tracks which download was requested. That button shows a spinner and aria-busy until preparation finishes; both download actions are disabled in the meantime, including original MP4 conversion.

POST /api/preview/recordings/mp4 is excluded from the Next Proxy matcher. Its own withAuth('AIManager:RunAgents'), rate limit, and streamed 100 MB cap remain authoritative. Proxy otherwise clones and truncates bodies at 10 MB: a valid WebM header can still convert successfully while losing the farewell. Returning early from Proxy does not avoid that clone. The browser E2E uploads a real WebM above 10 MB, verifies its final frames/audio, and checks anonymous requests still return 401.

POST /api/preview/recordings/mp4 accepts these validated settings as JSON in X-Recording-Trim. Native MP4 also goes through the endpoint when editing is enabled. A bounded FFmpeg scan uses silencedetect at -50 dB across all recorded channels. Intervals below the threshold are untouched; longer quiet intervals retain the configured pause, split 4/9 after the previous speech and 5/9 before the next. Optional edge trimming retains 350 ms before and 550 ms after speech. An entirely silent recording is retained.

The same keep intervals feed video trim and audio atrim, then synchronized H.264/AAC concatenation. Edits round outward onto the 30 fps frame grid, retaining slightly more quiet rather than accumulating audio/video offsets. Speech speed and order are unchanged; visible source timers can jump. Scan limits are 15 minutes/100 edits, plus the endpoint's existing 100 MB upload, concurrency, cancellation and temporary-file cleanup. Failures retain the original blob. X-Recording-Removed-Seconds reports the quiet time removed.

The prior Consumer Cellular fourth export's edit manifest records 182.606 seconds before editing and 124.23 after: eight long gaps plus lead-in/tail were shortened using the same 1.8/0.9-second pause defaults. That edit measured combined waiting time, not separate model, TTS or network latency.

Ownership, controls and cleanup

Both endpoints require AIManager:RunAgents and its API-token scope. Starting is rate limited. A Redis lease permits one active call per tenant/user; server-generated conversation IDs are registered in the existing phone owner registry for passive transcript, multimodal and Thinking access.

Mock Caller uses the same SMS routing as the ordinary Preview phone. smsDeliveryMode defaults to demo_notification; the phone's SMS switch selects real_sms. Session metadata includes that delivery mode and the server-created notification session ID. It does not suppress the multimodal handshake: identity and patient-plan links arrive as separate messages after the agent obtains permission. Failed sessions log their error class and code locations without provider response bodies. Numeric published references such as 4701:13 normalize to 4701:v13 before version resolution, ownership registration and transport startup. The caller can remain silent while the agent works or an operator completes a screen action. Empty or vocal-tag-only decisions skip speech synthesis and keep the call open; they do not count as a goodbye.

POST /api/preview/mock-call/control accepts conversationId, action (cue, wrap_up, stop) and optional text. A Lua ownership check writes a short-lived control mailbox for exactly that user's current call. This works across API replicas. The latest pending control replaces the preceding one. Caller directions never enter the agent's input stream.

Caller direction → Nudge caller sends a private cue to guide the customer's next planned turn. Wrap up sends wrap_up for a natural goodbye. Both controls wait for a connected call and disable while submitting. The nudge form supports Enter to send and Shift+Enter for a new line, and stays outside the recorded phone area. The existing mock-call API paths and local-storage keys are retained so saved settings continue to work after the Mock Caller rename.

Calls are bounded to 30–600 seconds. The caller is directed to finish near the limit or after eleven turns. On a goodbye, prefer the agent's own disconnect; a quiet 20-second fallback handles agents without an end-session tool. Browser cancellation, navigation, provider failure, or a deadline cancels the gRPC stream, caller model/TTS requests and PCM clock. Stream buffering and utterance sizes are bounded; the Redis lease is compare-and-deleted on cleanup. The underlying agent retains its ordinary tools, workflows and data; this mode does not sandbox or replace tool results.

Validation

  • lib/preview-mock/session.test.ts covers paced PCM, tenant/version/ANI, stale-answer replacement, private wrap-up, canonical transcripts and cancellation.
  • e2e/preview-mock-caller.spec.ts covers authentication/control ownership, header placement, voice search/sample playback, persisted setup, mode changes, private nudges, wrap-up, trim settings and a real H.264/AAC recording. It checks three-column geometry, top placement and caller-panel width in Preview and Recording, and inline action ordering, result updates and expansion. Frequency probes verify both audio lanes; the test forbids microphone access and checks that no Krisp assets load.
  • lib/preview-recording-conversion.test.ts checks actual silence editing with two speakers, overlap, short pauses and video markers aligned to audio.
  • Live local verification uses the saved Consumer Cellular draft and configured provider integrations, with synthetic ANI +12025550147.

See the user guide and Preview recording contracts.