connect-scv-translator

Status: Current | Last reviewed: 2026-10-14

The Salesforce Service Cloud Voice (SCV) BYOT (Bring Your Own Telephony) bridge, and the host for Live Translation on an SCV call. Lets a Salesforce agent take a call routed through Amazon Connect or Genesys Cloud while TrueServe translates both directions in real time.

Amazon translation modes

Published translation agents can select AWS → Voice Translation with either Polly PCM response streaming or bidirectional streaming. Both modes use the existing orchestrator provider, two directional streams and original-audio recovery. The browser logs [AWSTranslationPerformance] system events separately from transcripts, adding direction and current playback-buffer occupancy. See AWS translation for the measurement scope and repeatable comparisons.

Source

apps/connect-scv-translator

Runtime

Aspect	Value
Language	JavaScript (Node 20 / Express) + browser bundle (webpack)
Wire	SCV partner-telephony APIs, WebRTC, WebSocket → gRPC relay
Ports (Vanguard)	3031 Express, 8101 webpack-dev-server

What it does

1. Registers as an SCV BYOT connector (initializeConnector from @salesforce/scv-connector-base) and implements SCV call control — accept, end, mute, hold, transfer, conference.
2. Drives a concrete telephony provider server-side: Amazon Connect (amazon-connect-streams, embedded CCP), Genesys Cloud (genesys-cloud-webrtc-sdk), or a demo mode. A provisioned contact-center binding selects the backend provider and instance; unbound development deployments retain the TELEPHONY_PROVIDER fallback.
3. Posts call lifecycle and transcripts back to Salesforce over SCRT2 — when translation is on, both the original language and the translation, from the translation pipeline alone (below).
4. Runs Live Translation: taps the WebRTC audio, streams it to ai-orchestrator translation agents, and plays the translated audio back into the right ear.

Partner Telephony administration and inbound calls

The org-local TrueServe Contact Center Settings tab supplies administrator controls for outbound dialing, external/queue transfers, conferencing, translation availability/default languages/default agent/autostart, noise cancellation, accent conversion, and personal overrides. Deploy the org-local extension and grant its TrueServeContactCenterAdmin permission set alongside the existing named-credential grant. It calls the existing managed trueservedev__TrueServeCredential; a managed package replacement is unnecessary. The old custom0/custom1/custom2 XML sections were inert sample fields and are omitted from newly generated XML.

Native Queue Mappings relate Salesforce queues to real provider queue UUIDs. Native Contact Center Channels associate a claimed phone number with an active Omni-Channel routing flow and a mapped fallback queue. These features require the provider discovery APIs and an actual inbound call-flow integration; populating a list alone does not route a call. Discovery paginates Connect APIs, uses the bound instance region, and reports permission failures rather than returning demo data. Agent synchronization uses verified Connect identities; removing a Salesforce membership does not delete a shared Connect account.

Salesforce and Connect user mappings

For Amazon Connect centers, TrueServe Contact Center Settings → User mappings reads established identities from Salesforce Setup automatically. User.CallCenterId identifies membership; CallCenterRoutingMap.ReferenceRecordId and ExternalId pair that Salesforce user with a Connect user. Only user mappings for current members of the selected center are imported; queues, other centers and stale memberships are excluded. These rows are labelled From Salesforce Setup and are managed in Setup. They take precedence over connector-only overrides on both read and save. getSettings performs no writes; a save copies the effective pairs to the connector alongside other settings. Salesforce remains authoritative.

New mappings use a native Salesforce record picker (name/username) and a searchable Connect user selector (username/ID), backed by paginated ListUsers for the bound instance. Searches default to current contact-center members and their Connect identities. Include users not yet added expands Salesforce search to active, unassigned users and Connect search to the whole instance. Users assigned to another Salesforce center remain excluded. Existing saved selections remain visible even when outside the current search filter. Directory failures are visible and can be retried with Refresh Connect users. New selections require real directory IDs.

For a new user, save the mapping before using Add Users on the native contact-center setup page. The connector resolves the current settings on each request. An explicit mapping takes precedence over an exact username match; addUsers checks the user against Connect's inventory and calls DescribeUser in the bound instance. Saving validates ID format, not remote account existence. A stale or wrong-instance UUID fails visibly during Add Users. The editor does not create accounts, assign queue membership/routing profiles, or remove existing Salesforce membership when a mapping row is deleted.

Add Users queues Salesforce's background partner synchronization and reports completion by email. The managed TrueServeIntegrationImpl.addUsersToContactCenter passes the selected Salesforce users to /api/sfdc/users/add; the connector verifies existing Connect accounts and returns the identity pairs. Salesforce then records membership and routing mappings. Saving a pre-onboarding mapping in the custom page does not perform that membership operation, and this connector never creates a Connect account during synchronization. A successful native mapping cannot bootstrap its own first Add Users request; unmatched new users still need an explicit choice.

User sync status polls the service-only /api/sfdc/settings/user-sync endpoint through the permission-checked org-local Apex controller every 15 seconds, without reloading settings or discarding drafts. Attempts become visible when the partner request reaches TrueServe, not while Salesforce is initially queuing it. The connector records processing, verified or failed; Apex independently reads User.CallCenterId and native user mappings before the UI labels someone Added. Provider verification alone is Awaiting Salesforce. After 90 seconds without membership confirmation, pending rows say Status unknown or Needs review, not an indefinite processing spinner. Errors advise reviewing the mapping and retrying Add Users in Setup. This does not retry or write membership automatically.

The latest attempt per user is kept for 30 days, bounded to 5,000 records, in ${SCV_CONTACT_CENTERS_FILE}.user-sync.json beside the mounted configuration. It has the same single-process ownership requirement, private permissions and atomic replacement; keep that directory writable and persistent. Records are scoped by Salesforce org, center and provider instance. Recording failures never change the partner response; read failures surface a status-unavailable message. History starts with this deployment. A process interrupted during verification leaves an unknown outcome rather than fabricating a failure or success. Already saved mappings and confirmed members also appear even without recorded attempts.

The settings save retains the loaded revision. A concurrent update leaves the draft visible and asks you to reload instead of overwriting the newer settings. Deploy both org-local LWCs and the updated Apex controller/test separately from the managed package. The connector must expose the user-directory endpoint. The UI requires TrueServeContactCenterAdmin and the existing TrueServeSCVGrant named-credential access; it does not require Author Apex.

Run the org-local LWC regression suite from sfdc/trueserve-scv with npm run test:unit:org-local -- -- --runInBand. It covers Shifa's reported mapping, preservation of existing settings, edits/removals, validation and revision conflicts.

Binding and durable settings

Set SCV_CONTACT_CENTERS_FILE to a persistent, private JSON file outside the image and repository. One connector process owns the file. Saves validate types, enforce organization ownership and an expected revision, then atomically replace it. A configured missing/corrupt file fails visibly. An unset file retains the legacy unbound development behavior. Multi-replica deployments need a shared transactional store before enabling this administration API; a local file is not a cluster lock.

{
  "version": 1,
  "centers": [{
    "id": "<Salesforce 04v contact-center ID>",
    "orgId": "<Salesforce 00D organization ID>",
    "internalName": "trueServeContactCenter",
    "label": "SCV Translator",
    "provider": "connect",
    "partnerContactCenterId": "<Connect instance UUID>",
    "region": "us-east-1",
    "userMappings": { "<Salesforce 005 user ID>": "<Connect agent UUID>" },
    "policy": { "outboundEnabled": true, "translationAgentId": "3033" },
    "routing": {
      "enabled": true,
      "phoneNumber": "<claimed E.164 number>",
      "additionalPhoneNumbers": [],
      "omniFlowDeveloperName": "TrueServe_DevSDO_Inbound",
      "fallbackQueueId": "<mapped Salesforce 00G queue ID>"
    },
    "revision": 1
  }]
}

Omitted policy keys use the validated defaults in src/common/contact-center-policy.mjs. Use an instance UUID, never Salesforce's 04v ID, as partnerContactCenterId. The browser's existing provider configuration and the server binding must describe the same instance. Provider/instance changes also require corresponding Salesforce and Connect provisioning; saving them is not a migration of AWS resources.

SCV_AWS_PROFILE optionally selects a local shared AWS profile for Connect APIs; production uses its workload IAM role. CONNECT_REGION takes precedence over the host's general AWS region. SCV_ROUTING_SCRT_URL pins the verified Salesforce SCRT origin and the existing SCRT signing key authenticates requests. The routing client does not depend on an agent browser having initialized the server. This deployment uses one Salesforce org/SCRT origin per connector.

Service-bearer-only endpoints:

Endpoint	Purpose
GET /api/sfdc/settings?orgId=…	Read provisioned centers for the Apex caller's org.
PUT /api/sfdc/settings	Validate and save an authorized admin revision.
GET /api/sfdc/settings/discover	Read real provider queues and phone numbers.
GET /api/sfdc/settings/users	Read Connect users for the provisioned center and Apex caller's org.
GET /api/sfdc/settings/user-sync	Read recent connector attempts and saved user IDs for the caller's provisioned center; Apex adds live Salesforce membership.
POST /api/sfdc/routing/inbound	Prepare, route, or clear a bound Connect contact.

The org-local Apex controller derives the org ID and checks a custom admin permission. Browser sessions may read /api/contact-center-policy but cannot use these administrative/service endpoints. Agent actions refresh policy with a bounded wait and retain the last policy during a transient outage. Translation and accent start requests also enforce policy on the server. Personal defaults remain separate; restrictions take precedence, and allowed personal overrides are retained.

Inbound routing lifecycle

apps/scv-inbound-routing-lambda/handler.mjs authenticates to the connector with a service token read from Secrets Manager. The Lambda validates its instance and contact-center binding. The server obtains caller/number/contact data directly from Connect rather than trusting supplied destinations or caller identity.

1. The dedicated Connect entry flow sets TrueServeContactCenterId.
2. The server creates the inbound VoiceCall using the real Connect contact ID as vendorCallKey, then persists TrueServeVoiceCallId on that contact. Validated SalesforceFields are preserved in callAttributes; a valid Case case_id sets both RelatedRecordId and DCC_CaseId__c before Omni runs. Preparing the VoiceCall must not discard the AI's handoff fields.
3. It invokes SCRT PATCH /voiceCalls/{id}/omniFlow. The phone channel determines the routing flow; a configured Flow API name supports the no-channel case.
4. Salesforce resolves its queue/rep through CallCenterRoutingMap. The server validates that the resulting provider UUID/ARN belongs to the same instance.
5. Connect sets the working queue/agent and transfers the caller. The browser passes the pre-created VoiceCall ID to SCV when the agent receives the call.
6. Failure branches and an EventBridge DISCONNECTED rule clear pending Salesforce routing. A disconnected call that never reached an agent is also closed with its actual start/end times; a browser cannot send its call-ended event. Connected agent calls retain their browser-managed hold metrics. Unrelated contacts and inherited attributes on other transfer legs are ignored.

routing.phoneNumber and the optional routing.additionalPhoneNumbers form the explicit inbound number allowlist. Additional numbers must be E.164, with at most 20 entries. A producer transfer can enter this same route after its voice agent finishes; transferring directly to a Connect queue does not invoke Omni. For the Allianz configuration and ownership of Case/workspace writes, see producer handoff.

SCRT creation can exceed Connect's eight-second synchronous Lambda limit. Each prepare/route request returns pending after one second while its server job continues. The bounded entry-flow polling blocks reuse the in-flight job and its persisted result. Do not shorten the SCRT request timeout to the Lambda timeout: Salesforce can commit the VoiceCall after the caller has timed out. An ambiguous create after a process crash is marked for reconciliation instead of creating a second record. Find the existing VoiceCall by VendorCallKey before restoring its ID on the Connect contact. Changes/restarts during an in-flight create can require that reconciliation.

This is external routing through an Omni-Channel Flow, not Salesforce unified routing. Do not enable the unified-routing capability as part of this setup. Connect queue/agent transfers use actual Streams Quick Connect endpoint objects; queue mappings do not create Quick Connects. Associate a queue Quick Connect with a queue in the agent's routing profile to expose it in the phone's transfer list. Transfer-to-Omni-Flow is not advertised by the connector.

DevSDO provisioning

The October 2026 deployment binds contact center 04vDW0000000AkAYAU to dcc-dev-main-connect (9156b961-eb34-4d3a-92ad-f770d8f8b282, us-east-1). The dedicated number is +1 901 297 4032, the Salesforce and Connect queues are TrueServe DevSDO Inbound, and the active routing Flow is TrueServe_DevSDO_Inbound. The Connect entry flow is TrueServe DevSDO Salesforce Inbound and the Lambda is trueserve-devsdo-scv-inbound.

Erik Golden is the contact center's assigned Salesforce agent. His dedicated Connect routing profile copies all 21 prior queue/channel entries and retains BasicQueue as the outbound default, adding only the new inbound queue. Other agents' shared routing profiles are unchanged. To add an agent, save the provider user mapping in TrueServe Contact Center Settings → User mappings first, then assign Salesforce contact-center and queue membership and add the Connect queue to that agent's routing profile. Refresh the Salesforce console after changing the profile.

The Lambda role may read only its service secret and write its own logs. Its resource policy permits the configured Connect instance and disconnect rule. Associate the Lambda with the instance, attach the DID to the dedicated flow, and retain EventBridge retries for cleanup. Keep the service token out of flow attributes, Lambda environment values, source control and logs. Rotate the Secrets Manager value when the connector service token rotates.

Outbound queue and caller identity

The packaged agentConfig LWC replaces the former free-form Caller ID and Queue ID inputs with one provider-backed Outbound queue / caller identity control:

• ConnectTelephonyProvider.getOutboundQueueOptions() reads agent.getRoutingProfile().queues and defaultOutboundQueue from Streams. It requires no AWS SDK credentials or server integration.
• One assigned/default queue renders read-only. Multiple assigned queues render as a dropdown, using names as labels and queue ARNs as values.
• The selected ARN is stored by connector origin, Connect instance, and agent. A saved ARN is reused only while it remains assigned to that routing profile; otherwise selection falls back to the current default.
• dial() supplies the selected queueARN to agent.connect(). Amazon Connect then applies that queue's outbound caller-ID configuration.

The settings LWC cannot call Streams directly because it lives in Salesforce while Streams lives in the connector iframe. It exchanges OUTBOUND_QUEUE_OPTIONS_REQUEST, OUTBOUND_QUEUE_OPTIONS, and OUTBOUND_QUEUE_SELECT over ServiceCloudVoiceMessageChannel__c, with the same APPLICATION_SCOPE and source='CONNECTOR' return convention used by Call Settings. Providers that do not implement client-side queue discovery return supported:false, and the LWC explains that their configured default will be used.

The companion Agent Studio for Salesforce managed 2GP package is at sfdc/trueserve-scv/. It contains the partner connector plus the Copilot, Agent Assist, Conversation, Wrap-Up, Surface, and Call Settings LWCs. Apex types service_cloud_voice.* only exist in managed packages, so local development needs the package installed.

Native incoming-call preview

ScvBridge maps CustomerName, CustomerRecordId, CustomerOrganization and InteractionSummary from provider contact attributes onto the native SCV Contact. The phone number remains the dialable number. CustomerRecordId must identify a Salesforce Contact or Account; it also sets renderContactId. Connect Streams returns an AttributeDictionary of {name, value} objects; the preview unwraps those values. Genesys supplies plain strings. Do not test only flat attributes: that bypasses the actual Connect browser contract.

SalesforceFields is an optional JSON string containing actual VoiceCall field API names and primitive values. The connector puts the validated fields on PhoneCall.callAttributes with the sfdc- prefix, as well as in CallInfo.additionalFields. DevSDO's native incoming-call formatter strips the prefix and passes callAttributes to handleCallEvent when creating the call. It does not forward additionalFields on that path. Sending only that property leaves the incoming card empty even if an end workflow fills the record later. The explicit envelope accepts custom fields and a valid RelatedRecordId, with bounded counts and values. It excludes standard ownership/routing fields. Bad optional JSON produces empty additional fields rather than blocking the call. Calls without this envelope preserve the legacy attribute behavior. Authors must supply writable fields available to the Salesforce user, never formula fields. This mapping adds no model or network request while ringing.

The org's assigned VoiceCall compact layout controls the native call card. DevSDO's DCC_Connect_Handoff layout contains caller name/organization and the short summary, without the phone-number row or duplicate reason. A native call-creation check confirmed that the fields are stored with disposition new, before any answer or lifecycle patch; it did not exercise PSTN delivery. The Takeda demo supplies their writable inputs before routing to the human. No replacement Omni widget or custom styling is needed.

ElevenLabs accent normalization

Phone Telephony Settings offers independent Krisp and ElevenLabs (experimental) accent choices for the microphone and customer playback. The page's Call controls shows each selected profile with a live On/Off switch. Krisp remains the default. ElevenLabs is English speech recognition plus synthesis, so it changes the voice and may preserve STT mistakes; it is not an acoustic voice-preserving filter. Its optional grammar toggle is off by default.

The control plane is Settings → Tenant Settings → Accent normalization in Studio, stored as tenant_settings.accent_normalization. It is a tenant profile, not a new conversational agent subtype. The data plane is a dedicated Node streaming runtime in this service:

original microphone/customer track
  → mute/hold capture gate → AudioWorklet (16 kHz PCM, 20 ms)
  → MessageChannel → Web Worker → same-origin WebSocket
  → Scribe v2 Realtime → stable phrase release
  → optional, deadline-bound Gemini grammar → Flash v2.5 / v2
  → binary 24 kHz PCM → Worker → AudioWorklet bounded playback ring
  → microphone sender OR agent playback, never the opposite direction

Each direction owns separate provider sockets. They warm concurrently; each Scribe stream receives two seconds of paced silence before capture attaches. Playback uses a 600 ms initial reserve by default. The first stable two-word phrase bypasses grammar; later repair fits within available queued playback minus 200 ms. Late or unsafe edits retain the original text. The guard preserves numbers, negation, pronoun families and content-word order; it does not prove semantic equivalence. No LLM tools or conversational turn detector participate.

Adaptive playout. The AudioWorklet retains the authored startup hold and checks queued audio before releasing playback. After starvation it gathers a useful refill instead of immediately playing every small packet. The target grows with observed shortages, up to 1.2 seconds or half the maximum queue, whichever is smaller, with the authored startup reserve as its floor. Large refills pay only the newly learned margin, at most 120 ms. The target relaxes after ten seconds of uninterrupted playback; this never skips or accelerates speech. An empty interval of at least 2.5 seconds resets the target for a new phrase. These are arrival-time heuristics, not semantic utterance boundaries.

A final fragment below the target drains after 250 ms without another packet, subject to the startup hold. A trickle cannot defer release beyond the maximum target measured from the first buffered packet. Pause, hold and reconnect clear the queue and learned state; late packets during warmup are ignored. The existing maximum audio backlog still fails the accent branch without dropping arbitrary spoken samples, preserving working noise cancellation. Buffering cannot remove pauses already synthesized into PCM or make sustained slow generation keep up with speech.

ElevenLabsAccentFilter.getPlayoutStats() exposes once-per-second metadata: state, queued duration, target, refill count and time spent holding buffered audio. PCM still travels directly between Worker and AudioWorklet. The refill count includes short inter-phrase gaps and is not a measure of network loss. The buffer target is distinct from total end-to-end speech delay.

The 2026-10-02 deterministic replay used the identical PCM and arrivals for both implementations. One captured stream retained 2.035 s to first PCM playout and one interruption, with recovery growing from 72 to 152 ms. An earlier captured stream changed from two interruptions to one. A synthetic small-packet jitter stress test reduced 25 interruptions to two (about 859 and 81 ms), with 101 ms extra startup and 80 ms later completion. All samples drained in order. A separate overload scenario still triggered the queue limit in both versions. These are buffer comparisons, not live Connect latency guarantees.

The deployed worklet was then checked through the real managed Salesforce authorization bridge with controlled bidirectional WebRTC audio. It reported adaptive mode, a 600 ms target and zero refills during the short run. Received converted audio began 2.14 seconds after source speech; initial provider setup took another 3.14 seconds beforehand. Incoming customer audio remained audible with outbound accent conversion On and after Off. No errors were reported. The longer baseline capture used for arrival replay ended in a provider failure before all generated audio arrived; it is not evidence of sustained call reliability. Queue bounds remain enforced rather than permitting growing delay.

AudioWorklet owns capture and playback, and a transferable MessageChannel moves PCM directly to the Worker. UI scheduling is outside that PCM path. A Worker alone cannot protect provider credentials or remove STT/TTS latency. Pipecat could host a custom frame pipeline, but its conversational turn machinery is unnecessary here; no Pipecat-versus-Node overhead benchmark was performed. Keep this runtime colocated with the connector initially. A separate regional service is an operational scaling option, with shared admission state and session routing required before adding replicas.

Translation uses separate original/NC destinations in both directions and never transcribes ElevenLabs synthesis. ElevenLabs itself currently receives original audio rather than a serial Krisp NC→ElevenLabs cascade. Stable call destinations remain installed during settings changes. Mute gates outgoing capture and output; hold gates both directions. Both clear queued speech and terminate upstream sockets. Resume passes original audio while new streams warm, preserving words spoken during reconnect without replaying old synthesis. Provider failure, overload or unsupported browser features produces a visible fallback. An accent failure preserves working Krisp noise cancellation and the other audio direction. Raw audio is used only where no working filter is available.

Authorization and deployment

Deploy the managed package including TrueServeAccentAuthorization, TrueServeCallPreferences, TrueServeCallPreference__c, trueServeBridge, translationSettings, agentConfig, callDefaults, elevenVoicePicker and the updated TrueServeSCVService permission set. The existing VoiceCall lookup and ServiceCloudVoiceMessageChannel are dependencies. Use the configured SCV Named Credential (TrueServeCredential, or its namespace prefix through the LWC's Accent Named Credential property).

For an installed managed package, deploy a managed version. Its private message channel and bridge cannot be imported or composed by an unmanaged pilot. devsdo was migrated to beta 0.18.0.1 on 2026-10-02. Beta replacement requires uninstall, restoring page placements and re-provisioning credentials/permission grants. Treat future replacements as approved maintenance actions. Metadata validation of an unmanaged LWC does not demonstrate that it can reach the managed connector.

The always-mounted trueServeBridge answers connector grant requests through Apex, even when neither settings editor is open. Apex queries an active VoiceCall owned by the signed-in user with WITH USER_MODE, then calls the SCV backend using the Named Credential's service bearer. The backend also checks the configured Salesforce org. A browser-minted SCV session is insufficient to mint that grant.

The active-call predicate is CallDisposition = 'in-progress', alongside the vendor call key and current user's ownership. CallEndDateTime is required: SCV sets it to the start time while the call is live. Testing for a null end time rejected the live microphone toggle on 2026-10-02. Apex regression tests create that real VoiceCall shape and verify that completed calls and another agent's calls cannot obtain a grant.

Contract	Authorization / lifetime
Studio GET/PUT /api/settings/accent-normalization	Tenant ManageConfiguration; non-secret profile
Studio GET /api/settings/accent-normalization/runtime	Explicit accent:runtime tenant token only; token tenant is authoritative; credentials returned with private, no-store
SCV POST /api/accent/voices/search	Salesforce service bearer + configured org + current Salesforce user; {search,nextPageToken}; account voices only
SCV POST /api/accent/voices/preview	Same authorization; {voiceId}; bounded audio as {audioBase64,mimeType}
SCV POST /api/accent/voices/validate	Same authorization; up to two {voiceIds}; canonical names and IDs
SCV POST /api/accent/grant	Salesforce service bearer; body {orgId,userId,callId}; random grant expires in 60 seconds
SCV POST /api/accent/ticket	Browser session + CSRF header; body {grant,direction}; one ticket per direction, 15 seconds
SCV /ws/accent	Allowed Origin + cookie-bound, single-use ticket in first {type:"start",ticket,grammar,voiceId} message, within five seconds

PCM is binary signed 16-bit mono at 16 kHz upstream and 24 kHz downstream. Controls are {type:"pause"} and {type:"resume"}. Status messages include ready, paused, error and content-free timing; timings start at first captured PCM and are not audible-startup measurements. The server selects all provider URLs, models and integration keys. Browser-supplied models are ignored. There are no credentials or admission tickets in URLs.

Set these backend variables; both the tenant profile and connector gate must be enabled:

Variable	Value
SCV_ELEVENLABS_ACCENT_ENABLED	true to advertise the alternative; shipped default off
SCV_ACCENT_STUDIO_URL	Studio HTTPS base URL; localhost HTTP allowed for colocated development
SCV_ACCENT_STUDIO_TOKEN	Dedicated tenant API token with only accent:runtime; keep in secret storage
SCV_ACCENT_TENANT_ID	Expected tenant UUID; must match runtime response
SCV_ACCENT_SALESFORCE_ORG_ID	Exact 18-character org ID emitted by Apex

Studio selects existing encrypted ElevenLabs and optional Google integration secrets. The bounded profile allows Flash v2.5/v2, English, 0.8–1.2 speech speed, 150–500 ms stability, 300–1500 ms phrase wait, 200–1000 ms startup buffer, 2–8 seconds maximum queued audio, and 100–1200 ms maximum grammar deadline. The current grammar default is Gemini 3.5 Flash-Lite; 3.1 and 2.5 are selectable but availability must be verified against the integration. The tested account returned 404 for 2.5, so the captures use 3.5.

Profile reads coalesce and cache for 15 seconds; existing streams retain their session snapshot. Admission maps are process-local and bounded: 200 active streams, one active plus one warming replacement per user/browser/call/direction, and 60-minute sessions. Use one instance or sticky routing. Upload limits allow a 64 KB burst at the actual 32 KB/s PCM rate; download and text queues are bounded. Capture/transcript content is transient runtime memory, not application logs or recordings. Provider retention is governed separately by the selected integrations.

Personal voices and saved defaults

callDefaults is embedded only in the phone's Telephony Settings. The page/utility component shows live On/Off controls, call languages and read-only accent profiles. elevenVoicePicker provides debounced account search, pagination, local sample playback and explicit selection per direction. The editor never writes current-call controls. Its Save calls TrueServeCallPreferences.saveDefaults(settingsJson, expectedRevision, namedCredential); getDefaults() returns {revision,settings}. No method accepts a user ID.

Salesforce stores one private TrueServeCallPreference__c record per agent: AgentId__c (unique external ID), Settings__c (validated JSON), and Revision__c. Apex always filters by current user AND owner, uses user-mode reads/DML, locks existing rows, and rejects stale revisions. The packaged permission set grants only create/read/edit, with no view-all/modify-all/delete. Voice IDs are validated against the tenant's integration before saving; names come from that catalog. Back up these records as data before any future beta uninstall.

CALL_DEFAULTS_CHANGED contains no preference payload: the bridge re-reads Apex and sends CALL_DEFAULTS {revision,settings}. It also loads at mount and answers CALL_DEFAULTS_REQUEST (connector startup, call end, and every 60 seconds). The managed bridge serializes every outbound message with JSON.stringify before calling Salesforce's injected handleMessage. Lightning Web Security can proxy a parsed Apex object when it crosses namespaces; the native MessagePort.postMessage then throws DataCloneError. A successful Apex read alone therefore does not prove delivery. Salesforce's injected callback also calls Object.assign({}, message): JSON text becomes an object with one UTF-16 character per numeric key. decodeBridgeMessage validates contiguous keys and single-character values, reconstructs the text and parses it before ScvBridge.handleMessage dispatches. Direct JSON strings and older object messages remain supported. The same transport carries accent grants and live toggle patches without logging their payloads.

The connector keeps this baseline separate from live controls. CallAudioSettings.begin() captures the latest audio baseline for each new call; repeated media events preserve the same snapshot and manual switches. Connect can retain its contact ID during wrap-up, so getActiveCallId() alone must not decide whether a call is still live. Later saves apply to the next call, older revisions are ignored, and App Builder seed messages cannot overwrite personal choices. Call end restores translation and audio defaults even with the utility closed. A failed read preserves the last loaded baseline.

If the first Phone read is still pending when the call connects, accent processing waits rather than guessing a provider. Its response fills only the missing provider/voice/grammar profile and retains any manual On/Off request; it does not start late automatic defaults. Updates wait for audio-manager attachment to finish before changing the graph, so a defaults response cannot supersede an unfinished sender replacement. Status presents the agent's requested switches separately from the actual audio mode: a pending On remains On while the effective mode stays noise cancellation and the profile is loading. Off cancels that queued start.

The settings JSON contains translation (autoStart: auto|on|off, optional language/agent IDs) and audio.outbound/inbound (NC/accent auto|on|off, accentProvider, grammarRepair, voiceId, voiceName; outbound BVC auto|off). Empty voice/language IDs inherit defaults. Organization override gates still win. The bridge owns call authorization so an unopened utility cannot prevent startup.

The server searches ElevenLabs GET /v2/voices and verifies uncached IDs with GET /v1/voices/{id}. It never adds/clones voices. A bounded five-minute voice cache is keyed by tenant and credential digest; account search/save warms it before the call. Uncached selection requires one lookup before stream startup; there is no catalog lookup per audio frame. Filter reuse identity includes both voice and grammar preference; Phone profile changes take effect next call.

Samples come only from provider-returned HTTPS URLs under the exact storage.googleapis.com/eleven-public-prod/ bucket. Redirects are refused, provider credentials are omitted, and samples are capped at 1 MiB. Real account files include MP3 and WAV mislabeled text/plain; the server checks audio signatures and returns the correct MIME. LWC uses a revocable local Blob URL, so no extra remote media CSP entry is needed. Stale search/sample responses are ignored and only one picker sample plays at a time. Catalog errors never start paid synthesis or open a call stream.

The validated Salesforce beta is 0.19.0.1 (04tKf000000HKfeIAG), with 75% Apex coverage and validation skipping disabled. The isolated package's 60 UI tests and an expanded run of 198 connector tests pass. The connector build and targeted bridge lint pass. Full package lint retains existing errors in the unrelated Agent Assist test and legacy MFE bundle.

The approved devsdo migration from 0.18.0.1 to 0.19.0.1 is complete. All 110 restored metadata files match the fresh backup, 23 backed-up permission assignments are preserved, and the Call Center settings are unchanged. The saved private phone preference was restored with the same owner, revision and settings. A fresh browser check confirmed six page switches, no duplicate defaults editor and successful Phone defaults loading without saving. The installed managed bridge delivered both defaults and accent grants through Salesforce's unmodified native callback, its shallow copy, the browser MessagePort and the deployed connector decoder. The isolated browser announced connector readiness while blocking Amazon Connect, without opening a CCP session. A direct string-to-MessagePort test had missed the shallow copy; the regression test now includes that step. Authorization granted an owned in-progress probe call with the real start/end placeholder shape and rejected a completed call. The temporary probe record was deleted afterward. A controlled WebRTC run used those delivered defaults and grant, captured a new call's profile and sent a manual On patch containing a stale Krisp provider. The manager retained ElevenLabs/Sarah, replaced the WebRTC sender and delivered converted audio to the receiver. Simultaneous customer speech arrived through a real WebRTC receiver and continued through the agent's NC playback while microphone conversion was On and after Off. Off returned the microphone to Krisp noise cancellation, with no reported errors. Initial setup took 3.02 seconds; received audio began 2.19 seconds after input speech onset once setup completed. The receiver used a playing audio element as well as a Web Audio recording tap. Setup and speech latency are separate measurements; this does not establish live Salesforce/Amazon Connect call timing.

The existing unprefixed TrueServeCredential retains its service token, principal grant and trueservedev allowed-namespace entry. Its endpoint remains unchanged. The namespaced and unprefixed SCV credentials must be rotated together while both are retained; see the package migration runbook. Authenticated backend checks and voice catalog/sample calls from Salesforce passed after restoring the three managed credentials.

Measured implementation timing

On October 1, four real Chromium runs of the implemented Worker/AudioWorklet path against live providers used the office fixture, with grammar alternating off/on. Startup was 2.07, 2.08, 1.77 and 1.91 seconds. Provider initialization took 2.07–2.14 seconds before source playback. The first run also stalled the UI thread for one second. No capture errors occurred in these four retests. Audible onset was three consecutive 10 ms RMS frames above −50 dBFS, recorded on the same AudioContext timeline for original and output.

An earlier grammar run hit the fixed-window capture limiter and took 2.38 seconds; the bounded token-bucket limiter above fixed that admission error. Earlier no-grammar clean/office runs were 1.84/1.75 seconds. The sample is small, does not establish a p95, and does not measure Salesforce, Connect or headset latency. Warm startup is near two seconds; cold setup plus startup is roughly four seconds. Grammar did not establish an improvement and bypasses the first phrase by design. Lowering the playback buffer may reduce startup at the cost of more gaps; the default preserves the listening-test setting.

Bidirectional Krisp RTC enhancement

Amazon Connect SCV calls use a provider-neutral browser audio manager before Live Translation. It connects bypass audio immediately, then hot-swaps the licensed Krisp filter only after its model is ready. Unsupported browsers, model/download errors, sampling-rate errors, repeated buffer overflows, and device recovery failures retain available audio processing, falling back to raw audio where no working filter exists.

agent microphone → outbound NC / BVC / Accent / bypass → mute+hold gain
                → telephony sender + outgoing translation lease

customer receiver → inbound NC / bypass ─┬→ agent playback
                                         └→ incoming translation lease

customer receiver → inbound Accent ───────→ agent playback
                  └→ parallel inbound NC ─→ incoming translation lease

The stable destinations survive model swaps, mute/hold, translation start/stop, consult legs, and device recovery. Accent Conversion is a replacement branch, not an extra filter after NC: it already removes noise and background voices. When inbound Accent Conversion is on, the agent hears its result while Voice Translation receives the parallel NC-only branch. Local inbound BVC is not offered because the licensed browser BVC models are outbound-only.

Before redirecting the Phone's audio element to inboundPlayback, the manager starts one hidden HTML audio element on the original remote RTC stream at zero volume. Chromium otherwise stops supplying remote samples to Web Audio when that stream loses its last HTML audio consumer. Track state, a running AudioContext and NC's ready status do not detect this silence. The native Phone element still owns audible playback and its output-device selection; the source renderer is never an audible second copy.

_ensureInboundRenderer reuses the element for the same receiver track and starts a replacement before releasing the old element when the track changes. Generation checks discard superseded starts. An autoplay failure leaves the provider's original sender and playback untouched and reports inbound_audio_renderer_unavailable. Teardown restores native playback, removes the hidden element and clears its stream without stopping provider-owned tracks. Hold gates processed playback; microphone mute does not gate customer audio.

When customer NC and Accent Conversion are both Off, the Phone now plays the original receiver stream directly. The Web Audio destinations remain available for translation and diagnostics, but they no longer sit in the audible bypass path. A translation lease can keep its NC branch without enabling NC in the agent's ear. Hold temporarily selects the gated destination; resume restores the original stream. The native element retains its selected output device, volume and mute state.

play() rejection is reported as inbound_playback_blocked for browser permission failures or inbound_playback_failed for other failures. A later successful retry clears only that playback error. Superseded promises cannot mark a new source or retry as failed. Previously these errors were swallowed.

During an attached call, playback_health telemetry samples a 1.5-second window after setup, mode/hold changes, and every 15 seconds. It reports incoming packet and byte deltas, receiver energy where available, context/track/player state, gain, and whether playback uses the original or processed stream. A temporary rtc-level-meter AudioWorklet reports peak window RMS values before processing and after the playback gain. Only scalar levels leave the audio thread; no audio recording, waveform, device identifier or device label is sent. The server accepts only the defined enums, booleans and finite numeric fields. Hangup cancels periodic sampling; stale samples are discarded and metering taps release without disconnecting the call graph.

For a reported silent call, compare rawAudioRms and processedAudioRms while the customer is speaking, alongside playbackSource, gain and player state. Packets alone can carry silence. Receiver energy is also not sufficient by itself: controlled Chromium playback remained audible in a run whose energy counter stayed at zero. A quiet sampling window is not automatically treated as a fault. These diagnostics locate the failing stage; they do not measure whether a physical headset is audible.

A 2026-10-02 browser reproduction used a real customer-to-agent WebRTC receiver. Replacing native playback produced zero raw and processed samples; adding the silent source renderer restored both. The fixed manager passed NC On/Off, microphone mute, hold/resume and teardown checks, including a live remote track and no leftover renderer after teardown. A local MediaStreamDestination input does not reproduce this browser behavior and is insufficient for this test. The deployed two-way probe also kept incoming processed speech above 0.05 RMS while ElevenLabs/Sarah converted the microphone and after conversion was turned Off, with one zero-volume source renderer and the native playback element playing the processed stream.

Unchanged policy refreshes and equivalent per-call overrides preserve the live filters. When settings change, the previous graph keeps feeding the stable destinations until the replacement filters are ready; status reports loading alongside the modes still in use. A failed replacement leaves the installed graph connected. A running filter failure removes only that filter; an ElevenLabs failure reconnects the existing NC node immediately and preserves the other direction. A selected but unavailable ElevenLabs provider remains selected and reports accent_disabled_by_policy; it never silently switches to Krisp accent conversion. Re-enabling provider policy allows a retry. Status reports degraded with the failure reason. The master disable switch still selects bypass. Superseded model loads release their own filters and cannot overwrite a newer graph. Starting incoming translation requests its NC branch even when both customer-audio toggles are off; releasing the last translation lease removes that extra branch if customer playback does not need it.

Validate NC with received audio as well as effective_mode telemetry. A loaded model or requested On does not prove that the current WebRTC sender carries its output. Check the running AudioContext, installed NC node, stable destination track on the sender, and an Off → On → Off comparison at the receiver. Disable accent, translation and BVC for this test. Use the same speech/noise input each time, measure noise-only intervals, and check that speech remains present; a silent broken stream must not count as successful suppression. The agent microphone test must be heard at the customer end, not through local sidetone. See the agent test procedure.

On 2026-10-02, a controlled two-peer WebRTC test using the deployed manager and Krisp 2.3.9 reduced receiver noise by 43.3 dB in noise-only intervals of an 11-second recorded-speech fixture mixed with synthetic fan/hiss and keyboard-like bursts. Speech remained present; switching Off restored noise within 0.5 dB of the first Off pass. All three passes retained the stable sender track and reported no filter errors. This is a repeatable processing-path check, not a measurement of a live customer's audio or a guarantee for other noise types.

Models and krispsdk.mjs are extracted from the Git LFS archives in repository-root krisp/ by prepare_krisp_rtc_assets.py and served at /krisp/2.3.9/* with immutable caching. The SDK is loaded with useSharedArrayBuffer: false; NC models are preloaded, BVC loads only for a compatible ≥16 kHz microphone, and the much larger Accent models load only when enabled. A clean local checkout needs git lfs pull before either Krisp Docker image is built; the four CI build workflows enable LFS for both images.

GET /api/rtc-config is a non-secret policy endpoint. It returns defaults, allowed overrides, rollout percentage, version, and same-origin asset URLs. The connector refreshes it every 30 seconds; disabling the master switch hot-swaps every active graph to stable bypass audio without interrupting an active translation lease.

In Vanguard, webpack-dev-server proxies /api, /socket.io, and /krisp to the Express target derived from SERVER_PORT (3031). Do not hard-code the app's package default 3030 there: that port can belong to another service, so the SDK import fails with a proxy 502 while the direct Express asset is healthy. Compare GET :8101/api/rtc-config and GET :3031/krisp/2.3.9/krispsdk.mjs when diagnosing model-load failures. Set SCV_KRISP_RTC_ENABLED=false and restart the service to keep new calls on the original raw WebRTC tracks until a live-call validation succeeds.

Asset delivery is only the first contract. The licensed 2.3.9 ESM wrapper exports its SDK constructor as default (plus __esModule), not as a named KrispSDK export. The browser adapter accepts both shapes, preferring the named form for forward compatibility. A direct dynamic import and sdk.init() probe should pass before enabling the master switch; an HTTP 200 alone does not prove that the application can construct the SDK.

For an accent-only test, enable SCV_KRISP_RTC_ENABLED on the connector and set Start translation automatically → Off in the phone's My call defaults. The deployed devsdo Orchestrator Console assigns VoiceCall to Voice_Call_Record_Page; its trueservedev:translationSettings instance has autoStartTranslation=false for this test setup. Refresh Salesforce after the change. With Translate this call off, either Accent Conversion toggle works directly on the call audio without opening orchestrator translation streams. The accent defaults remain off so each direction can be compared independently. The component's language pair and translation agent are only used when translation is enabled.

The comparison page has the stable URL /accent-normalization/comparison.html, served from public/accent-normalization/ by the existing static-file handling in both development and production. Its Agent Studio presentation follows apps/ai-manager/app/studio.css and StudioBrandLockup.tsx; Open Sans and its license are bundled in the page's assets/ directory. The page shows titles, eighteen mono examples, ten expandable stereo timing players, individual downloads, and Download All Examples. Supporting transcripts and detailed metadata remain in the ZIP. Generated media is served from runtime/krisp/listening-tests/<run>/. Update the page's root-relative media references to a new run when replacing recordings because /krisp assets have immutable caching. These runtime artifacts are ignored by Git; preserve the recordings, transcripts, processing metadata, and branding provenance with the comparison. Capture and trim lossless PCM masters, then encode the playback/download files as 192 kb/s MP3s (mono examples, stereo timing); the download ZIP includes MP3 audio and supporting metadata, without WAVs. Keep intermediate PCM masters outside the published revision after processing.

The current bundle, accent-2026-10-01-fast-startup-v13, has seven rows: female Krisp clean and office pairs, the supplied Sanas pair, two ElevenLabs fast startup rows, and two earlier buffered test rows. The new rows show Original, Phrase buffering, and With grammar repair (Gemini 3.5 Flash-Lite). They use the first clean and office v4 replays, comparing the matched clause-buffer-only and grammar paths. The earlier rows retain Original, Buffered voice, and With grammar repair. Three-column rows stack below 900 px.

Timing · original L / output R under each processed Krisp/ElevenLabs player opens the matching stereo example. New mono and timing MP3s are copied byte-identically from v4, including startup silence. Their Time to first voice labels show 1.89 s for both clean paths and 1.78/1.80 s for office buffering/grammar. These are speech-onset-to-first-audible-output measurements; the earlier rows retain their different Median word delay metric. The v4 warm-session/simulated-playback limitations still apply.

All eighteen earlier MP3s are copied unchanged from v12. The ZIP contains exactly the twenty-eight page MP3s, source metadata, and transcripts. comparison.json records hashes and source runs; fast-startup-selection.json maps the four new outputs to v4. The full v4 results/events accompany them, while earlier/ retains the unchanged v12 metadata and listening cuts. The old male Krisp MP3s remain excluded from page and ZIP; their diagnostic bundles remain available.

The earlier standalone ElevenLabs cuts remove only leading zero-valued PCM from lossless playout masters, retaining 50 ms before the first nonzero sample. No spoken samples, internal pauses, or trailing audio are removed. Durations on the page and in the ZIP describe the shortened files. earlier/listening-edits.json records exact cuts and hashes. All six earlier stereo timing files remain byte-identical to v11, so timing players and latency metrics preserve their original capture/playback clock. earlier/elevenlabs-results.json explicitly assigns the timing statistics to those stereo files, not the shortened mono examples.

The female Krisp mono and stereo files share fresh real-time captures through the deployed KrispAudioEnhancementManager. A 48 kHz AudioBufferSource feeds its MediaStream input at playback speed; the deployed manager and both recorder inputs use one 44.1 kHz AudioWorklet clock. Consecutive frame indices verify a continuous capture. The accent model is ready before source playback, translation stays off, and no fallback or overflow is reported. This is streaming browser processing, not an offline render. Apply only one common leading/trailing cut to both channels and to their mono exports; never shift the processed signal to line up with the source. Private WAV masters and source videos are excluded from the bundle.

The timing recordings put original input on the left and processed output on the right. Krisp approximate RMS-envelope lag is 270 ms in both runs, using 10 ms windows; successive four-second windows agree. This includes the test MediaStream graph and excludes model loading. It is not intrinsic model latency or the matched-word statistic used for ElevenLabs. timing-results.json records methods, common trims, correlations, model/source hashes, and the selected ElevenLabs measurements; processing-female.json records the new capture. Krisp uses female speech, while ElevenLabs uses male speech and simulated buffered playback, so do not treat these as a controlled provider benchmark or Salesforce/Connect call measurement. The supplied Sanas excerpts use different speech and cannot establish a controlled L/R timing comparison.

The female Krisp inputs are Script1 _Aparnim.mp4 and Script 2 _Aparnim.mp4. ElevenLabs uses Script1_S 1.mp4 and Script2_S 1.mp4 as male source speech, with Joseph as the synthesized voice. Each speaker has a clean row from Script 1 and an office-background row from Script 2. Extract audio without publishing the video; preserve source hashes, recorded wording, and preparation details in the bundle's recording metadata. Add Office Conversation audio only to each Script 2 example. Within each original/normalized pair, retain the same prepared source recording. Regenerate both pairs when replacing the speaker. Earlier synthetic examples retain their ElevenLabs voice IDs and generation parameters in their archived bundles; do not carry synthetic-voice attribution into recorded-human examples.

Intelligibility is a separate acceptance check. The male recordings exposed wording loss even when the accent model reported ready and the output was non-silent. On 2026-09-29, real-time browser replays reproduced transcript differences in untrimmed PCM output, before MP3 encoding. Differences persisted without the added office noise, with an 80 Hz high-pass filter, in a shared 48 kHz context, and when bypassing the input MediaStream bridge. The supplied Indian-specific outbound v3.12 model also retained wording errors. These checks do not conclusively isolate the cause between the model and runtime behaviour.

An experimental Krisp NC → Accent cascade did not establish a fix; do not make serial preprocessing the default based on model readiness, noise reduction, or output RMS. The supplied 2.3.9 browser accentReduction reference app wires the microphone directly to the accent filter. Its capture guidance recommends echo cancellation enabled and browser noise suppression disabled, with AGC enabled for an 8 kHz device. That is microphone-capture guidance, not evidence that applying those controls to an existing recording restores lost words.

The diagnostic bundle runtime/krisp/listening-tests/accent-2026-09-29-streaming-diagnostics-v9/ contains diagnostics.html, MP3 examples, and diagnostics.json with the exact experimental settings. It is separate from the main comparison page and does not change live policy. Compare original and processed wording using both human listening and transcription; ASR edit counts are diagnostic proxies, not human-verified intelligibility scores. Inspect fallback/overflow telemetry and preserve the model tail, but do not treat absent events as proof that no transient underrun occurred. A browser replay does not replace a live Salesforce/Amazon Connect call check before accepting an integration fix.

ElevenLabs live transcription/resynthesis experiment. The separate runtime/krisp/listening-tests/elevenlabs-live-relay-2026-09-29-v1/streaming.html artifact replays the male clean and office-background clips through scribe_v2_realtime and persistent eleven_flash_v2_5 WebSockets. The selected voice is Joseph – Customer Support Agent (8fcyCHOzlKDlxh1InJSf), an American male conversational voice, at API speed 1.1. Three parallel paths use the same Scribe events: earlier transcript release, committed segments, and committed segments with optional GPT-4.1 mini grammar repair. No translation is performed.

The benchmark sends 100 ms PCM chunks only after their samples would have been captured, with two seconds of silence before source time zero. Earlier release holds two trailing words and requires 300 ms of unchanged text; VAD commits use 350 ms silence. Flash uses auto_mode, per-chunk flush, and 24 kHz PCM. Arrival times plus an ordered 50 ms playback buffer estimate the output timeline, including queue buildup and gaps. Offline forced alignment measures matched-word onset delays solely for evaluation; reference wording never enters the live generation path. These are host-side streaming measurements, not a Salesforce/Connect call or a headset latency measurement.

In these clips, earlier release had median matched-word delays of 4.9 s (clean) and 3.7 s (office), with 95th percentiles of 6.7 s and 5.1 s. Committed-text paths were slower; the optional repair requests took 0.43–0.83 s each. This tested configuration does not meet a sub-second target. Grammar repair preserved an ASR substitution of “pay for the order” for “place another order”; it cannot establish recovery of missing source words. The artifact contains fourteen MP3s, including stereo timing comparisons (original left, relay right), plus results and event metadata. It does not change the live call configuration or the main comparison page.

Buffered interim speech and early grammar repair (v2). The follow-up artifact is runtime/krisp/listening-tests/elevenlabs-live-relay-2026-09-29-v2/streaming.html. It contains eighteen MP3s: two originals, plus output and stereo timing recordings for the earlier baseline, a buffer-only control, continuous buffered speech, and continuous buffered speech with early grammar repair. The baseline and control reuse identical v1 PCM; the two new synthesis paths share a new Scribe session per recording. Provider/recognition variability therefore remains a factor when comparing v1 with v2.

The relevant Agent Studio implementation is apps/ai-orchestrator/lib/pipecat/services/elevenlabs_single_stream.py: one stream-input connection, plain text sends without per-chunk flush, and empty text at stream completion. pipeline/service_wrappers/elevenlabs_services.py forces auto_mode=true for translation and supplies [50,50,50,50] as the character schedule. V2 follows this wire pattern. Its Scribe-specific text buffer retains the 300 ms stability/two-word hold, groups roughly 50–80 characters, prefers punctuation, and bounds the phrase wait at 1200 ms. It adapts the existing phrase-grouping approach rather than running the Deepgram-oriented PhraseSplitter unchanged.

pipeline/builders/translation_builder.py inserts the actual frame_processors/phrase_aware_buffer.py processor with 800 ms PCM prefill, 2000 ms target, zero reserve, 8000 ms maximum, cross-session buffering, and a 300 ms inactivity drain. Those paths are relative to apps/ai-orchestrator/lib/pipecat/. Captured API packet arrivals were replayed through that processor at their original intervals; every PCM byte and its order was verified. PCM prefill is not a fixed wall-clock delay when a provider packet already contains substantial audio. Experiments with a 400 ms reserve and 2000 ms prefill could introduce gaps by retaining whole chunks; measurements are retained in results.json.

The published buffered variants use the builder defaults plus an experimental one-time 1500 ms playback hold after the first buffer release, then a causal FIFO playback clock with 50 ms transport allowance. This additional hold is a simulated client buffer, not an orchestrator default or deployed setting. It removed inserted packet gaps in these traces. Buffering alone does not remove pauses already synthesized into the PCM: the old clean output had no packet gaps. Continuous synthesis reduced generated interword pauses over 180 ms from 19 to 12 for clean audio and 13 to 7 for office audio. Natural sentence pauses count too, so this is a cadence proxy, not a human fluency score.

Optional GPT-4.1 mini repair now consumes punctuation-delimited clauses from interim text before the utterance commits. Incomplete clauses pass unchanged after an 1800 ms hold; ordered requests prevent reordering. An initial fragment-based pilot invented sentence completions, so the published path also rejects proposed content-word, number, negation, pronoun, or fragment-boundary changes. These conservative checks are not proof of semantic equivalence. The model receives only previous original context and the current clause, with the original clause as predicted output. Requests took 0.36–0.59 s. These mostly grammatical recordings yielded one substantive accepted edit, four rejected edits, and one incomplete pass-through. The ASR mistake “pay for the order” was still retained.

Published path	Clean median word delay	Office median word delay
Earlier baseline	4.9 s	3.7 s
Earlier baseline + buffer only	6.7 s	5.8 s
Buffered continuous speech	5.1 s	4.5 s
Buffered continuous speech + early grammar	7.1 s	6.8 s

These are matched-word onset estimates, with coverage and P95 in the result metadata. All configurations remain above the one-second target. MP3s preserve startup silence and playback timing; no offline pause trimming or time stretching was used. The ZIP contains MP3s and metadata, with no WAV downloads. This experiment does not measure Salesforce/Connect playback, enable translation, or change live-call configuration. The main accent comparison now includes the v2 originals, buffered outputs, and their stereo timing recordings, with and without early grammar repair. The earlier baseline and buffer-only controls remain on the separate experiment page.

Two-second startup experiment (v3). The acceptance target for this experiment is approximately two seconds from source speech onset to the first audible output. This differs from median matched-word delay and from click-to-start time. Preserve original wording in the first phrase; optional grammar repair must not block that phrase. Later repair also needs a playback deadline with an unchanged-text fallback. Grammar repair is disabled in this benchmark; Gemini is not tested.

The separate artifact runtime/krisp/listening-tests/elevenlabs-startup-2026-10-01-v3/ contains fourteen MP3s, results, stream events, file hashes, and the experiment scripts. Six stereo files retain original input on the left and simulated output on the right; six mono outputs retain startup silence. The other two files are the originals. The main comparison page and live-call configuration are unchanged. The scripts retain the host experiment's private source/config paths; credentials and configuration files are not included.

Three live Scribe replays per source each feed both Flash models. The first send releases at least two complete-looking interim words after 150 ms without a revision, without the earlier two-word hold or 1200 ms phrase wait. Subsequent sends retain one trailing word and bound phrase waiting at 300 ms. The first phrases are “Thanks for” and “I found”; these are real source words, not a synthesized acknowledgement. Flash keeps a persistent auto_mode connection, continuous text sends, Joseph at speed 1.1, and 24 kHz PCM. The selected playback simulation uses a one-time 600 ms startup hold plus 50 ms transport allowance. Every generated PCM byte and its order are preserved.

Model	Clean first audible output	Office first audible output	Largest inserted packet gap
Flash v2.5	1.82–1.94 s	1.80–1.92 s	143 ms
Flash v2	1.89–1.93 s	1.78–1.92 s	96 ms

Both models meet the startup target in all six warm-session replays. Flash v2 does not establish a consistent startup advantage. A 400 ms hold starts Flash v2.5 in 1.60–1.74 s but increases inserted gaps, reaching 343 ms. The selected 600 ms hold leaves no inserted gaps in four Flash v2.5 replays; the other two have approximately 142 ms and 27 ms. These counts exclude pauses already synthesized inside the audio.

Startup success does not validate this release policy for live calls. Every source replay has a later committed segment that differs from words already sent to synthesis. Examples include “you plans” before Scribe changes it to “your plans,” and “I have for it” before “I've arranged for it.” Generated speech still finishes roughly 4.0–5.3 s after the source with Flash v2.5. Improve continuation stability and verify word preservation before promoting this speculative release policy.

Connections and two seconds of real-time silence precede source playback. The source-speech anchor is 200 ms into each recording, from the existing dry-source forced alignment used only for evaluation. Audible onset requires three consecutive 10 ms PCM windows above −50 dBFS RMS. Input scheduling error is below 1 ms. These measurements exclude actual browser, Connect, microphone and headset latency; they do not establish a two-second cold-start guarantee.

Grammar after fast startup (v4). The separate runtime/krisp/listening-tests/elevenlabs-grammar-startup-2026-10-01-v4/ bundle compares four paths sharing each of six live Scribe replays: the v3 fast release policy, a clause-preserving buffer without repair, and that same buffer with either Gemini 3.1 Flash-Lite or Gemini 3.5 Flash-Lite. Gemini 2.5 Flash-Lite returned HTTP 404 twice for the configured Google integration, despite appearing in its model catalog; this does not establish global model unavailability. The available alternatives are named explicitly in the results.

All four paths send the first short phrase unchanged and retain Flash v2.5, Joseph at speed 1.1, and the 600 ms startup hold. The clause-preserving buffer releases punctuation-delimited tails, including a single word such as “message.”, instead of attaching the next sentence's “If you” to that tail. It otherwise targets 50 characters/eight words and releases at least three words after a 650 ms wait. This gives Scribe more time to revise the next clause before its words become immutable.

Repair requests run concurrently on fragments of at least three words, using only prior original context and the current fragment. Both Gemini models use minimal thinking. The playback deadline is the observed queued-audio end minus 200 ms, capped at 1200 ms from request start; missed deadlines send the original text. Late responses remain in the metadata as unplayed evidence. The local guard rejects changed content words, numbers, negation, speaker or recipient families, final punctuation, and unfinished trailing pronouns. It permits grammatical case changes such as “you” to “your” within the same person. These checks are not a semantic-equivalence proof.

Path	First audible output	Applied repair edits	Median inserted gap total per replay
Fast original wording	1.77–1.94 s	—	0 ms
Clause-preserving buffer only	1.78–1.93 s	—	387 ms
Buffer + Gemini 3.1 Flash-Lite	1.80–1.93 s	0	295 ms
Buffer + Gemini 3.5 Flash-Lite	1.80–1.91 s	1	387 ms

Each row covers three clean and three office replays. Both repair models meet the two-second startup target in all six trials. Complete repair-response latency has medians of 626 ms for 3.1 and 502 ms for 3.5, across 68 requests per model. The deadline rejects 27 and 21 responses respectively; another 20 short fragments per model bypass repair. Nine on-time 3.1 proposals are rejected by the content guard, including invented “one” and “it” completions.

Repair adds no material improvement in these recordings. Buffering alone fixes the clean sample's “you plans” to “your plans” in all three replays. Gemini 3.1 makes no applied edits. Gemini 3.5 changes “be sent to” to “to be sent to” once, leaving the malformed surrounding phrase “arrange for a new to be sent” unresolved. Both repair paths retain the recognition error “pay for the order” instead of the reference's “place another order.” The aggregate normalized text edit distance from the existing dry-source reference is 30 for fast release and 27 for each of the other three paths. This is a text diagnostic against an earlier ASR reference, not a human intelligibility score.

The grouped control also develops packet gaps, so do not attribute all additional gaps to the LLM. TTS generations are separate and variable; shorter duration or a lower gap count does not independently establish better fluency. First-phrase bypass preserves startup, but neither model demonstrates recovery of missing source content under this policy.

The bundle contains results for all 24 lanes, exact stream events, experiment scripts, hashes, and eighteen MP3s: two originals plus mono and stereo timing audio for all four paths in the first clean and office repeats. Selection is by run number, not best result. Startup silence and both stereo clocks are preserved. Eleven content-guard checks, all generated PCM bytes and ordering, and MP3 channels/durations are verified. Input scheduling error is below 2 ms. The v3 warm-session and simulated-playback limitations still apply. The main comparison includes the first-repeat grouped-control and Gemini 3.5 examples in v13; the v4 experiment bundle retains all four paths. Live settings are unchanged.

RTC_AUDIO_CONFIG carries a directional patch such as {outbound:{accentConversion:"on"}} from Call controls. The connector merges only switch fields into this call's snapshot, preserving voice/provider/grammar and the other direction. Provider, voice and grammar fields in legacy full messages are ignored, as are patches received outside a connected call. The snapshot must not rebase on newly saved defaults during a call; RTC_AUDIO_STATUS returns requested/effective modes for outbound, inboundPlayback, and inboundTranslation, BVC compatibility, model state, and the fail-open reason. The connector clears overrides on callEnded, even when the Salesforce utility panel is closed, so every new call starts from the agent's saved defaults, falling back to organization policy for inherited fields. RTC_AUDIO_STATUS_REQUEST rehydrates a panel that mounts mid-call. Structured telemetry covers model load, effective mode, fallback, overflow, device change, and parallel inbound processing. The bounded label includes saved and captured profile revisions plus the two selected providers (phoneRevision=…;callRevision=…;accent=outbound/inbound). Use the directional effective mode to distinguish elevenlabs_accent from Krisp accent_conversion; the manager's top-level telemetry provider label can still be krisp when ElevenLabs is active. Telemetry never includes audio content.

Live Translation architecture

The browser is a thin audio tap. All STT / translation / TTS runs in an ai-orchestrator translation agent authored and published in ai-manager, so no vendor API key ever reaches the browser.

flowchart LR
  cust([Customer]) -->|remote WebRTC track| capIn[PCM capture 16 kHz]
  mic([Agent mic]) -->|getUserMedia → gain gate| capOut[PCM capture 16 kHz]
  capIn -->|WS /ws/orchestrator| relayIn[Express relay]
  capOut -->|WS /ws/orchestrator| relayOut[Express relay]
  relayIn -->|gRPC stream: flow_direction=incoming| orch[ai-orchestrator]
  relayOut -->|gRPC stream: flow_direction=outgoing| orch
  orch -->|translated customer| earA[hidden audio element<br/>agent hears]
  orch -->|translated agent| earC[replaceTrack on PC sender<br/>customer hears]

Two streams, one per direction

This is the load-bearing invariant. ai-orchestrator's translation_builder.py builds a single flow per gRPC stream, chosen by the flow_direction metadata key on SessionConfig. AudioChunk.track_id is a dead proto field — the orchestrator neither reads it on input nor sets it on output — so one stream carrying both legs gets its audio conflated and translated by one flow in one language pair. The production path proves the shape: ari-controller → media-proxy allocates two sessions per translated call (allocator.go), A2B and B2A.

Stream	flow_direction	direction	Captures	Plays into
Customer	incoming	A2B	remote WebRTC track	hidden <audio> — agent's ear only
Agent	outgoing	B2A	getUserMedia mic	RTCRtpSender.replaceTrack() — customer's ear only

Each stream is one OrchestratorStream (src/translation/orchestrator-grpc/OrchestratorStream.js) with its own WebSocket to the Express /ws/orchestrator relay; the relay opens one gRPC stream per WebSocket, so two streams come for free with no relay changes.

Crossing the wires is the classic failure: play both streams onto the sender and the customer hears a translation of their own voice while the agent hears nothing.

Capture is an AudioWorklet (public/lib/pcm-capture-worklet.js) that decimates 48 kHz → 16 kHz behind an anti-alias FIR and emits ~20 ms Int16 buffers, with a ScriptProcessorNode fallback. The mic tap is never taken from the PC sender track — that track gets replaced by the translated stream.

SessionConfig per stream

Mirrors media-proxy (internal/proxy/session.go):

Field	Value	Why
session_type	1 (ASSISTANT)	0 is AGENT. Translation behaviour comes from the agent's agent_type=translation_agent, not from this enum
language	"auto"	The default pair lives in the agent's config.translation.flows.{incoming,outgoing}; a per-call override rides metadata.source_language/target_language instead (next section)
agent_id	a published speech_to_speech / translation pipeline id	Versioned :suffix ids are only honoured for ai-manager test calls
metadata.flow_direction	incoming | outgoing	Selects the flow
metadata.direction	A2B | B2A	Sent for parity; the orchestrator accepts either key
metadata.conversation_id	the SCV vendorCallKey	Shared by both streams so the legs land on one conversation
metadata.call_source	scv-connector	Provenance

Audio in is PCM16 @ 16 kHz; audio out is PCM16 @ 24 kHz, but playback trusts ServerHello.output_audio_format rather than assuming. skip_recording in ServerHello metadata is a media-proxy-only directive; echo_volume is honoured (next section).

Echo ducking is per-leg, and the two legs start from opposite extremes

The agent's flows.<direction>.echo_volume sets how loud the other party's untranslated voice is in this listener's ear — quiet presence under the translation, ducked further while the translation actually speaks. media-proxy has always done this on the SIP path; the browser path ignored the metadata, and the result was maximally asymmetric, which is why neither leg read as a bug on its own:

Leg	Before	Why
Agent's ear	raw customer at 100%	the telephony stack's own <audio> element keeps playing alongside the translation — two voices, both full volume
Customer's ear	raw agent at 0%	replaceTrack() substitutes the synthesized track outright, so no presence at all and dead silence between translated phrases

EchoDucker (src/translation/orchestrator-grpc/EchoDucker.js) applies one decision table to both legs through two different sinks, because the mechanisms are structurally opposite — one leg attenuates audio that is already there, the other adds audio that isn't:

Leg	Sink	Mechanism
Agent's ear	mediaElementSink	the original is already rendered by an element the telephony stack owns (Streams' #remote-audio; Genesys creates a private unnamed one). We cannot insert a node into that path, so we set element.volume — on the element the provider names, not one we find (below)
Customer's ear	gainNodeSink	the original is not in the customer's path at all, so it is added as a GainNode from the mic into the synthesized track's MediaStreamDestination

Four non-obvious properties:

• The wire value is a percentage. metadata.echo_volume is "10", not 0.1 — the orchestrator forwards the authored value verbatim and media-proxy divides by 100. parseEchoVolume keeps the /100; reading it as a fraction requests 1000% gain. It returns null rather than the default when absent, so the started log distinguishes 10% (default) from an authored 10%.
• The level is per-flow, so it is read per stream. incoming.echo_volume governs the raw customer voice in the agent's ear, outgoing.echo_volume the raw agent voice in the customer's; each ducker takes its level from its own leg's ServerHello.
• Mute must gate the echo path. The customer-leg echo graph feeds the destination directly, bypassing the sender track platform mute acts on — so setMuted reaches the ducker too. Without it a "muted" agent still leaks their raw voice at the echo level: quieter than before this feature existed, and therefore easier to miss.
• Idle releases rather than pinning a gain. Stop and hold both restore the element's original volume (remembered per element the first time it's touched, so we hand back what the platform set rather than a hardcoded 1.0) and drop the added gain to zero. Ducking hold music to 10% would be worse than not ducking at all. stop() un-ducks as step 0 — if anything later in teardown throws, the agent must not be left on a call they can barely hear.

getElements() is re-invoked on every change and reassertOutgoingTrack() calls refresh(), since a renegotiation can attach a new element at full volume. A leg with no element found warns loudly and leaves the status quo — the failure mode is "no ducking", never "no audio". Depth (0.01) and fade (20 ms linear) mirror TranslationDuckedFloor / translationFadeTimeMs so both paths sound the same, and neither leg is ever ducked up past an authored value below the floor. Unit tests: echo-ducking.test.mjs.

Ask the telephony provider for the element — a DOM scan can't find it

originalVoiceElements() (in EchoDucker.js) asks TelephonyProvider.getRemoteAudioElements() first, and only then falls back to scanning the DOM for an element with a live inbound audio track. The scan alone does not work on either real provider, and the reason is worth keeping:

volume is a property of the element, not of its srcObject. Setting it before playback starts is valid and durable — connect-rtc assigns srcObject inside its _ontrack and never touches volume. So requiring a live stream before touching an element adds a condition the mechanism doesn't need, and that condition is unmet in the normal case:

• Connect hands #remote-audio to the softphone session at startSession, but srcObject only lands when the first inbound track arrives — and translation starts from callConnected, which fires on signalling. On an outbound call that is well before any media (see waiting for the track), so the correct element is sitting right there and a stream-gated scan skips it. Reading it back off the session doesn't help either: session.remoteAudioElement is setter-only in connect-rtc 1.1.26. (Until 2026-08-05 there was a second, worse reason: the CCP iframe ran its own competing softphone session and frequently won the call, putting playback in a cross-origin document we could neither read nor write. allowFramedSoftphone: false ended that — see one softphone, in our frame.)
• Genesys keeps its element entirely private (defaults.audioElement), so it never appears in a scan until its stream goes live.

The tell is [EchoDucker] agent-ear: no media element is rendering the original voice logged repeatedly on a call whose audio is otherwise perfect — the customer leg ducks correctly (it owns its own GainNode), so only half the feature is missing.

Each provider therefore returns its own element by construction: Connect returns #remote-audio regardless of stream state (ensureRemoteAudioElement() guarantees it exists in our document, and that guarantee is separately load-bearing for the session itself — see #remote-audio); Genesys returns defaults.audioElement plus any per-session override. The base implementation returns [], which is safe: ducking degrades to "no ducking", never "no audio". The DOM scan is kept as a backstop for the demo/browser providers and for an element a renegotiation attaches that the provider doesn't know about; it still requires a live track there, because without one an arbitrary muted <audio> on the page is not evidence of anything. Results are deduped by reference, since once playback starts the provider's element also satisfies the scan.

The provider must be threaded to all three initialize() call sites — startTranslation in index.js, and updateConfig's restart path, which re-passes this._telephonyProvider because initialize() assigns unconditionally and omitting it would silently drop ducking on every mid-call restart.

Session-level language override

The published agent authors the default language pair, but a session may override it: the translation builder reads source_language / target_language (or the direction-prefixed incoming_* / outgoing_* forms) from SessionConfig metadata — checked before session variables — and lets them beat both the flow- and stage-level authored values (_resolve_flow_languages_with_override in translation_builder.py; the override is logged loudly as SESSION LANGUAGE OVERRIDE, so an unexpected pair in production is diagnosed from the session metadata, not the agent config). Absent keys = authored config, so media-proxy/ari-controller callers are untouched.

The SCV panel's Customer/Agent language pickers ride this: the connector maps them per direction (incoming targets the agent language, outgoing targets the customer language) into each stream's metadata — but only when the user explicitly picked them (languagesExplicit); an untouched panel sends nothing, so the config.js demo defaults can't override an authored agent. Changing languages (or the agent id) while translation is active restarts both streams (~2-3 s gap) — provider connections bake the language in at connect time (Gemini's translationConfig rides the WS setup message, Deepgram's language is a connect param), so there is deliberately no mid-stream mutation. Flipping the panel toggle on with no call at all arms translation — it starts automatically on the next callConnected instead of silently doing nothing. A toggle during a live call always starts immediately; see A manual start must ask the provider for the live media.

Hands-off operation: default-on, automatic languages, automatic stop

An agent shouldn't have to touch the panel. Three App Builder properties on translationSettings remove that need, and all three are fail-open — every fallback lands on "translate with the agent's authored pair", never on "no translation":

Property	Effect
Start translation automatically on each call	autoStart in TRANSLATION_CONFIG → translation starts on callConnected
Ignore the agent's Salesforce Locale (off by default)	Off = @salesforce/i18n/locale reduced to its language subtag (en_US → en, es_AR → es) → the Default Agent Language property. On = always the Default Agent Language property. Inverted because LWC rejects a public Boolean initialized to true (LWC1503)
Customer Language VoiceCall Field	API name of a VoiceCall field (e.g. Customer_Language__c) read on call_connected, so routing/flow/IVR can set it

The customer-language read goes through Apex (TrueServeVoiceCallLookup), not LDS. The utility-bar placement resolves by the telephony vendor call key because it has no record context. The record-page placement requires that key to match the page's VoiceCall or a VoiceCall whose RelatedRecordId is the page record, preventing a stale page from contributing settings to another active call. The field API name is admin-supplied, so it's validated against the VoiceCall describe (exists + accessible + text-like) before reaching the query, and agents need class access — granted by the packaged TrueServeSCVService permission set (labelled Agent Studio for Salesforce User).

Ordering matters: the connector publishes TRANSLATION_STATUS with reason: 'call_connected' before auto-starting and then holds for up to 4 s for the panel's language push (perCallLanguages), rather than starting on stale values and restarting a moment later. A missing field or slow read just lets the hold expire.

Automatic stop covers the whole chain, not just the browser: callEnded → stopTranslation() → TYPE_DISCONNECT on both streams + replaceTrack restore + both WebSockets closed → the relay's ws.on('close') calls grpcStream.end(), so each orchestrator session is torn down rather than left running on a dead call. publishTranslationStatus('call_ended') then resets the panel.

ScvBridge must emit call lifecycle events before its own bookkeeping

_onCallConnected and _onCallEnded both used to return early when the call wasn't in SCV's _activeCalls map — and in _onCallEnded that return sat above the local eventEmitter.emit. Since translation start/stop is driven purely by those local emits, both were hostage to SCV bookkeeping that can legitimately miss a call (a reconnect, a refresh-driven transition, a duplicate onEnded, a provider event-ordering race, an already-reaped contact). Symptoms were asymmetric and both bad: translation silently never started even though media was live, and on the stop side both gRPC streams stayed open with the synthesized track still on the sender and the orchestrator session billing away after hangup.

Both now emit first and unconditionally, carrying callId separately because an unregistered call emits call: null. Any new lifecycle handler in ScvBridge should follow the same rule: the local emit is the translation layer's only signal, and media liveness does not depend on SCV having a PhoneCall object.

Connector → panel status channel

The panel's toggle would otherwise be write-only: it flips optimistically and has no way to learn that the call ended or the start failed, so it sits on "Translating…" and the next click reads as "stop" and does nothing. publishTranslationStatus() in src/connector/index.js publishes TRANSLATION_STATUS (active, armed, reason, callId, agent id, language pair) on every transition, via ScvBridge.sendToSalesforce() → publishEvent({eventType: SHARED_EVENT_TYPE.MESSAGE}) → trueServeBridge.publishMessage() → {source:'CONNECTOR', payload}. Two rules: panels accept only source === 'CONNECTOR' (the mirror of the bridge's filter), and a utility-bar panel must subscribe with APPLICATION_SCOPE — the utility bar is outside the active navigation scope, so a default-scoped subscription silently never fires. baseConnector.js does no payload validation on the MESSAGE event, so an arbitrary object rides it. Full contract: docs/translation-lwc-integration.md.

A manual start must ask the provider for the live media, not the event cache

lastPeerConnection/lastRemoteStream/lastCallId in src/connector/index.js are populated only by the callConnected event. That event does not re-fire for a call already in progress, so anything that misses it leaves the cache empty for the rest of the call: a panel that loads mid-call, a console tab reload, or Connect capturing the RTCPeerConnection after contact.onConnected already ran.

The toggle used to gate directly on lastPeerConnection, so in that state enabling Translate this call mid-call fell through to the arm branch — the panel showed Waiting for call while a call was live, and because arming only starts on the next callConnected, translation never started at all and nothing reached the orchestrator. Every server-side signal was silent rather than wrong, which is what made it read as a translation failure instead of a start that never happened.

resolveLiveMedia() now asks telephonyProvider.getPeerConnection() first — both providers derive that from live session state — and derives the remote stream off that PeerConnection's own receivers (streamFromPeerConnection: the first receiver whose track.kind === 'audio' and track.readyState === 'live'), falling back to getRemoteAudioStream() and only then to the cache, which it re-seeds so a later language change restarts against the same leg. The receiver-first ordering matters because Connect builds a new RTCPeerConnection per leg: a cached stream belongs to whichever PC was live when callConnected fired, so preferring the cache can pair a stale customer track with a fresh PC. startTranslation resolves the vendor call key the same way, via the new optional TelephonyProvider.getActiveCallId() (Connect: primary contact, else the first connected agent leg; Genesys: the session map is keyed on conversationId), because an empty conversation_id would split the two gRPC streams onto separate conversations. Arming remains correct for its actual case: the toggle pressed with no call at all.

An outbound call connects before its media, so the start must wait for the track

callConnected fires when signalling says connected. On an outbound call that is before the customer's media is negotiated, so pc.getReceivers() has no live audio track yet and OrchestratorGrpcProvider.initialize() rejects the start with remoteAudioStream is required. Auto-start is the path that hits it, every time, because it fires the instant the event lands (AgentStateChange: CallingCustomer → Busy in the console is the tell that the call was outbound).

Note that remoteAudioStream is required has two unrelated causes, and this is only one of them — a null peerConnection skips the wait below and produces the identical message. Check for callConnected: no peer connection captured in the log first; if it's there, the cause is the PeerConnection, not the media (next section).

waitForRemoteAudio(pc) in index.js closes the race: it resolves immediately if the track is already there, otherwise it listens for track and polls every 150 ms — ontrack may already have fired for a track attached before we subscribed, and a track can reach live after its event — then gives up after REMOTE_AUDIO_TIMEOUT_MS (5 s) with a warning and resolves null, so a genuinely media-less call surfaces the provider's own error instead of hanging.

updateConfig's restart path re-derives the customer track the same way (liveAudioStream(pc) in OrchestratorGrpcProvider.js) rather than reusing the snapshot it was initialized with: a renegotiation between start and now (consult, conference, device change) ends that track, and initialize() only checks the stream is present, not that it carries live audio.

Streams tells you about the RTC session before it has a PeerConnection

Both of Streams' documented capture hooks hand back the RTCSession, and session._pc is not populated yet when either fires. The ordering inside connect-streams 2.25.0 is fixed:

line	what happens
:18257	rtcSessions[agentConnectionId] = session
:18259	publishes SESSION_INIT — onSoftphoneSessionInit runs here
:18333	session.connect(...) — _pc is assigned in here

So a capture site that reads getSession(connectionId)._pc at SESSION_INIT reliably sees null, gives up, and _peerConnections stays empty for the whole call. getPeerConnection() then returns null, callConnected is emitted with peerConnection: null, and the start fails — reporting remoteAudioStream is required, because a null PC skips the media wait (above) and the stream is what initialize() complains about first. The misdirection is total: the message names the audio, the cause is the PeerConnection, and the console shows callConnected: no peer connection captured several lines earlier as the only real clue.

ConnectTelephonyProvider therefore captures at construction instead of asking Streams. installPeerConnectionObserver() (called before initCCP) wraps window.RTCPeerConnection with a pass-through that records each instance in a bounded observedPeerConnections list; newestObservedPeerConnection() returns the newest PC with a live inbound audio receiver, else the newest negotiated one (see the pre-warmed spare). This is timing-proof because the vendored connect-rtc's StandardStrategy builds the PC with a bare new RTCPeerConnection(a, b), resolved against the call-time frame global — and that global is ours: our frame is the softphone master (connect.isCCP() is false for us, since our upstream conduit is an IFrameConduit, not ConnectSharedWorkerConduit, so initSoftphoneManager takes the non-CCP branch and competes for master here), and public/lib/connect-rtc-*.min.js is loaded from our own origin by app.html. The confirming evidence in a browser log is any connect-rtc-1.1.26.min.js:* line (session is idle from long time…, refreshing peer connection) — those strings exist only in our copy, never in connect-streams.js.

Our frame running a softphone session does not mean it is the only one: with allowFramedSoftphone: true the embedded CCP iframe runs its own session too, and that one is what actually carries the call's audio if ours fails to build a PC. Both frames' logs land in one console (see #remote-audio), so attribute every line to a frame before drawing conclusions from it.

Three consequences to preserve:

• The wrapper must not change behaviour — it calls the native constructor and only records the instance, copying statics via Object.setPrototypeOf (connect-rtc calls RTCPeerConnection.generateCertificate()). An RTC.js new enough to ship RtcPeerConnectionManager may build PCs elsewhere; then the observer records nothing and the Streams hooks still apply. RtcPeerConnectionManager does NOT exist, please upgrade RTC.js in the console is not an error — the vendored 1.1.26 predates it, so Streams takes the legacy new connect.RTCSession(...) branch, which works (Amazon's own comment at :17901 says so).
• _bindPeerConnection(connectionId, context) is the one resolver both hooks call. It polls getSession(connectionId)._pc briefly (20 × 50 ms — authoritative about which leg the PC belongs to) and falls back to the observer (which knows the PC but not the leg). When the bound PC differs from what was already recorded it re-emits callConnected via _announcePeerConnection, because that event does not fire twice and a late PC would otherwise never reach the translation layer.
• getPeerConnection() falls through to the observer as a last resort, and startTranslation re-resolves a null PC through it rather than trusting its caller — failing with an explicit "no RTCPeerConnection for this call" rather than a misleading stream error. Correspondingly, the toggle's arm decision now keys on getActiveCallId(), not on whether media resolved: arming during a live call is always wrong, because arming only fires on the next callConnected, which never comes for a call already in progress.

The newest observed PC is usually a pre-warmed spare, not the call's

Construction-time capture solves the timing problem but introduces a selection problem: which observed PC belongs to the call? "The newest still-open one" is the intuitive answer and it is wrong, because Connect keeps a warm PeerConnection on standby to skip ICE setup on the next call.

RtcPeerConnectionFactory.get() (connect-rtc 1.1.26) hands its warm PC to the call and then immediately builds a replacement via _requestPeerConnection, refreshing that spare on a timer (RTC_PEER_CONNECTION_IDLE_TIMEOUT_MS) — which is where the session is idle from long time… / refreshing peer connection console lines come from. Streams takes this path whenever RtcPeerConnectionManager is absent (connect-streams.js:18332), i.e. on our vendored RTC — the same condition that logs does NOT exist, please upgrade RTC.js. So by the time a call is connected, the newest constructed PC is normally the untouched spare and the call's PC is an older observation.

The spare is invisible to every health check: signalingState is 'stable', it is not closed, and it has no transceivers at all. Handing it back produced callConnected audio path: {senders: Array(0), receivers: Array(0)}, which then tripped _recoverOutboundAudio into getUserMedia + addTrack — attaching the agent's mic to a PC nobody was listening to, and renegotiating it.

isNegotiatedPeerConnection(pc) is the discriminator: a negotiated PC has transceivers (connect-rtc's addStream does addTransceiver per track, and the remote answer adds receivers) or at least a local/remote description; a spare has neither. newestObservedPeerConnection() prefers live inbound audio, then the newest negotiated PC, and never falls through to an un-negotiated one — a wrong guess is worse than none here, since a spare looks perfectly healthy while carrying no media. The callConnected diagnostic also skips recovery outright when the PC isn't negotiated, leaving the background _bindPeerConnection to re-announce the real one.

Note session.connect(pc) adopts a passed-in PC (a && "closed" != a.signalingState ? b._pc = a : …), so session._pc remains the authoritative answer about which leg a PC belongs to — the observer is only ever the fallback.

app.html must carry <audio id="remote-audio">, or the softphone session dies half-built

The most deceptive member of this family: the call sounds perfect and translation cannot start, because our frame ends up with no PeerConnection at all.

SoftphoneManager.startSession resolves the remote audio sink like this (connect-streams 2.25.0 :18328; setSpeakerDevice repeats it at :18528):

session.remoteAudioElement = document.getElementById('remote-audio')
  || window.parent.parent.document.getElementById('remote-audio');

We are doubly framed — Salesforce hosts our frame — so window.parent.parent is cross-origin. With no #remote-audio in our own document the first operand is null, the || evaluates the second, and it throws SecurityError on the line immediately before session.connect(pc) (:18333).

Everything downstream then points away from the cause:

symptom	why
onSoftphoneSessionInit fires normally	SESSION_INIT is published at :18259, above the throw
_bindPeerConnection polls the full 1 s and warns no RTCPeerConnection for <id>	session.connect() never ran, so session._pc is never assigned
session is idle from long time… / refreshing peer connection	the factory's warm PC was never consumed by a call
the only observed PC is an un-negotiated spare	correctly refused by newestObservedPeerConnection()
the error reads Script error. null, naming nothing	cross-origin exceptions are opaque to the host page
contact::refresh::<contactId>' event handler failed.	the one honest clue, logged right after our StandardStrategy initialized

While the CCP iframe still ran its own softphone (before the single-session fix) this was even more deceptive: the call worked, audio both ways, because that other session carried it. Now the element is the only thing standing between a working softphone and none at all, so the same defect would take the call down with it — louder, and easier.

Reading the console correctly matters here. Streams relays the CCP iframe's log entries into our logger (conduit.onUpstream(connect.EventType.LOG, …), :11894), so lines from the CCP's newer Streams build print with our connect-streams.js:15616 prefix. RtcPeerConnectionManagerV2, SharedMediaSession, Using pre-fetched standby peer connection, Initializing Voice SDK Client, and PCM creation to first call appear nowhere in our vendored bundle — they are the CCP's, not ours. Conversely connect-rtc-1.1.26.min.js:* lines are only ever ours. Grep the vendored file before attributing a log line to it. (Those particular lines should no longer appear at all now that the CCP's manager is stood down; if they do, check that both allowFramedSoftphone call sites are false.)

The fix is one element in public/app.html:

<audio id="remote-audio" autoplay></audio>

ensureRemoteAudioElement() (called before initCCP, beside installPeerConnectionObserver()) injects it and warns if the page is missing it, so a future template edit degrades to a named warning instead of a silent dead session. Keep the id exactly — Streams matches it literally.

One softphone, and it must be ours

Everything above describes finding the call's RTCPeerConnection in our frame. That only works if the softphone session that carries the call runs here — and until 2026-08-05 it usually didn't, because two frames ran a SoftphoneManager and the CCP's normally won.

allowFramedSoftphone gates whether the CCP iframe's own manager competes (initSoftphoneManager: if (data.softphone && data.softphone.allowFramedSoftphone)). With it true, both frames answered the same RTC invite, Lily rejected the second with 409 Duplicate RTC invite, and on every observed call ours was the one rejected. The result was the whole family of symptoms above at once, with none of their individual causes: the call sounded perfect (the CCP's session carried it), our session._pc was never assigned, the only PC the observer saw was the un-negotiated spare, and the start failed no RTCPeerConnection for this call.

The softphone has to run in our frame, and not for preference — by construction:

• translation needs same-document access to the call's RTCPeerConnection (to tap the remote track and to replaceTrack the sender) and to #remote-audio (to duck the original voice);
• a MediaStream is not structured-cloneable, so it cannot cross postMessage.

A CCP-owned session is therefore unreachable, not merely inconvenient. Both call sites must agree (ConnectTelephonyProvider.js, config.js):

connect.core.initCCP(container, { softphone: { allowFramedSoftphone: false }, … });
connect.core.initSoftphoneManager({ allowFramedSoftphone: false });   // in _onAgentReady

The second false is easy to get wrong. That parameter lands in connect.core.softphoneParams, and triggerReadyToStartSessionEvent routes the webrtc-session kick upstream to the CCP iframe when it reads true — where no manager runs any more. true here alongside false in initCCP is how the two competing sessions were configured in the first place.

An older comment claimed false silently broke outbound dial. That predated #remote-audio in app.html: without the element our session threw cross-origin one line before session.connect() (previous section), so standing the CCP down left no working softphone anywhere. The element exists now, and the 409 proved our session completes getUserMedia, ICE and signalling — it lost only to the duplicate.

One operational consequence: mic permission now matters for our origin. It propagates through the outer Salesforce iframe's sandbox/allow attributes, but if the called party can't hear the agent, grant mic access to scv-translator.<domain> from the lock icon in the URL bar.

Build the provider into a local — never assign the singleton before it can throw

startTranslation used to assign the module-level translationProvider and then await initialize(). When initialize threw, a half-built provider stayed installed, and that one line produced a cascade:

• Every later config push took updateConfig's restart path, because currentAgentId was still undefined so agentChanged always fired — and each restart re-threw the same error. One uncaught rejection per panel keystroke.
• !translationProvider guards elsewhere read the corpse as "running", so handleTranslationToggleEvent would not retry the start.

The provider is now built into a local, initialize() is wrapped so a failure calls provider.stop() and rethrows, and the singleton is assigned only on success. start() is wrapped the same way (failure → stopTranslation({silent: true}) + rethrow), and handleTranslationConfigEvent's restart is wrapped so a config push — a side effect of the agent typing — reports start_failed: to the panel instead of escaping as an unhandled rejection.

Lifecycle transitions are serialized — a config push landing mid-start used to tear down the start

Start, stop, a config-driven restart, and a track re-assert all mutate the same audio plumbing: the provider's two gRPC streams, its two capture graphs, and the synthesized track installed on the PeerConnection sender. Each is many awaits long — two WebSocket connects, a getUserMedia prompt, a replaceTrack renegotiation — so two running concurrently means the loser dereferences fields the winner already released.

That is exactly what happened. start() reaches this._toCustomerAudioStreamManager.getAudioTrack() four awaits deep; stop() nulls both audio managers but leaves _remoteAudioStream set, so start()'s entry guard (if (!this._remoteAudioStream)) still passed. A TRANSLATION_CONFIG push arriving mid-start took updateConfig's restart path (stop() → initialize() → start()), and its stop() pulled the managers out from under the in-flight start, which died on Cannot read properties of null (reading 'getAudioTrack'). The panel reported "Could not start translation" for a start that was healthy right up until something else tore it down.

The push is not an unlucky coincidence — it is structurally concurrent with the start it unblocks. callConnected publishes publishTranslationStatus('call_connected'), which is the panel's cue to resolve the customer language from the VoiceCall record and push it back; that same push releases waitForPerCallLanguages() and then continues into the config handler. Every hands-off call runs both paths at once.

Two independent layers now close it:

Layer	Where	What it does
One lane	serializeLifecycle() in index.js	Every transition (toggle, config, callConnected, callEnded, participantConnected, the exported updateTranslationConfig) queues on a single promise chain, so each one sees a settled provider. A rejection does not poison the lane.
Generation counter	_lifecycleGeneration in OrchestratorGrpcProvider.js	start() and stop() both bump it; start() captures its value and re-checks after every await via _startSuperseded(). A mismatch (or a nulled manager) abandons the start quietly.

Four details that are easy to get wrong when touching this:

• The stop() bump happens before its early return. An in-flight start must learn it was superseded even when stop() finds nothing to tear down — the start may not have assigned its streams yet (it can still be inside getUserMedia). Without the early bump, that start ran to completion and re-installed its synthesized track after the stop had restored the agent's mic.
• Abandonment releases the start's own handles, never this._*. Each start tracks what it opened in a local owned bag. By abandonment time a newer start() may have installed its own stream objects on those fields, and stopping those would close the winner's WebSockets and kill translation silently. Instance fields are cleared only when they still point at our handles.
• A superseded start resolves; it does not throw. There is nothing for the agent to act on, and reporting it would flip the panel to "Could not start translation" while the winner is starting successfully. startTranslation therefore re-checks translationProvider !== provider before publishing started.
• waitForPerCallLanguages() stays outside the lane. It waits for a push that needs a lane slot of its own; holding a slot while waiting for it would deadlock both until the 4 s timeout. The resulting order is the good one: the push lands in activeConfig, releases the hold, its own (now no-op) lane entry runs ahead of the start, and the start begins with the resolved languages already in place — no start-then-restart gap at the top of the call.

Related: updateConfig's restart path is now gated on this._running for both an agent change and a language change (it was agentChanged || (langChanged && running)). A stopped provider has nothing to restart, and restarting one is how a failed start became permanent — the connector's error paths leave a stopped provider installed for a moment, and a config push arriving then rebuilt it, re-threw, and reported another failure. The values are merged instead, and the next real start picks them up.

A config push must compare, not just arrive — the panel re-sends everything on every push

Two sibling bugs, both from treating a value's presence as evidence it changed. The panel sends its whole config on every push (there are no partial updates), so any handler that keys on presence fires on every keystroke, every status-driven resolve, and every mount.

The engine-swap branch. handleTranslationConfigEvent's first branch is the crude full-rebuild path — stop() then startTranslation() — meant for an engine change (orchestrator-gRPC ↔ the browser demo modes). It was gated on lwcConfig.translationProvider && translationProvider && lastPeerConnection. But the panel's engine picker was dropped for platform mode, so _sendConfig() pins translationProvider: 'orchestrator-grpc' on every push — making that presence check always true and every push a stop-and-start of a healthy translation (~2–3 s of dead air), skipping updateConfig's change detection entirely. Worst case is a panel mounting mid-call: its connectedCallback seed tore down a live translation for nothing. Now gated on a real comparison against the previous value, captured before the merge that would erase it.

The mount-time language clobber. The same seed carries the panel's design-attribute default languages, because _applyStatus discarded the customerLanguage/agentLanguage the connector reports. Once the engine branch was fixed, those pushes correctly routed to updateConfig — where a default that differs from the live pair reads as a language change and restarts both streams into the wrong language. A worse failure than the gap it replaced, and only reachable via that fix, which is why the two ship together.

The panel can't decide this alone: it mounts mid-call on any console tab reload or utility-bar reopen, and its status request is still in flight when the seed goes out. Withholding the languages unconditionally would fix the clobber and break the good case — an auto-start would then begin on the authored pair and restart when the panel's push landed, which is the gap waitForPerCallLanguages() exists to avoid. So the seed is labelled, and the connector decides:

• The panel marks its mount-time push seed: true (_sendConfig({seed: true})). It still carries the languages.
• handleTranslationConfigEvent applies a seed's languages only when nothing is running (seededLanguagesIgnored = seed && translationProvider). Idle → they land, and a later auto-start begins with them. Live → they're ignored and logged, and they are also stripped from the updateConfig call, or it would restart the very call the check protected.
• _applyStatus adopts the connector's languages while status.active and the agent hasn't picked by hand, so the pickers show what is actually being translated rather than stale defaults. Independently correct: the pickers are what the agent believes is happening.

The general rule for this panel: an omitted key means "unchanged"; a present key does not mean "changed". Compare against current state before acting on anything expensive.

Mute, hold, and renegotiation

Platform mute operates on the original mic track, which is no longer on the sender — so mute has to be enforced in our graph:

• Mute gates the mic GainNode (OrchestratorGrpcProvider.setMuted) and cancels the customer-bound playback — which means stopping scheduled sources, not emptying a queue (below).
• Hold pauses both captures and cancels both legs' playback.
• Unmute and resume re-assert replaceTrack. genesys-cloud-webrtc-sdk's setAudioMute can release and re-acquire the mic device, installing a fresh track on the sender and silently discarding ours — the customer would hear the raw agent for the rest of the call. reassertOutgoingTrack() is idempotent, so it runs unconditionally on those transitions rather than feature-detecting.
• New legs re-assert too. Connect's participantConnected (from both contact.onConnected and the onRefresh transition) carries the agent-leg peerConnection, because resolving it later via getPeerConnection() returns the primary contact's PC — the wrong leg during a consult.

stop() restores the original sender track, sends Control TYPE_DISCONNECT on both streams, stops the worklets and mic, and removes the audio elements.

Playback is scheduled on the audio clock, never chained on onended

Reported as "the customer translation is taking a LONG time", which sounds like a network or provider delay and is neither: playback was being stretched.

Two facts combine. First, the wire carries ~40 ms chunks — audio_out_10ms_chunks defaults to 4 in upstream pipecat's own TransportParams (pipecat/transports/base_transport.py:69, in the installed package, not our tree) and nothing on the translation path overrides it, so a one-second utterance arrives as ~25 separate buffers. Second, the client used to await source.onended before starting the next one. onended is a main-thread event, so every chunk paid a full event-loop round trip of silence after its audio finished. That gap is per chunk, so it accumulates: at ~10 ms of scheduling latency per 40 ms chunk, playback runs at roughly 0.8× realtime and slips further behind for as long as anybody keeps talking. A short turn sounds fine; a ten-second turn ends seconds late.

Latency you can only see on long turns is almost always a rate error, not a delay. Look for something proportional to the number of units processed.

PlaybackScheduler (one per leg) places each buffer with source.start(when) at an absolute time on AudioContext.currentTime, so the join is sample-accurate and happens entirely inside the audio thread. The only latency it adds is PLAYBACK_LEAD_S (60 ms), a deliberate fixed jitter buffer — chunks cross a WebSocket relay, and scheduling at exactly currentTime means any chunk arriving a hair late is scheduled in the past and clicks.

Three consequences worth knowing before touching it:

• It's a timeline, not a queue — so cancelling means stop()ing sources. Chunks are scheduled ahead of the clock, so on mute/hold/stop there is no array to empty: stop() on a not-yet-started source is what cancels it. Emptying a queue instead would let the lead plus the whole scheduled tail keep reaching the customer after the agent hit mute.
• A chunk that fails to start must not advance the timeline. Advancing it for a buffer that never played would schedule every later chunk into a phantom future, turning one failed buffer into silence for the rest of the run.
• The duck spans a run, not a chunk. A run is the contiguous scheduled span; it ends when playback catches up (the speaker paused), plus a grace that outlasts a provider's mid-sentence pause. Ducking per 40 ms chunk would pump the original voice up and down underneath the translation.

Nothing upstream smooths these chunks. The orchestrator's PhraseAwareBuffer is inserted only after a stage whose service_type == "tts" (translation_builder.py), and a single-stage S2S translation flow — OpenAI Realtime Translate, Google S2ST, i.e. both agents in production use — has no TTS stage at all. Don't assume the client is receiving phrase-sized audio.

Tests: playback-scheduler.test.mjs. The drift-detecting ones run a 250-chunk (10 s) run against a frozen fake clock and assert the scheduled span equals exactly 250 × 40 ms; a clock that only moves when the test moves it is what turns an accumulating per-chunk gap into arithmetic instead of bad audio.

Transcripts: two channels, two Salesforce messages

Both languages come from the translation pipeline, and only from it — Contact Lens and any other platform transcription source are deliberately not merged in. Each translation flow emits two text channels:

Channel	TextEvent.kind	What it is
source	KIND_TRANSCRIPT (0)	what the speaker actually said, in their language
translation	KIND_RESPONSE (1)	what the listener hears

Speaker comes from the stream, not the kind. On the outgoing stream both the ASR and the translation belong to the agent. Deriving the speaker from kind mislabels every agent utterance as the customer.

Eight properties of this path, each of which was a defect:

• TranslationNormalizer is inserted LAST in the flow, and it is load-bearing. ConversationAPIPublisher has no bare-TextFrame branch, so without the normalizer neither channel reaches it and a translated call produces no conversation_messages rows at all — silently, because the publisher forwards frames it doesn't recognize. It is one normalizer for the whole flow, not the per-stage registry lookup universal_builder does: the two channels it tells apart are a property of the flow, and a cascaded flow begins with deepgram/stt, whose registered DeepgramNormalizer has no translated-text branch and would drop that channel. Last position matters twice — after every service, so a cascaded flow's TTS still receives the raw TextFrame it synthesizes from, and after PhraseAwareBuffer for the same reason. The normalizer also owns spacing: it marks every fragment accumulation_mode="append" and inserts the separator per the flow's text_granularity (token for OpenAI's sub-word deltas, whose own whitespace is authoritative; phrase for Google's whitespace-free phrases; auto infers, and can space a mid-word split before it learns). Spacing here, boundaries in the publisher — putting either in the other place produced doubled spaces or truncated lines.
• Both channels are fragment streams with no "done" event. Providers stream phrases (Google) or sub-word deltas (OpenAI), and there is no LLMFullResponseEndFrame on a translation flow. Accumulation and the turn boundary (an idle timer) live in ConversationAPIPublisher, which is the only component that knows both the full text and when the utterance ended.
• The wire frame is TranslationTranscriptFrame, a DataFrame — deliberately not a TextFrame. It carries the accumulated snapshot plus channel / is_final / turn_id. The gRPC output transport forwards this and drops bare TranscriptionFrame / InterimTranscriptionFrame, so a future processor emitting text on a translation flow cannot leak fragments to Salesforce. Note TranscriptionFrame is a TextFrame subclass — an unguarded isinstance(frame, TextFrame) branch sent source ASR out as KIND_RESPONSE, i.e. as the translation.
• text_out_enabled must be on for a translation session (has_text or uses_s2s_translation), while text_in_enabled stays keyed on has_text alone — a translation flow emits text but doesn't accept it.
• Closing an utterance finalizes both channels, i.e. two wire events. The client pairs them per utterance and emits on the first final; the second is the companion channel arriving for a pair that already went out, and forwarding it would add a duplicate Salesforce line. _handleText in the provider holds that per-stream accumulator, and stamps the entry with when the speaker started — a rate-limited batch can flush seconds later.
• TextEvent.kind arrives at the browser as the enum NAME, not its number, and the client compares numerically. The relay loads voice.agent.proto with enums: String (server.mjs), so msg.text.kind decodes to "KIND_RESPONSE" and it relays what it decoded; _handleText's kind !== TEXT_KIND_TRANSCRIPT && kind !== TEXT_KIND_RESPONSE guard therefore matched every transcript and swallowed it down the system-message branch. protoEnumValue in OrchestratorStream.js normalizes both forms at the one point they enter the client (text and control), returning null for an unrecognized name so a future enum member can never be mistaken for member 0 — KIND_TRANSCRIPT/TYPE_DISCONNECT, the two values worst to invent. Normalize here rather than changing the loader: the relay is a dumb pipe whose options are free to change, and a client that accepts both forms can't be broken by them again. The symptom is the reason this took a whole debugging session: every server-side signal was green. The orchestrator produced and finalized both channels, both gRPC streams opened and closed cleanly, conversation_messages filled up, the native Transcript component showed both languages — and Salesforce received nothing, with the only trace an info-level system text: line per utterance. An unresolved kind now warns that it is dropping a transcript. Regression: proto-enum.test.mjs, which asserts against the wire form — a test written with numeric fixtures passes on the broken code.
• A TextEvent that omits kind is not unlabelled — it says "the caller said this". proto3's default for an enum is 0, and KIND_TRANSCRIPT is 0, so the transport's catch-all else (whatever the serializer stringified that isn't audio, an end/cancel, a transcript or a chat reply) built a TextEvent that the connector faithfully forwarded to Salesforce as the caller's first utterance. What fell through was pipecat's own telemetry: PipelineWorker installs an RTVIObserver by default (enable_rtvi: bool = True), its MetricsMessage is pushed as an OutputTransportMessageUrgentFrame, BaseOutputTransport routes that to the transport's send_message(), and every translated call opened in Salesforce with {"label": "rtvi-ai", "type": "metrics", "data": {}} — twice, one per direction, since a translated call runs one flow and therefore one observer per direction. Upstream has the filter for exactly this (FrameSerializer.should_ignore_frame(), which twilio/plivo/genesys/protobuf/ vonage/exotel all call in their OutputTransportMessage* branch) but ours was unusable: GrpcFrameSerializer never called super().__init__(), so self._params was never set and that method raised AttributeError rather than filtering. Three layers now close it, each independently justified: the serializer drops RTVI frames (returning None, which _write_frame treats as "nothing to send"); the transport's catch-all sets kind=KIND_SYSTEM explicitly, so the next unlabelled frame can't be read as speech; and the client's isTelemetryEnvelope guard in _handleText stays, because the connector talks to whatever orchestrator is deployed and because /api/createTranscription feeds both sinks (server.mjs calls recordTranscript beside the SCRT2 write), making the client the only single point that also keeps the envelope out of the Contact-Lens S3 document. The guard keys on the RTVI label, not on "looks like JSON" — a caller can read a JSON payload aloud. Regressions: telemetry-envelope.test.mjs (which also asserts, by reading the provider's source, that the guard sits above the utterance accumulator and returns) and three tests in test_translation_transcripts.py, one of which pins KIND_TRANSCRIPT == 0 as the reason any branch omitting kind is a bug.
• A translation session must put TranslationTranscriptFrame on the wire and nothing else — there were TWO producers, and the second one made every fragment its own Salesforce message. _write_frame in grpc.py routes text in two branches, and the text-chat one (_write_text_frame, which hardcodes kind=KIND_RESPONSE, is_partial=False) also caught the translated fragments: the publisher emits its accumulated frame and re-pushes the raw fragment downstream — it must, the pipeline continues past it — and NormalizedTTSTextFrame inherits through TTSTextFrame → AggregatedTextFrame → TextFrame. So the wire carried the accumulation as a correctly turn-bounded partial and the fragment as a final the client honours. The asymmetry is the tell: the cumulative source text sat beside a non-overlapping delta translation (me amo o me / "I love myself", then me amo o me hombre Eric / "or I'm a man,"), and one joined string cannot produce that — two producers can. The gate is the session-level param text_out_translation_only (uses_s2s_translation and not has_text), not a frame-type exclusion, because a Google ADK text_chat session delivers its genuine reply as that same class. Fixing this before the message split mattered: splitting first would only have turned each duplicated fragment into two messages.

Salesforce is the constraint on presentation: SCRT2's messages body has exactly one text field (content), so an utterance in two languages cannot be one message — it is two, posted in order by createTranscription (scrtConnector.mjs):

+18049091812 · 4:27:08 PM   Hola, me llamo Eric.
+18049091812 · 4:27:09 PM   [Translation] Hello, my name is Eric.

splitTranscriptMessages decides what those messages are, and SCV_TRANSLATION_PREFIX sets the label (default [Translation]; an empty value means unlabelled, which is not the same as unset). SCV_TRANSCRIPT_TRANSLATION_ONLY=true drops the original entirely; the default keeps it, because a Salesforce transcript is a record of what was actually said and the translation is our annotation of it. A missing or identical translation yields one part — never a label with nothing after it, and no caller may assume two.

This replaced a single message with the two languages newline-joined. Omni renders a message as one chat bubble, so the translation was crammed under the original inside the same bubble with no speaker line and no timestamp of its own, and a reader scanning one language had to read both. Separate messages get the platform's own rendering for free — but only if three things hold, each of which is a way to lose half the transcript silently:

• Distinct messageIds. messageId is the platform's dedup key, so the two halves sharing one means Salesforce accepts the first and swallows the second. Only the translation is suffixed (-t), so a single-message utterance keeps exactly the id the client sent and nothing downstream correlating on it changes.
• Partitioned time windows. Both messages describe the same stretch of speech, so identical startTimes give Omni no basis to order them and it renders the translation above the original as often as below. partitionTranscriptWindow slices the utterance's window by each part's share of the text, laid end to end. The slices stay inside that window: letting the translation run past would arguably be more honest — the translated audio really does play afterwards — but the window's end is estimated from text length, so an overshoot could push the translation's startTime past the next utterance's and reorder the transcript. A degenerate window (an explicit endTime == startTime, or an estimate leaving under a millisecond per part) is widened just enough that every part gets its own instant.
• Sequential POSTs, not Promise.all. Two concurrent POSTs can be applied in either order, and the timestamps only order what Salesforce has already accepted.

The fan-out is server-side, not in the browser, because the same /api/createTranscription request also feeds the S3 analysis document, which counts turns — two client POSTs for one utterance would record it twice there. And that document deliberately does not split: a Transcript[] entry is a segment of the call, one utterance is one segment, and splitting would double every count a consumer derives from it (turn counts, talk time, non-talk time). Consumers that want the languages apart read TrueServeMetadata.Translations, which keeps Original and Translated as their own fields. The two sinks diverge on purpose.

The transcript gate: one producer per call, decided per contact

"Only from the pipeline" needs enforcement outside our process, because the platform transcribes the same audio in parallel and its own consumer writes to SCRT2. On Connect that's Contact Lens → Kinesis → amazonMultiorg-MultiorgContactLensConsumer, provisioned by Salesforce's CloudFormation stack. Left alone, a translated call gets every utterance twice — and the platform's copy of it is often garbage, since Contact Lens transcribes a bilingual call under one LanguageCode and the half it guessed wrong comes back phonetic ("Hola, me llamo Erik" → "Ola Leamo Erik").

The gate is a contact attribute the connector writes, read per segment by scv-transcript-gate-lambda:

Step	Where
TrueServeTranslationActive=true before the first gRPC stream opens	markTranslationGate(true, call) in src/connector/index.js
…=false on stop (unless keepGate)	stopTranslation
the platform write	TelephonyProvider.setTranslationActive → POST /api/connect/translation-active → UpdateContactAttributes
the read	the gate lambda, per Kinesis batch, GetContactAttributes

Five things make this work, and each is a place it could have failed:

• The mark goes up BEFORE the first stream opens. Platform segments lag the audio by seconds, so an attribute written at translation start is already visible when the first one lands. Writing it after would leak the opening utterances.
• The scope is translation-active only. Nothing is suppressed for a call that isn't being translated, or for the stretch of a translated call before the agent switched it on — Contact Lens behaves exactly as it did before the gate existed.
• It is best-effort by contract, and fails OPEN. A failed write, an unreadable attribute, or a Connect API error all mean "platform transcripts also appear" — visible and recoverable. Failing closed would mean a call with no transcript at all. setTranslationActive swallows its own failures for exactly this reason; translation must never fail because the gate did.
• Clearing is a write of 'false', not a delete. UpdateContactAttributes has no per-key delete, and the gate keys on the literal string 'true'.
• TelephonyProvider.setTranslationActive is a no-op on the base class, so a provider without a gate (Genesys today, demo) degrades to today's behaviour rather than throwing. Genesys's equivalent is participant attributes plus a custom transcript connector — not built yet.

Contact-Lens-shaped analysis JSON: the second sink

Suppressing the platform's segments leaves a hole. Contact Lens also writes a post-call analysis document to S3, and everything downstream that consumes it (Bedrock summarization, conversation-intelligence jobs, an org's own S3 triggers) goes empty for a translated call. So the connector publishes its own, in the same shape: server/contact-lens-publisher.mjs.

This is a second sink, not a replacement: SCRT2 createTranscription still writes to Salesforce live, per utterance. The two differ in cadence because the artifacts do — a Contact Lens analysis file describes a whole conversation, so there is nothing to publish until the call is over.

POST /api/call-mapping/register  ──→ noteCallStarted()        anchor
POST /api/createTranscription    ──→ recordTranscript()       accumulate
                                  └→ ScrtConnector            (live, per utterance)
POST /api/connect/transcript-analysis ──→ publishAnalysis()   flush → S3

Three orderings are load-bearing, each for a different reason:

• recordTranscript runs before the SCRT2 write, so an SCRT2 outage doesn't also cost the S3 document.
• noteCallStarted runs during /api/call-mapping/register, independently of TrueServe conversation startup. It anchors on call start because Contact Lens offsets are milliseconds from there; anchoring on the first translated utterance would silently shift every offset by however long the agent talked first.
• The flush runs AFTER stopTranslation, because TranscriptionBridge.stop() awaits a per-speaker _flushQueue and that flush is where the call's last utterance reaches the server. Publishing first omits it.

Accumulation is inherently translation-only — the only feed is /api/createTranscription, which only TranscriptionBridge posts to, and the bridge only runs while translating. There is no second gate to keep in step with the contact attribute.

Two deliberate departures from real Connect output, both refusals to invent data:

• No Sentiment or LoudnessScore on turns, and no Sentiment / TalkSpeed / Interruptions in ConversationCharacteristics. We don't analyse audio. A consumer reading an absent field as unknown is correct; one reading a fabricated "NEUTRAL" as a measurement is not.
• A TrueServeMetadata block carrying the per-direction language pair, the original/translated text per turn joined by transcript Id, and OffsetAnchor. The schema has exactly one top-level LanguageCode — the very limitation that garbles Contact Lens's own bilingual output — so the pair has nowhere in-schema to live. Namespaced and additive, so a strict reader ignores it.

Where it lands is a deliberate near-miss of Connect's own layout. The bucket is normally the instance's own storage bucket, so an operator finds these where they already look — but the key is Translations/Voice/{YYYY}/{MM}/{DD}/{contactId}_translated_{ISO8601Z}.json, not Analysis/Voice/…_analysis_…. Both the prefix and the filename infix differ, because a consumer may glob on either, and one of ours read as real Contact Lens output would silently double-count the call. What does match Connect is the date/timestamp convention — UTC date path, second-precision with no milliseconds — so anything written to walk that layout walks this one too.

Three regions, three variables — and no single value satisfies all of them. The process's region, the Connect instance's region, and the bucket's region are independent facts, and a real setup has them differ (Vanguard: process us-west-1, instance us-east-1, bucket us-west-2). So the server-side Connect client reads CONNECT_REGION/AWS_CONNECT_REGION and the publisher reads SCV_TRANSCRIPT_ANALYSIS_REGION, each falling back to AWS_REGION — sharing one AWS_REGION between them cannot be made correct by changing its value. Both failures are quiet in the way that matters: UpdateContactAttributes against the wrong region raises ResourceNotFoundException, and the attribute write is best-effort by contract, so the only symptom is the duplicate transcripts the gate was supposed to stop; PutObject against the wrong region fails PermanentRedirect and costs the whole document. The publisher's startup line prints its resolved region for exactly this reason.

Smaller invariants worth not re-deriving: a call is marked published before the upload so a racing retry can't write a second file (which reads downstream as a second call), and re-armed on a real failure; a turn with no endTime — the normal case, since TranscriptionBridge sends none — spans to the next turn's begin rather than emitting a zero-length turn; and SCV_TRANSCRIPT_ANALYSIS_BUCKET unset disables publication entirely, which is the shipped default.

Genesys will need its own equivalent (a custom transcript connector rather than an S3 document) — not built yet.

SAML Connect instances: the login popup needs loginUrl

A SAML-identity Connect instance (e.g. dcc-dev-main-connect) has no local login — its /login page always renders "Session expired", and an unauthenticated CCP iframe serves frame-ancestors 'none' pages, so the console shows CSP violations + /auth/refresh 401 loops that look like an approved-origins problem even when the origin is approved. Auth must start at the IdP. The connector threads an optional connect.loginUrl through to initCCP — set the Call Center XML key reqConnectLoginUrl, or the connector env SCV_CONNECT_LOGIN_URL (served via GET /api/translator-config, fetched before CCP init; XML wins), to the IdP-initiated SSO URL (Authentik: the app's …/application/saml/amazon-connect/sso/binding/init/ launch URL) and Streams pops that on auth failure instead of the dead instance login. No IdP session → the popup shows the IdP login form; existing session → the SAML POST completes silently and the popup auto-closes. The Authentik provider's default_relay_state must point at the Connect federation endpoint (https://<region>.console.aws.amazon.com/connect/federate/<instance-id>).

Agent selection

resolveTranslationAgentId() in src/connector/index.js, first non-empty wins:

#	Source	Set by
1	TRANSLATION_CONFIG.translationAgentId	the translationSettings panel (or its App Builder property)
2	Call Center XML reqTranslationAgentId	Salesforce admin, org-wide
3	SCV_TRANSLATION_AGENT_ID via GET /api/translator-config	connector server env
4	—	nothing set → error event (context: 'agent_selection') and start throws

There is no 'default-translator' fallback; an unconfigured org fails fast with an actionable message. Step 3's route is deliberately unauthenticated and secret-free — a published pipeline id is not sensitive, and the WS relay still authenticates every audio stream at upgrade (SCV_SERVICE_TOKEN / browser session ticket / origin allowlist). Because app.use('/api', requireScvServiceTokenMiddleware) guards the whole /api prefix, /api/translator-config must stay listed in PUBLIC_API_PATHS — the softphone fetches it with credentials: 'omit' before it has a session, so dropping it from that allowlist makes every agent-selection attempt 401 and translation refuses to start.

Salesforce → connector: the FQDN and the bearer token

Salesforce reaches this service two ways, and both use the same public origin: the connector page SCV loads in an iframe (connectorUrl on ConversationVendorInformation), and Apex partner callouts through the TrueServeCredential Named Credential.

The FQDN needs its own nginx server block. Without one, scv-translator.<domain> falls through to the first listen 443 block — ai-manager, the implicit default_server — which answers every request with a 307 to its SSO /login. The wildcard cert covers the name, so TLS succeeds and the misroute is invisible until Apex reports HTML where it expected JSON. The block is in trueserve-vanguard.conf.template; it splits /api/, /socket.io/ and /ws/ to Express (:3031) and everything else to webpack-dev-server (:8101), because in dev Express only mounts the static handlers under NODE_ENV=production. Do not route /api through :8101 — its devServer proxy targets SERVER_URL, which the launcher doesn't export, so it defaults to :3030 (webex-connector) and every call 502s.

Every /api/sfdc/* callout must present a bearer token. app.use('/api', requireScvServiceTokenMiddleware) guards the whole prefix, and only the handful of paths in PUBLIC_API_PATHS are exempt — so a no-auth Named Credential gets 401 {"error":"unauthorized"} on all 11 partner routes while DNS, TLS and routing look perfectly healthy. The token rides on an AuthHeader formula parameter of the TrueServeSvcToken External Credential ({!'Bearer ' & $Credential.TrueServeSvcToken.ServiceToken}), so all 11 callouts inherit it from one place and no secret enters Apex source. $Credential references stay unprefixed inside a namespaced package — the installed Five9 BYOT package proves the same shape. Three org-side facts that each produce the same misleading 401:

• the ServiceToken auth parameter can't ship in metadata (the Custom protocol rejects a declared AuthParameter), so it is set per org — POST to /services/data/vXX.0/named-credentials/credential creates it, PUT only updates an existing one, and credentials is an object map, not an array;
• the running user needs External Credential Principal Access, granted by an org-local permission set — committed as TrueServeSCVGrant in sfdc/trueserve-scv/org-local/, deployed and assigned separately from the package — including the admin who creates the contact center (Setup → Contact Centers → New fires connect() as them);
• generateAuthorizationHeader stays false — the header comes from the credential, not the platform's built-in generation.

That grant must not live in the package at all — this is the correction to earlier guidance. externalCredentialPrincipal is unprefixed in package source but resolved in the subscriber org at install time, before the credential the same install creates exists, so a clean install can never satisfy it and fails with PermissionSet(TrueServeSCVService) Unable to find ExternalCredentialPrincipal: TrueServeSvcToken-ServiceAccount. Version 0.5.0.1 only appeared to install fine because an unnamespaced TrueServeSvcToken left behind by earlier unmanaged testing satisfied the reference; once the beta uninstall removed that, both the version build (its scratch-org validation install) and the real install broke. --skip-validation gets a version cut and then fails identically at install — it is not the fix. The grant is org-local post-deploy work, which costs nothing since the ServiceToken secret can't ship in metadata either and a beta swap drops the grant regardless. 0.6.0.2 is the first version cut without it; it installs clean and an Apex callout through the credential returns 200.

Two consequences of that same unprefixed-resolution rule remain: an unnamespaced same-named credential wins the binding (callouts fail with "we couldn't access the credential(s)" despite everything looking configured, and deleting the impostor drops the grant rather than re-pointing it), so the org-local set should name trueservedev__TrueServeSvcToken-ServiceAccount with the namespace. And every version cut so far is an unpromoted beta: betas can't be upgraded in place, so installing a new one means uninstalling the old — which takes the credential secret and every permission-set assignment with it, and is itself blocked until you delete those assignments and remove the packaged LWC from any Lightning page hosting it. All written up in sfdc/trueserve-scv/README.md.

Setup steps and a copy-paste Apex verification snippet: sfdc/trueserve-scv/README.md.

A beta replacement also changes the Named Credential's record ID. The contact center retains that ID in CallCenter.reqHvcc.NamedCredential; restoring the old XML can leave a deleted reference even after secrets and grants are working. The Setup page then reports a deleted named credential and shows a blank credential field. Validate the replacement credential with an authenticated callout, then repair only that metadata item and verify the Setup page. See the credential-binding repair.

SCRT2 lives on its own domain, so the SSRF allowlist must name it explicitly

ScvBridge.init() forwards SCV's callCenterConfig.scrtUrl to POST /api/configureTenantInfo, which stores it in tenantInfo — the base URL every later transcript and voiceCall call is built from. assertAllowedScrtUrl (server/scrtConnector.mjs) gates that value against ALLOWED_SCRT_HOST_RE before we sign a JWT Bearer for it, because a caller-supplied base URL would otherwise exfiltrate the signed token to an internal or attacker-controlled host.

The trap is that SCRT2 is not under salesforce.com. Its hosts are <mydomain>.my.salesforce-scrt.com (sandbox: <org>--<sandbox>.sandbox.my.salesforce-scrt.com) — salesforce-scrt.com is a separate registrable domain, so an allowlist listing only my.salesforce.com / my.salesforce-setup.com / force.com / salesforce.com can never match it, no matter how the org is configured. The result was that every real scrtUrl Salesforce sends was rejected: configureTenantInfo answered 400 scrtBaseUrl host is not on the Salesforce allowlist, tenantInfo stayed empty, and getScrtUrl then produced the relative base URL /telephony/v1, so every createTranscription / createVoiceCall / updateVoiceCall POST died with ECONNREFUSED against the connector's own host instead of reaching Salesforce. Calls and translation work fine throughout — only the Salesforce write-back is dead — so the symptom is "transcripts never appear in Omni", one configureTenantInfo failed console warning at init, and nothing else.

Add legitimate Salesforce domains to the alternation; don't relax the pattern. The leading \. and trailing $ are what make it an exact-suffix match, so my.salesforce-scrt.com.evil.com, my-salesforce-scrt.com, bare salesforce-scrt.com, localhost and 169.254.169.254 all stay rejected, and the https:-only check is separate. Verify domain ownership before adding one — openssl s_client on *.my.salesforce-scrt.com presents a cert with O = "Salesforce, Inc.".

…and the signing key it needs is the other half of the same dead path

Fixing the allowlist alone still left write-back dead, because getToken() read its RS256 key from a hard-coded <cwd>/src/server/private.key — inherited verbatim from the Salesforce demo connector, which really does keep server code under src/server/. This repo doesn't: src/ holds only browser code (common, connector, login, telephony, translation) and server code lives in server/, so that path never existed and no such key was ever tracked or gitignored.

getToken() is called inside the Authorization header object literal at all five SCRT2 call sites (createVoiceCall, createTranscription, sendRealtimeConversationEvents, updateVoiceCall, executeOmniFlow), so the ENOENT threw before any HTTP request was built — the same silent shape as the allowlist bug, and the same symptom. The two are independent: either one alone stops transcripts from reaching Omni.

loadScrtPrivateKey (server/scrt-signing-key.mjs) resolves in order, first hit wins, caching the result:

1. SCRT_PRIVATE_KEY — inline PEM, \n escapes tolerated (same handling as SF_PRIVATE_KEY in server/ottAppLib/sfdc-auth.mjs)
2. SCRT_PRIVATE_KEY_FILE — explicit path (the repo's X / X_FILE convention)
3. server/private.key — conventional location beside the module
4. src/server/private.key — the upstream demo path, for a verbatim port

With none of them set it throws a message naming every option instead of an ENOENT on one guessed path, because that error is the only signal an operator gets. The key's public half must be the certificate the Call Center was created with, or Salesforce rejects the Bearer:

openssl genrsa -out server/private.key 2048
openssl req -new -x509 -key server/private.key -out server/cert.pem -days 730

Candidate 3 sits inside a tracked directory, so both of those filenames are gitignored explicitly — check that before renaming either the file or the fallback.

The public half is loaded by a different module, and its fallback guarantees a mismatch

The two halves of this pair have separate owners, which is how they drifted:

Half	Owner	Reaches Salesforce via
private (signs the Bearer)	scrt-signing-key.mjs	—
public (verifies it)	telephony-adapters/cert-loader.mjs	GET /api/sfdc/key (Apex getPublicKey) + the generated Call Center XML

Nothing tied them together, and cert-loader's lookup didn't even include server/cert.pem — it fell through to certs/wildcard_vanguard_fullchain.pem, the nginx wildcard TLS certificate, whose private key is ACM-managed and is emphatically not what we sign with. So following the recipe above produced a valid-looking setup that could never authenticate: /api/sfdc/key returned a real, unexpired, Salesforce-trusted certificate (CN=*.vanguard.dev.trueserve.deloitte.com), the Call Center XML embedded it, and every SCRT2 call was then signed by an unrelated key.

server/cert.pem now sits above the wildcard fallback, and every resolved certificate passes through warnIfCertKeyMismatch, which compares DER SPKI bytes (so PEM formatting is irrelevant) and logs:

[scrt-key] MISMATCH: the certificate from <path> is not the public half of the
SCRT2 signing key, so Salesforce will reject every transcript / voiceCall call.

Worth logging loudly because Salesforce's own rejection is a 401 that says nothing about key material. certMatchesKey returns null — "unknown", never a false confirmation — when either side won't parse. Keep the wildcard fallback last: it exists so the Call Center XML has something to embed, not because it can work.

…and the org must actually be holding that certificate — it gets it by PULL, once

A locally-correct pair still 401s if the org's contact center holds no certificate, or holds an older one. CallCenter.reqTelephonyIntegrationCertificate is what SCRT2 verifies the Bearer against, and in a broken org it reads:

<item><name>reqTelephonyIntegrationCertificate</name><value xsi:nil="true"/></item>

The certificate reaches Salesforce by pull, not by push. Apex TrueServeIntegrationImpl.getPublicKey(contactCenterInfo) calls GET /api/sfdc/key?contactCenterId=… on the connector, and only at contact-centre setup / refresh — so regenerating server/cert.pem does not propagate, and a contact centre created before the connector could serve a key keeps a nil value forever. The tell is the sibling field: reqTelephonyIntegrationKeyPairExpDate dates the last pull, because getPublicKey falls back to Date.today().addYears(1) when the connector returns no expiration. An expiry a year out from some date before your key existed means the pull predates it.

The failure mode is the most misleading on this whole chain: SCRT2 rejects at authentication, before any call lookup, so the 401 is identical to a missing bearer token, a wrong signing key, or an unallowlisted host. Nothing in the response mentions key material.

CallCenter is metadata, not queryable by SOQL. Read and write it with the CLI:

sf project retrieve start --metadata "CallCenter:trueServeContactCenter" -o <alias>
# …edit reqTelephonyIntegrationCertificate + reqTelephonyIntegrationKeyPairExpDate…
sf project deploy start --metadata "CallCenter:trueServeContactCenter" -o <alias>

Retrieve → deploy is not idempotent: retrieve omits three reqGeneralInfo items that deploy requires (reqDescription, reqVersion, reqProgId), and the error names only the ones missing from the current attempt, so a naive round trip fails two or three times in a row, each time looking like a different problem. Back up the retrieved file before editing.

Verify by probing SCRT2 directly with the real signing key: a 401 means the org isn't holding your certificate; a 400 with object has missing required properties means the Bearer was accepted and the call reached body validation — i.e. authentication and the call lookup both passed, which is the outcome you want from a deliberately bogus vendorCallKey.

…and then a guard that rejected 100% of legitimate traffic

With the key pair fixed, every SCRT2 write still failed — now locally, with Unknown or inactive voice call identifier, never reaching Salesforce at all.

activeCallIdentifiers is an authorization guard: it stops a browser from POSTing transcripts against an arbitrary vendorCallKey. It is ours, not upstream (added in 433e30eb "Security fixes and minor tweaks + docs"; absent from .backup/byo-demo-connector/src/server/scrtConnector.mjs). But it was populated by exactly one function — rememberCallIdentifiers(), called only from ScrtConnector.createVoiceCall, the demo connector's flow, where the connector itself creates the VoiceCall record over SCRT2.

TrueServe never takes that path. Real telephony starts the call, ScvBridge publishes CALL_STARTED, and the SCV platform creates the VoiceCall record; /api/createVoiceCall has zero callers anywhere in src/. So the set stayed empty for the entire lifetime of the process and resolveActiveCallIdentifier threw on every transcript, every voiceCall update and every realtime event — a guard rejecting all legitimate traffic and all illegitimate traffic alike. A security control added in isolation, on a path that was never the path in use.

The fix registers the call where its key first becomes known: registerPlatformCall(vendorCallKey) from server/call-mapping.mjs's POST /api/call-mapping/register — the one request the client already makes for every call that starts (ScvBridge._registerCallMapping, from both _onCallStarted and dial, inbound and outbound). The guard keeps its teeth: that route is behind the browser-session check, so an id must still have been announced by a real CALL_STARTED.

Two ordering details are load-bearing:

• registerPlatformCall runs independently of conversation startup. Registration returns {success:true} after authorizing SCRT2 writes and anchoring offsets. The obsolete POST to conversation-api's nonexistent /api/call-mapping route caused a misleading 502 on every configured call and has been removed. The audio connector creates the conversation using the vendor call key; Salesforce resolves it through /conversations/by-external-id/{id}.
• Registering also points the module-level vendorCallKey/voiceCallId fallbacks at the live call, so a caller that omits the id resolves it instead of falling back to ''.

ScvBridge._onCallConnected's unknown-callId branch was a narrower instance of the same hole: it emits callConnected to the translation layer but never registered the mapping, so a call SCV's bookkeeping missed would start translating and then have every transcript rejected. It now calls _registerCallMapping as well.

HUMAN_AGENT transcripts threw a TypeError, so only half the conversation reached Omni

participantId for an agent-side transcript was onlineUserIds[params.metaData].userId. That map is filled only by the socket.io presence handler (server/server.mjs:134). The demo connector emitted that event from toggleAgentPresence() (.backup/byo-demo-connector/src/main/vendor-sdk.js:225); TrueServe's connector has no socket.io client at all. So the map was permanently empty and the lookup threw Cannot read properties of undefined (reading 'userId') — a TypeError, from inside the transcript builder, on every single agent utterance. END_USER takes a different branch and was unaffected, which is why the signature is half a transcript rather than none.

The id's real source is where the demo read it too: SCV hands the connector callCenterConfig.userId at init() (.backup/byo-demo-connector/src/main/vendor-sdk.js:503). ScvBridge.init() already POSTs scrtBaseUrl/orgId/callCenterName from that same object to /api/configureTenantInfo, so agentUserId rides along — resolved once per session instead of per utterance — and resolveAgentParticipantId uses it when no metaData is supplied. An explicit metaData still wins, so the multi-agent / supervisor paths are unchanged, and agentUserId is optional on configureTenantInfo so an un-rebuilt client bundle still configures its tenant. With neither available it throws a message naming both ways to supply one.

…and a restart loses the tenant, because SCV only tells you once

tenantInfo is a module-level object in scrtConnector.mjs — process memory — and it has exactly one populator: ScvBridge.init() POSTing /api/configureTenantInfo. SCV calls the connector's init() once per browser session. So any server restart after the agent signed in leaves the process permanently amnesiac for the rest of that session: a deploy, a pod eviction, an HPA scale-out, or nodemon in dev. The agent notices nothing — they are already signed in, calls connect, audio flows, translation works — and every Salesforce write fails until they reload the console.

Both halves of tenantInfo go missing at once and fail at different points, which is why one cause reads as two unrelated bugs:

Empty field	Failure	Reads as
scrtBaseUrl	getScrtUrl returned the relative /telephony/v1, so axios resolved it against nothing → connect ECONNREFUSED ::1:80	a network or infrastructure fault
agentUserId	resolveAgentParticipantId throws No Salesforce User Id for the agent…	the participantId defect above, still unfixed

That ECONNREFUSED ::1:80 is the same wire symptom the host-allowlist defect produces, for the same proximate reason (an empty scrtBaseUrl) from a completely different cause — so the signature alone can't tell you which one you have. Distinguish them by grepping the log for TenantInfo configured: present means the POST ran and the value was rejected or lost some other way; absent means it never ran in this process, i.e. a restart.

Three layers close it:

• getScrtUrl refuses to build a relative URL and throws a message naming the cause. Silently degrading to a relative base URL is what turned "this process was never told where Salesforce is" into a network error, on a chain where every failure is already invisible.
• ScvBridge re-sends the tenant identity on /api/call-mapping/register — the one request the client already makes for every call that starts, carrying the same values configureTenantInfo does. Both paths build it from one _tenantInfoPayload() helper reading the stored callCenterConfig, so they cannot disagree about a value. No new auth surface: BROWSER_API_PATHS in server.mjs already contains both routes, behind the same browser-session check.
• The server applies it only when it has nothing (ScrtConnector.isTenantConfigured()). A configured tenant is authoritative and must never be overwritten from per-call data, or a stale tab — or a second org's console — could repoint an already-working connector. configureTenantInfo still applies the full assertAllowedScrtUrl guard and field validation, so the recovery path is not a way around either; a rejected payload leaves the process unconfigured rather than half-configured, and the re-assert failure is logged non-fatally because the writes that follow fail loudly on their own.

All six in-repo defects are covered by npm test (server/scrtConnector.test.mjs); the nil org certificate above is org state, so nothing in this repo can assert it — probe SCRT2 instead. The key tests run each probe in a child process — the loader caches in module state and reads process-wide env, so in-process tests would contaminate each other. They assert on the next guard in line, which proves execution got past the throw under test without doing real I/O: Unknown or inactive voice call identifier for the key tests (which never register a call), a DNS failure against an allowlisted-but-unresolvable SCRT2 host for the call-identifier and participantId tests. That trick has a caveat this chain demonstrates: the key tests originally documented the identifier guard as merely "the next check in line" when it was itself defect 4 — so establish that the guard you land on is one legitimate traffic actually clears. The identifier tests drive the real /api/call-mapping/register handler rather than calling registerPlatformCall directly, because the wiring was the defect, and one test asserts an unregistered key is still rejected so the guard can't be silently defanged. The cert-ordering test asserts PAIRED=true and the loaded cert from …server/cert.pem log line, so a reordering that happens to still pair can't pass silently; it self-skips when real deployment material is already on disk. The restart probes need their own child-process variant that deliberately never calls configureTenantInfo — that absence is the condition under test, and the shared probe helper always calls it — and the never-overwritten case asserts on the wire (the DNS failure names the host actually dialled) rather than by reading state back, because both hosts in that test are legal and a state read cannot distinguish "kept" from "re-set by the assertion itself".

Reading the failure, once it fails

Every write-back defect above is off the call path, so the Express log line is the only signal — and until now it discarded the one part that says what to fix. SCRT2's complaint lives in the response body: status=401 with no body is the key / certificate / allowlist family, while a body of object has missing required properties (["endTime","participantId"]) is a schema problem on a request that authenticated fine. Both of the obvious renderings lose it. ${err} prints only AxiosError: Request failed with status code 401, and JSON.stringify(err) goes through axios's own toJSON(), which omits response entirely — so no amount of care at the call site recovers the body. Every SCRT2 failure log now goes through describeScrtError (in scrtConnector.mjs), which emits status=… body=… message=… with status and body as separate tokens so grepping for one can't match the other. Its tests build a real AxiosError rather than a plain Error with a response bolted on — a hand-rolled stand-in doesn't reproduce toJSON(), so it would pass while production still logged nothing useful.

After changing any of this, verify the whole chain end to end — the two routes Salesforce actually reads must agree with the key we sign with:

curl -s -H "Authorization: Bearer $SCV_SERVICE_TOKEN" \
  'http://localhost:3031/api/sfdc/key?contactCenterId=<id>'   # Apex getPublicKey
curl -s -H "Authorization: Bearer $SCV_SERVICE_TOKEN" \
  http://localhost:3031/api/sfdc/call-center-xml              # embedded cert

Authenticating the WS relay by hand

Reproducing a stream outside the browser needs both halves of the browser-session pair, which is easy to get wrong:

1. GET /api/scv-session with a same-origin Origin header (blank SCV_ALLOWED_ORIGINS means same-origin only, so a mismatched origin is 403 origin not allowed).
2. Open ws://…/ws/orchestrator?scv_session=<token> and send back the scv_session cookie from step 1.

verifyBrowserSession treats the cookie as the signed payload and the query param as its CSRF counterpart; presenting either alone is 401. Then send {type:'session', …} — not type:'start', which the relay silently ignores, producing an open socket that never returns a ServerHello.

Step 2 is read in ScvBridge._readCallCenterConfig and emitted as a callCenterConfig event on every init() — not only on the lazy-telephony-boot path, which is skipped whenever localStorage credentials already booted the provider.

The panel → connector path

The panel LWC and the connector live in different LWC trees (the connector is an iframe SCV owns), so a bubbling CustomEvent is silently dropped. The working transport is Lightning Message Service on ServiceCloudVoiceMessageChannel__c, with source set to anything other than 'CONNECTOR' (trueServeBridge filters that value to avoid re-consuming its own outbound publishes) — and the reverse direction publishes as 'CONNECTOR', see Connector → panel status channel. The subscription uses APPLICATION_SCOPE, so the same component works from a record page or the utility bar. translationSettings remains the bundle API name for Lightning-page compatibility, and its App Builder label remains TrueServe Call Settings; the visible card is Call controls. It is the single publisher for TRANSLATION_CONFIG, TRANSLATION_TOGGLE, and RTC_AUDIO_CONFIG; agentConfig alone hosts the saved-defaults editor; its queue Save remains independent.

The panel displays Call controls with visible On/Off switches, disabled between calls. Provider, voice and grammar appear as read-only profiles; their only editor is Phone Telephony Settings. The MessageContext wire initializes messaging directly when it arrives, without waiting for a user-triggered render. Every connected call emits status, including calls with translation off. Model loading and fallback details remain visible. Full contract: docs/translation-lwc-integration.md.

Browser translation modes (demo only)

browser-aws (Transcribe/Translate/Polly via Cognito) and browser-direct (Deepgram/OpenAI/ElevenLabs) still work and are untouched, but they hold vendor credentials in the browser. They are demos; orchestrator-grpc is the production path and the only mode the packaged panel selects.

Dependencies

Service	Purpose
ai-orchestrator	Translation agent sessions (gRPC :50051)
ai-manager	Authoring + publishing the translation agent
Salesforce	SCV partner-telephony APIs, SCRT2 transcripts
Amazon Connect / Genesys Cloud	Real call control and WebRTC media

Environment variables

Name	Required	Purpose
SERVER_PORT	yes	Express HTTP port (Vanguard 3031)
TELEPHONY_PROVIDER	yes	Connect, Genesys, or demo
ORCHESTRATOR_GRPC_ENDPOINT	yes for translation	ai-orchestrator gRPC, e.g. localhost:50051
SCV_TRANSLATION_AGENT_ID	no	Lowest-precedence default translation agent (a published pipeline id)
SCV_TRANSLATION_PREFIX	no	Label on the translated line of a transcript entry. Default [Translation]; an empty value means unlabelled (not the same as unset)
SCV_TRANSCRIPT_TRANSLATION_ONLY	no	true sends only the translation, dropping the original. Default false
SCV_TRANSLATION_ACTIVE_ATTRIBUTE	no	Contact attribute the transcript gate reads. Default TrueServeTranslationActive; must match the lambda's TRANSLATION_ACTIVE_ATTRIBUTE
SCV_TRANSCRIPT_ANALYSIS_BUCKET	no	S3 bucket for the Contact-Lens-shaped post-call document — normally the instance's own storage bucket. Unset disables publication entirely (the default)
SCV_TRANSCRIPT_ANALYSIS_PREFIX	no	Key prefix for that document. Default Translations/Voice — deliberately not Connect's Analysis/Voice; the filename infix is _translated_, not _analysis_
SCV_TRANSCRIPT_ANALYSIS_REGION	no	That bucket's region. Falls back to AWS_REGION; a wrong value fails PermanentRedirect and loses the document
AWS_CONNECT_INSTANCE_ID	for Connect	Instance id used for contact-attribute writes and stamped into the analysis document's CustomerMetadata.InstanceId
CONNECT_REGION	for Connect	The instance's region — the CCP embed and the server-side control-plane client. Falls back to AWS_CONNECT_REGION, then AWS_REGION
SCV_CONNECT_LOGIN_URL	for SAML Connect	IdP-initiated SSO URL popped in place of the instance's dead /login (Call Center XML reqConnectLoginUrl wins)
SCV_SERVICE_TOKEN	no	Service token accepted at WS upgrade
SCV_BROWSER_SESSION_SECRET	no	HMAC secret for browser WS session tickets (auto-generated in Vanguard)
SCV_ALLOWED_ORIGINS	no	Origins allowed to mint sessions / open WS. Blank = same-origin
SF_SCRT_INSTANCE_URL, ORG_ID, CALL_CENTER_NAME	no	SCRT2 fallbacks. Normally unset — SCV supplies all three at runtime through /api/configureTenantInfo; SF_SCRT_INSTANCE_URL is only read under IS_LOCAL_CONFIG=true. There is no SCRT_BASE_URL
SCRT_PRIVATE_KEY / SCRT_PRIVATE_KEY_FILE	for SCRT2	RS256 key signing the SCRT2 Bearer — inline PEM (\n escapes accepted) or a path; falls back to server/private.key. Public half must be the Call Center's certificate
DEEPGRAM_API_KEY, OPENAI_API_KEY, ELEVENLABS_API_KEY, COGNITO_*	no	Browser demo modes only

Full list: apps/connect-scv-translator/config.env.template.

Vanguard

overmind restart connect-scv-translator

Then, to exercise translation:

1. Author a translation agent in ai-manager (type speech_to_speech, subtype translation) with both flows configured, and publish it.
2. Set SCV_TRANSLATION_AGENT_ID (or the Call Center XML key, or the panel field) to its id.
3. Place a call through Connect or Genesys and start translation from the panel.

Expect two SessionConfigs in the orchestrator log per translated call, one incoming and one outgoing.

Production

Helm chart at charts/connect-scv-translator. Live Translation has only been exercised in Vanguard so far — KMS/EKS hardening (relay scaling, binary WS frames instead of base64 JSON, token refresh for Genesys implicit grant) is not yet done.

See also

• Translation LWC integration — the panel↔connector message contract
• Realtime translation — the orchestrator-side translation pipeline
• Universal pipeline builder
• Agent Studio for Salesforce package
• SIP integrations

For Takeda’s care-model handoff, native CustomerRecordId remains the coordinator Contact while SalesforceFields.RelatedRecordId accepts the parent Case. The preview parser validates these independently; a Case cannot become the caller Contact. The compact layout keeps its existing writable reason/summary inputs and formulas. See the Takeda care model.