Human Supervision — the supervisor copilot loop

Last reviewed: 2026-09-29

How a live voice call reaches a human supervisor's chat copilot, how the copilot stays silently informed, how questions flow both ways without blocking, and what happens when the call ends. Authoring-side detail (the capability card, request_support, prompt tokens) lives in ai-manager-layout.md; this page is the runtime loop.

Escalation and response are separate paths

For the Salesforce Human Supervision pattern, attach a short asynchronous workflow to request_support: START → Salesforce action → END. Publish the configured escalation event/record with the host-bound related conversation ID. The workflow finishes after dispatch. A wait node is needed only for a separate business process that actually consumes a later callback; supervisor chat does not need one. The legacy human_review handler remains for older configurations, but its manifest has palette: false; do not generate it for this pattern.

The human response is not a workflow output:

Voice agent ── request_support ── Salesforce escalation ── workflow finishes
     │                                │
     └── transcript relay ── connected supervisor chat ← human specialist
     ↑                                │
     └──── inject_message_to_conversation (wait_for_reply=false) ────┘

Salesforce automation routes the request and opens its supervisor chat with related_conversation_id. The voice agent's supervisor_agent_id and that embedded agent must match. Bind the injection target to this runtime variable, not model arguments. The caller's answer returns through the transcript relay. Workflow acceptance proves only queuing; an event receipt proves dispatch, not record creation, approval or completion of an underlying business action. Explicit human approval can be relayed through chat. Claim submission only from an actual submission result or an explicit specialist completion confirmation, according to the scenario's chosen owner. Customer transcripts cannot supply that authority. Test the paths separately and then rehearse their connection.

The one-conversation rule

There is exactly one supervisor conversation per (supervisor agent, related call), with a deterministic id: uuid5(NAMESPACE_URL, "trueserve:supervisor:{bare_agent_id}:{related_row_uuid}") — derived by derive_supervisor_conversation_id (db_queries.py) and served by conversation-api's POST /conversations/supervisor-session. Both parties call it and therefore converge on the same thread:

• agent-host's transcript relay (stream_transcript_to_supervisor, mcp_server.py) when the voice agent escalates, and
• the CRM copilot embed (surface-app resolves it whenever its session vars carry related_conversation_id; the Salesforce LWC already sends that — no SF-side change was needed).

The endpoint is idempotent: a live pipeline answers live: true (nothing created); an existing-but-ended conversation is re-opened on the SAME id with resume_history metadata; only a truly new pair creates a fresh conversation. The related id is canonicalized to the conversation row uuid first, so the relay (holding the telephony session id) and the embed (holding the uuid from the Case record) can't fork the thread — the two-addresses rule of conversation-api.

Before this existed, the relay created its own conversation while the supervisor chatted in a second one the embed minted: the transcript pumped into a thread nobody was reading.

Resume: the copilot remembers

When supervisor-session re-opens an ended conversation it stamps resume_history: "true" into the session metadata. The orchestrator's build_llm_context (llm_context.py) then folds the stored conversation_messages (final turns, capped at 80) back into the LLMContext behind a restored-history system note, and suppresses every greeting mechanism. The SSE replay restores what the viewer sees; this is what makes the agent remember — it answers questions about the earlier session and never re-greets or re-sends an approval the history already shows.

Silent monitoring: transcript kind + the [[SILENT]] gate

Finalized transcript revisions are deduplicated before injection by normalized (role, turn_id, text). Providers may publish the same final text under two Redis ids as successive revisions; those are one logical turn. A correction whose text changes still passes through.

Each remaining [Caller]/[Agent] turn is injected with kind: "transcript" and an explicit trigger policy. Caller turns use trigger_response: true, because they can answer a relayed question or require intervention. Agent turns use trigger_response: false: they are appended as context but do not cause the supervisor agent to paraphrase what the voice agent just said. The orchestrator wraps triggered transcript turns (external_injection_processor.py) with a contract: this is the call, not a message to you; reply [[SILENT]] unless it genuinely needs the supervisor. The sentinel is imported from text_turn_serializer.STAY_SILENT_SENTINEL so the instruction and the suppressor cannot drift.

The suppressor is a single-branch TextTurnGate inserted right after the LLM for a linear text_chat agent with settings.text_turns.allow_silence: true (_maybe_insert_single_branch_silence_gate, universal_builder.py) — the same incident-hardened gate multi-agent text turns use. A turn whose whole text is the sentinel is swallowed before the publisher and the assistant aggregator: nothing reaches the chat surface, nothing pollutes the context. This is opt-in — an ordinary chat agent must answer every turn.

A raw [[SILENT]] never reaches the chat surface — on any turn. The gate always buffers-and-inspects a turn when allow_silence is on (the withhold releases the instant the text stops being a prefix of the sentinel, so a normal reply streams live after its first token). Streaming a turn live and only then discovering it was sentinel-only cannot be un-sent, which is exactly how the literal string [[SILENT]] used to render as a chat bubble.

What the transcript-vs-direct distinction decides is only what a sentinel-only turn means, not whether it surfaces. The gate reads its LLM context at generation start (built with context= from _get_aggregator_context) and checks whether the triggering message carries the [LIVE CALL TRANSCRIPT] marker (text_turn_serializer.TRANSCRIPT_LINE_MARKER, the same constant the wrapper stamps):

• Transcript line → expected abstention. The copilot correctly stays silent on the call itself; the turn is swallowed quietly.
• Direct supervisor message, call_status notice, or the opening escalation brief (no marker) → model error. The copilot was told to answer and instead abstained; the raw sentinel is still suppressed (never rendered) and the gate warn-logs the misbehaviour. The prompt steers the model to answer these turns — the transcript wrapper re-teaches per line that the stay-silent rule applies to transcript lines only, and the opening explicitly says do not stay silent — so this is a last-resort guard against a rare model slip, not the common path.

Earlier iterations got the tradeoff wrong in both directions: first the gate dropped every sentinel-only turn regardless of origin, so a direct question the model wrongly answered [[SILENT]] produced no response at all; the fix for that streamed direct turns live without inspecting, which surfaced the raw [[SILENT]] string whenever the model abstained anyway. Always-inspect + suppress-everywhere is the resolution: a real reply always surfaces, the raw sentinel never does.

History: without the gate and the kind, every relayed line triggered an unfiltered generation wrapped as "[ADVISOR INSTRUCTION]". Triggering the voice-agent side as well, and relaying repeated final revisions, then multiplied one exchange into several near-identical messages. Both boundaries are enforced at the relay now.

Questions both ways, without the 20-second cliff

The supervisor's guidance reaches the call via inject_message_to_conversation. Two rules:

• Prefer fire-and-forget (wait_for_reply: false) whenever transcripts are streaming: the caller's answer arrives as a transcript line and the importance gate is exactly what lets the copilot react to it — no blocking, no window to miss.
• When wait_for_reply is used anyway, the default window is 45 s (a voice round trip — TTS speaks the instruction, the caller answers, STT finalizes — routinely takes 20+; the old 20 s default timed out on answers plainly given), and a timeout result now says the instruction was delivered and the answer will arrive in the transcript, so the model neither re-sends nor reports failure.

Call end: notice + card expiry + post-call mode

When the source call emits STATUS_ENDED, the pump (before stopping):

1. Expires approval cards — scans the supervisor conversation's Redis stream for rendered ApprovalRequest components and republishes each as an updateComponents with callStatus: "ended" on its original surface (_expire_approval_cards). The scanner accepts both persisted payload forms: the inner updateComponents body and the full A2UI operation envelope. The card (copilot-support catalog ≥ v1.1.0) hides its buttons and shows "Call ended — this approval is no longer actionable".
2. Injects a kind: "call_status" platform notice (triggered): the agent acknowledges once with an outcome summary and switches to post-call mode — Q&A about the call, summaries, follow-ups; no new approval cards.

The chat session itself may later end on the worker idle timeout; that is fine because the next visit resumes it (above).

Guidance default

transcript_guidance left blank now seeds a platform default (_DEFAULT_TRANSCRIPT_GUIDANCE in agent-host): monitor silently, speak up on answers/decisions/risks, never narrate tools or cards. Authored guidance replaces it. The seed happens only on a created conversation — a resumed one already carries it in history.

Proactive opening: summary + escalation reason

The escalating agent passes an escalation_reason to stream_transcript_to_supervisor (a model-provided, non-bound arg: the live agent knows why it is escalating this specific call). On a created supervisor thread the relay seeds ONE triggered opening (kind: "raw", which carries no transcript marker, so the silence gate never suppresses it) instructing the copilot to open the chat now — before the supervisor types — with a brief situation summary and the escalation reason. Absent a reason it still opens proactively. The opening leads with that instruction and scopes the standing silence guidance to "the rest of the conversation"; leading with the "stay silent unless…" guidance instead primed the copilot to abstain on its very first turn, so the opening came back as a bare [[SILENT]]. Before any of this, the chat sat idle waiting for the supervisor to speak first, and the earliest opening seed was itself swallowed by the silence gate.

The opening seed is retried until a live subscriber receives it

The opening is the ONLY message carrying the live-call context — the situation summary, the escalation reason, and the interpolated guidance block (which is where the customer's phone number and any other captured variables reach the supervisor; see below). It is delivered by POST /conversations/{id}/inject, which publishes over Redis pub/sub — and pub/sub has no backlog. The supervisor pipeline only subscribes to inject:{conversation_id} once it is running, so an opening published in the ~100 ms window before that subscription completes is dropped on the floor while the later transcript lines — sent once the subscription is live — land fine. The visible symptom was the supervisor chat agent never knowing the caller's phone number even though the voice agent had captured it correctly.

conversation-api reports a delivered count on every inject (the number of live pub/sub subscribers that received it — 0 means the session is dead or not yet subscribed). stream_transcript_to_supervisor (mcp_server.py) now retries the opening seed until delivered >= 1, bounded by _OPENING_SEED_MAX_ATTEMPTS × _OPENING_SEED_RETRY_INTERVAL_S (~3 s), and stops the instant it lands — so exactly one live delivery triggers a greeting (no duplicate opens). The retry runs before the transcript pump starts, which also guarantees the subscription is live before the first relayed line is sent. A supervisor chat that never opens within the window logs a warning and gives up rather than stalling the escalating tool call. Transcript lines are unaffected: by the time the pump runs the subscription is confirmed live.

Passing context via {{variable}} tokens

Authored guidance may embed {{variable}} tokens (e.g. History: {{payment_history}}) to hand the supervisor chat agent context captured during the call. The tokens are resolved in the orchestrator, not agent-host — agent-host has no view of the live session's variables, so the guidance value must arrive already interpolated.

The guidance is bound to the stream_transcript_to_supervisor MCP tool as a parameter_bindings["guidance"] value in tool_config_resolver.py resolve_builtin_mcp_tool_configs. resolve_variable_bindings only substitutes a value that is entirely one token, so embedded tokens fall through as literals; the MCP binding wrapper additionally runs interpolate_variables / interpolate_binding_values (binding_utils.py) over resolved string bindings against a live variable context rebuilt at call time (var_ctx_provider in callback_factories.py, wired in tool_wiring.py). The live context matters: the wrapper is built at pipeline-assembly time, before any call activity, so a payment_history written mid-call is not in the build-time snapshot. A token whose variable is absent is left literal so partially-authored guidance degrades gracefully.

The author types tokens directly into the Transcript-handling guidance box in the Human Supervision capability inspector; there is no separate mapping step. That box is a TokenPillTextArea (TokenPillTextArea.tsx), so {{variable}} tokens render as the same colored pills the main prompt editor shows (it reuses tokenPillsExtension() from token-pills.ts) — the stored value stays raw {{…}} text, the pills are view-only. Session variables are the same ones any prompt {{variable}} reference or set_variables write uses.

Typing @ or {{ opens a variable-only autocomplete (unlike the main prompt editor's hierarchical @/{{ menu, this offers only variables — no tools, subagents, components, or logic statements — because guidance can embed nothing else). Selecting one inserts {{name}}. The variable list is the same union the prompt editor's picker shows (useAgentPromptSources().lifecycleVariables plus {{…}} references detected in the stage config): declared variables, phone variables ({{ani}}/{{dnis}} on speech-capable agents), lifecycle workflow outputs, and capability-managed variables. ServiceConfigForm computes it at the top level and threads promptVariables → NodeCapabilitiesAndBehaviors → HumanSupervisionCapability → TokenPillTextArea (the capability inspector renders in ServiceConfigForm, not the child ConfigFieldsEditor where the prompt editor's own list lives).

Build-time wiring: the agent-host MCP fetch must not abort the build

Exposing stream_transcript_to_supervisor is a build-time MCP fetch: resolve_builtin_mcp_tool_configs emits a ResolvedMCPConfig pointing at http://agent-host-mcp:8003/mcp, and phase 1 of the universal builder calls MCPManager.get_tools_schema_only against it before the LLM/S2S service is created. That fetch is authenticated by the shared internal service secret: the orchestrator sends X-Internal-Service-Token (AGENT_HOST_INTERNAL_SERVICE_TOKEN) only when the value is set, and agent-host accepts it only when its own AGENT_HOST_INTERNAL_SERVICE_TOKEN matches (_is_internal_service_call, constant-time). Both services must carry the same value. On KMS it is wired into both subcharts' secretEnv from trueserve-encryption-secrets — in the base charts (ai-orchestrator, agent-host) and the per-env stack overrides that wholesale-replace those lists (values-kms-dev.yaml, values-kms-qa.yaml) — with optional: true so a not-yet-provisioned key does not crash the pods. The secret value itself is still a two-step follow-up that must land together: add the agent_host_internal_service_token property to the AWS Secrets Manager secret trusrv-<env>-orch-encryption-key (Terraform, infra/stacks/aws-kms/<env>/) and then uncomment the matching ESO entry in deployment/<env>/externalsecrets-values.yaml. Do not add the ESO entry before the property exists — a remoteRef to a missing property errors the whole trueserve-encryption-secrets ExternalSecret (which also carries MASTER_ENCRYPTION_KEY). Until the value lands the fetch is rejected 401 and the feature is inert, but — because of the hardening below — the greeting is unaffected.

A failed fetch must degrade to no-tool, never abort the build. Earlier it did the opposite: a 401 makes raise_for_status() fire inside the MCP client's anyio task group, whose cross-task teardown surfaces to get_tools_schema_only as a spurious asyncio.CancelledError — a BaseException its except Exception could not catch. It escaped through phase 1, aborted the whole pipeline build, and the caller heard total silence: no greeting. Because the toggle is the only thing that adds this fetch, enabling "stream transcript to the chat agent" (or stream_transcripts defaulting on with a supervisor_agent_id) killed the greeting while disabling it restored it. get_tools_schema_only now catches that spurious cancel (re-raising only a genuine cancellation of its own task, distinguished by task.cancelling()) and returns (None, None); merge_tools_schemas keeps the tools already added, so a missing supervisor-relay tool no longer takes the greeting down. Regression: test_mcp_schema_fetch_failure.py. The token is still the real fix for making the feature work — the hardening only stops its absence from being catastrophic.

Pieces and their tests

Piece	Where	Test
Deterministic id + ensure endpoint	conversation-api supervisor-session	test_supervisor_session_id.py
Relay: kinds, seeding, call-end	agent-host mcp_server.py	test_transcript_relay.py
Transcript wrapper + silence gate	ai-orchestrator	test_supervisor_silence_gate.py
MCP fetch failure ≠ build abort	ai-orchestrator get_tools_schema_only	test_mcp_schema_fetch_failure.py
Guidance {{variable}} interpolation (live)	ai-orchestrator binding_utils/callback_factories	test_binding_utils.py, test_binding_wrapper_signature.py
Resume seeding	ai-orchestrator build_llm_context	exercised live; greeting suppression asserted by the resume log line
Embed resume	surface-app ensureSupervisorConversation (both sdk.ts copies) + /api/conversations/supervisor-session proxy	—
Card expiry rendering	copilot-support catalog ApprovalRequest (callStatus prop)	—

Known gaps / follow-ups

• Case-context tools: the post-call copilot answers from restored history and get_conversation_transcript; it has no Salesforce case-details tool yet.
• The greeting-path build_tool_name_map_sync limitation (no DB) still applies to HS names.
• Multi-call-per-case ("hybrid") threading was considered and deferred — the thread is per (agent, call), not per Case.

See also

• ai-manager-layout.md — authoring the capability
• conversation-api — inject kinds, the two addresses
• universal-pipeline-builder.md — text turns, injection chain