Allianz annuity demonstration¶
Status: Current | Last reviewed: 2026-10-09
Use this runbook to rehearse the two scenarios from Annuity Contact Center Sample Use Cases_Initial Demo.docx in the local TrueServe tenant. The demonstration uses real speech and language providers with executable synthetic insurance services. No tool sends a claim, illustration or service note to an insurer.
See the validation report for passing scenarios, measured latency, evidence identifiers and remaining integration work.
For the newer Allianz Script v2.docx, use the October 9 script validation. The older sample-use-case passes below do not establish that the new producer script passes.
The SVG-based producer experience is covered by the native producer workspace. It extends existing Illustration Assist 4870 with recurring-withdrawal capture and the native Salesforce center; the voice and beneficiary scenarios below retain their existing routes.
Latest console feedback: references and concise recaps¶
The September 30, 17:12 UTC console conversation
b9659440-ddca-40a4-b275-329133022ea5 exposed a separate reference defect:
Taylor read the asynchronous workflow's execution_id as the caller's
illustration reference. That UUID proves neither business submission nor human
approval. The call also expanded a recap-only request into a long closing,
and an acknowledgement ran into the beginning of its substantive answer.
The preparation fixture now returns review_reference: REV-291640 alongside
the unchanged internal request ID and revision. It tracks the synthetic review.
After streaming and escalation succeed, Taylor gives the short review reference
with the pending status. IFR-291640 remains the script's submitted-illustration
reference, usable only after the related specialist explicitly confirms
completion for the current assumptions. Neither the internal prepared ID nor a
workflow/conversation UUID may be spoken or displayed as a customer reference.
If the service returns no short reference, Taylor must say none is available;
truncating a UUID is not a fallback. These fixed references belong to the demo
fixture; a production service must issue and persist its own references.
The default spoken recap now covers outcomes, outstanding work and next steps, normally 35–55 words and at most 65, counting individually spoken reference letters and digits. It does not repeat verification or financial figures already explained. One short customer reference may help follow-up. Commission retains its final-processing, eligibility and adjustment qualification, and pending review stays pending. A detailed recap remains available on request. The internal end-session summary is separate from speech; full tool receipts, assumptions and recorded notes also remain in the conversation evidence.
A recap-only request keeps the call open. A later goodbye closes once without repeating a just-delivered recap. Note-save confirmations are one short sentence after the requested answer has actually been spoken; an interrupted or undelivered commission answer still needs its amount, cycle and qualifications. The voice must not add a note receipt number to the backend's confirmation. The voice prompt allows a standalone acknowledgement, asks the model to keep listening through unfinished phrases, and keeps application/duplicate-task notices silent. Prompt guidance reduces these speech problems but does not guarantee that every native voice turn will be fluent.
refine-console-feedback.ts fixtures updates only the named preparation tool,
its native mocks, and the matching capability tool/test; the producer capability
is 17.0.1. Other capability settings and tool gates are preserved.
refine-experience.ts saves only the voice/backend prompts with revision checking.
For this release, refine-console-feedback.ts apply selects GPT-5.6 Terra
for the client backend with reasoning effort none, after comparing it with
GPT-4.1 at default sampling and temperature 0.1. GPT Live, the Gleam voice,
authentication and review gates stay configured as before. Both the owning
stage's model selection and its nested backend configuration are updated;
temperature is removed for the new backend.
refine-console-feedback.ts publish publishes only agent 4444 through its
native API and leaves CCaaS availability untouched. Existing sessions retain
their starting configuration. The earlier full-figure closing requirements
below describe historical rehearsals, not the current default recap.
Agent 4444 version 4 was published on September 30 through that API.
Publication readback verified the tested voice/backend prompts, Terra with
reasoning effort none, the expanded short-reference tool schema, and the
unchanged enabled CCaaS integration
ec667d5f-cb9c-4423-a6d1-0626d1c9ef72. Start a new console preview to use
the updated configuration.
Validation includes eight fixture unit checks, four implementation checks in
the native isolated code executor, and eighteen real backend probes.
The backend suite reads the actual native end_session argument schema,
including a nonempty text summary. It checks separate recap/closing intent,
new submission versus review references, reference availability, retained
financial facts in metadata, goodbye after an intervening clarification, and
delivery of still-unanswered commission details before confirming a note.
Detailed financial answers remain available; recap-only does not invoke
end_session; workflow UUIDs stay out of spoken output. GPT-5.6 Terra with
reasoning effort none passed all eighteen probes. These isolated probes use
synthetic state and do not execute Salesforce or SMS actions.
Earlier audio attempts are retained. Batch
56840611-7bb7-4d8d-9204-4c233a6b9b8d found duplicate closing recaps and an omitted
commission caveat. Batch 69763a10-1df1-432e-bf57-0c185b500a55 passed the completed
journey at 87%, with a specialist update splicing into the earlier review status;
its pending case failed in the simulated caller's stopping decision and cannot
be counted as a passed agent test. The subsequent caller fixture explicitly says
goodbye before stopping and provides all identity fields in its opening.
A caption split after “We” in the first batch had only 193 ms between audible
starts; adjacent caption fragments alone do not establish an audible stumble.
Three opening probes with an earlier acknowledgement candidate measured 1.527, 1.772 and 2.641 seconds from acoustic speech end to first audible PCM. They waited for the subsequent identity question and detected no internal backend/delegation narration or repeated answer start. An earlier candidate included a 6.371-second turn without an acknowledgement, so these small samples do not establish a latency guarantee. Full responses and phone playback have additional latency.
The release candidate's three fresh opening probes measured 5.636, 1.397 and 2.041 seconds (median 2.041 seconds). They passed the consent and internal-narration checks. The first skipped its acknowledgement and waited for the verification question; the third repeated “To verify you” within its answer. These results confirm that prompt changes alone have not eliminated hesitation or guaranteed immediate acknowledgement.
console-regression-test.ts start saves and queues two native audio tests:
pending review and specialist-confirmed submission, each with a recap-only
request followed by another question before goodbye. Read a batch with
console-regression-test.ts <batch-uuid>. voice-latency-test.ts
console-feedback-release also waits for the substantive identity question so
an acknowledgement alone cannot pass its internal-narration check. Private
reports are in .local-data/allianz-demo/console-* and sept30-*.
The GPT-4.1 temperature-0.1 batch
e6e051c4-47a4-4356-9f35-272ac70738e5 still omitted the not-yet-generated
document status in the completed journey. Its pending review also received
judge flags for the portal route and commission as-of date; the latter is
not the requested processing-cycle date. These findings motivated the backend
comparison rather than treating isolated prompt checks as full-call validation.
The initial GPT-5.6 Terra batch 125d2291-43da-43a3-984f-83ee5b360a21
exposed an undelivered commission answer followed by a note-only confirmation.
The completed journey also added an unsupported note receipt in the voice
layer. These findings led to the explicit unheard-answer rule and a simpler
voice prompt forbidding extra note receipts.
The subsequent audio batch a5b1220a-133a-4b5d-89e2-4f0e30a89ef0 is pinned
to draft revision 2026-09-30T18:42:19.486Z and configuration hash
d0472f77dfc4c3c5da3d4a6c15fc7b491b9b22bff9767d2c75d540d5d66ccc36.
Both native audio conversations completed. The specialist-completed journey
76bca71a-fa62-4860-939e-bd3de000ed87 passed at 96%: the supervisor
chat delivered completion, the agent spoke the actual submission reference
and document readiness, answered commission before confirming the saved note,
gave a compact recap, answered the subsequent guarantee question, and ended
with one goodbye. Its remaining minor grader finding was a redundant
verification instruction after the test authentication had already completed.
The pending journey 56d6be07-b261-4990-9c56-02637dea0c8e scored 72%
and failed its full scenario evaluation. It spoke
REV-291640, gave a compact recap and kept the call open for the guarantee
question. It correctly ended with the review expired and the illustration
not submitted. Its asynchronous expiry update, however, interrupted the
commission answer after “so payment”; the caller had already heard “No” and
the processing/eligibility/adjustment qualifications. The final response also
repeated the broader recap after the changed review status. The grader's
major finding was two confirmations: assumptions before preparation, followed
by separate dispatch confirmation, despite the prompt requiring one combined
confirmation after preparation. It also flagged the unfinished guarantee
answer and an omitted portal route in the pending recap; the pending state
does not establish that a document will be produced. Its score arrived after
publication. The reference and recap-length fixes worked, but this is not
recorded as a fully passing voice test. Prompt-level sequencing and fluency
remain limitations, especially when a supervisor/status update arrives
during speech.
Targeted TypeScript checking, fixture tests and whitespace checks passed.
ESLint reported no errors and 22 no-explicit-any warnings in the demo scripts.
September 30 call and customer-surface refinement¶
Reviewed Vanguard call c1b0a09e-e4e1-4884-9a56-d79d8d1830b9
(16:28–16:36 UTC) against the Scenario 2 contract extracted in
docs/_planning/allianz-annuity-agent-demo.md.
The original DOCX was not present in this checkout or its Git history during
this review. Its extracted contract supplies the policy figures, illustration
assumptions, commission question and factual closing; human supervision is the
added demo variant. The source supplies no review SLA.
The call verified the producer, retrieved the right figures, dispatched review, saved the note and correctly closed with review pending. It also tried a policy lookup with missing identity fields, repeated “One moment,” never displayed the illustration confirmation, and invented a one-to-two-business-day review estimate and direct specialist phone attendance. The first policy display also incurred a failed card attempt before opening the visual session.
conversation-experience.ts now separates a concise GPT Live speech policy from
the backend procedure. The voice can acknowledge while tools run, interrupt for
corrections and repeat a still-current fact. The backend owns permission,
verification, assumptions, source-backed answers and actual completion. It
collects missing identity fields before lookup, opens visuals before sending
cards, renders each matching component at its step, preserves one combined
assumption confirmation and saves the full recap for closing. It cannot promise
a review time, immediate attendance, priority routing or a specialist joining
the phone call. “No additional withdrawals” remains the exact assumption;
shortening it to “no withdrawals” changes its meaning.
refine-experience.ts uses revision-checked native draft saves and updates the
inline backend template, system message and voice prompt together. It preserves
model selection, authentication, workflow bindings and receipt gates. A focused
delegation_instructions addendum follows native tool guidance: an ordinary
goodbye requires the factual recap, and an estimate confirmation must retain
“not a final surrender quote.” The
installer uses the same prompt definition. Earlier refine-voice.ts and
voice-refinement.ts describe the previous rehearsal revision; do not reapply
them over this replacement.
Compiled Allianz components¶
Producer catalog 1.0.1 includes the Allianz wordmark, blue, actual Allianz Neo regular/bold fonts, financial typography, responsive fields and controls, and distinct review/submission states. A pending or merely approved illustration does not display a submitted reference. Revised assumptions reset the form; confirmation disables duplicate submission.
Three integration defects explained the missing branding:
- The compiler prefixes CSS with
.tsf-catalog-allianz-producer; each component now emits that ancestor. Putting it on the styled element itself is insufficient for descendant selectors. - The catalog loader injects CSS through a blob URL, so relative font URLs had no usable HTTP base. The catalog registers font faces against the actual surface origin, once per loaded module.
- Public brand files now live at
/assets/allianz/, which bypasses page routing. The previous/allianz/surface paths could require a session or redirect to a locale page. Surface brand logo settings use the public asset path as well.
The old raw-CSS browser fixture bypassed native scoping and falsely suggested
that styling worked. surface-catalog-test.ts now uses the native compiler,
presigned bundle URLs, integrity verification, catalog registration and an
authenticated real surface-app iframe. Both preview and published artifacts
passed at desktop, 390 px and 320 px widths, including loaded font faces,
decoded logo, exact amounts, keyboard confirmation, duplicate suppression,
pending-reference suppression and corrected assumptions. browser-test.ts
now covers only the retired review desk; it is not styling evidence.
Response timing and repeatable checks¶
Three fresh native voice sessions per revision used paced PCM16 input at the negotiated rate. The acoustic boundary is the last 10 ms caller window at or above -54 dBFS. First response means received non-silent output audio, which can be an acknowledgement; it is not the time to a completed business answer, browser playback or production phone audio. Caption timestamps are not used.
| Prompt revision | First audible response, each probe | Median |
|---|---|---|
| Before | 5,536 / 2,749 / 6,111 ms | 5,536 ms |
| Initial September 30 refinement | 2,846 / 1,737 / 1,361 ms | 1,737 ms |
| Final refinement, including closing contract | 1,409 / 1,595 / 1,939 ms | 1,595 ms |
This small sample shows improvement but does not establish a production percentile or a subsecond guarantee. Models and native transport were preserved. The nine isolated real-backend-model checks cover missing identity, values-only requests, repeated confirmation, unsupported turnaround/phone-join promises, approval versus submission, completed document-delivery wording and a pending final recap, including an ordinary goodbye after the facts were already spoken. Those checks execute no Salesforce/SMS tools.
The final native audio rehearsals use draft revision
2026-09-30T16:55:20.462Z, GPT Live with the existing GPT-4.1 client backend,
and sandbox insurance/authentication/Salesforce responses. The specialist
response still traverses the configured supervisor chat.
| Rehearsal | Conversation | Outcome |
|---|---|---|
| Complete Scenario 2 | 3d8281cf-7be3-4e0b-9f44-898ec367c516 |
Passed, 99%; one minor verification-acknowledgement disfluency |
| Estimate only | 0dcc104b-18fe-4a3f-abe4-1510339d7257 |
Passed, 100%; qualified estimate, no illustration, factual closing |
| Specialist rejection, before the final closing addendum | 593e8da7-f7a9-4162-ba5c-3d516d70b42d |
Passed, 100%; no submission/reference |
Independent event checks confirm one end action and finalized transport in both
final-revision runs. The complete journey renders PolicySnapshot, an
IllustrationRequest in review_required, pending and submitted states, and
CommissionStatus. It obtains one assumption confirmation before escalation,
receives the explicit specialist submission/reference, saves the note and
delivers the closing figures, assumptions, reference, cycle and final-processing
qualification. Eligibility and adjustment qualifications were also spoken in the
earlier commission answer; the final recap shortened that wording. The separate
ServiceReceipt component passed browser checks but was not emitted in this call;
the updated illustration card carried the completed request status.
The first full and estimate-only attempts scored 72% and 75%: the backend
omitted the factual closing and the “not a final surrender quote” phrase,
respectively. The focused backend output addendum corrected these defects in
the repeated native runs; those original failures remain in the evidence.
TypeScript checking and targeted ESLint completed with no errors (the test
scripts retain advisory any warnings). These are demo tests, not live carrier
delivery or insurer-system validation.
Run from the repository root, using the configured city environment:
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/experience-model-test.ts
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/surface-catalog-test.ts published
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/voice-latency-test.ts after
Private evidence is in .local-data/allianz-demo/sept30-*. Start a fresh
4444:draft preview for the updated speech policy. The customer catalog is
published; an already-running call retains its original configuration.
Installed assets¶
| Asset | Local identifier | Purpose |
|---|---|---|
| Allianz Producer Services | Agent 4444 |
GPT Live voice servicing for David Chen and Helen Carter |
| Allianz Producer Specialist | Agent 4445 |
Human supervisor chat and live-call guidance |
| Allianz Beneficiary Assist | Agent 4446 |
Bereavement coaching, claims answers, verified facts and intake |
| Specialist request | Workflow 1847 |
Publish the Salesforce escalation and finish asynchronously |
| Supervision instructions | /allianz-demo in AI Manager |
Salesforce and connected-chat rehearsal instructions; no approval controls |
| Allianz surface brand | 89e91d54-281e-4244-8da0-41636b4de0cf |
Official wordmark, Allianz Neo and blue |
| Producer catalog | allianz-producer, version 1.0.1 |
Policy, assumptions, commission and receipt components |
| Beneficiary catalog | allianz-assist, version 1.0.0 |
Verification-aware claim facts |
Identifiers are local installation results. The installer reuses named assets and writes its resolved IDs to .local-data/allianz-demo/installed.json. It does not change CCaaS routing.
Scenario 1¶
Jordan helps Elena Martinez report Robert Martinez’s death. The Copilot listens to separate caller and representative audio channels and never speaks to Elena.
The first-response path matches approved phrases in final speech transcripts and publishes short coaching or cited knowledge cards without a model call. The knowledge pack covers documents, secure submission, benefit qualifications, tax referrals and review timing. Unsupported phrases leave the previous card in place; they do not generate an unsupported answer. This is a bounded demo knowledge pack, not a production knowledge connection.
Independent workers use copilot.parallel_workers: true; slow fact retrieval and form extraction cannot hold the other workers’ queues. Shared enhancement and actionability gating, when configured, remain upstream. This demonstration disables the shared model gate. approved_responses and approved_only are advanced configuration fields on each Copilot type, supported for card-producing suggestion workers. Cached fields must match the worker’s extraction schema. They are suitable for non-personal guidance only.
Deepgram Nova-3 receives stereo 16 kHz PCM with 150 ms endpointing. The form extracts from final transcripts even when the transport has no local VAD. GPT-4.1-mini handles intake and wrap-up.
Start the Copilot session while the call is being connected so provider and worker startup finishes before the caller's first statement. latency-warm sends six seconds of paced silence before the test phrases; it records both clip-end and acoustically detected speech-end timings. This models an already connected Copilot, not a guaranteed production startup time.
The facts worker uses tool_result_bindings to project fields from a single tool result directly into its response schema. Missing fields become null; invalid or ambiguous results emit nothing. This bypasses the model finalizer, preserves exact amounts and qualifications, and deduplicates unchanged facts. The tool still enforces trusted verification.
The trusted CRM input claim_verified defaults to false. Spoken assertions do not change it. With a verified fixture, the claim-facts tool returns an estimated gross benefit of $312,500, including the $27,500 rider enhancement. The enhancement is never added twice. The initial claim and checklist services require verification and explicit representative confirmation; the listening workers do not execute those writes.
The required demo documents are a certified death certificate, completed claimant statement and government-issued identification. Tax and payment items are conditional. Approximately five business days after a complete packet is a synthetic review estimate, not an Allianz service commitment.
Speech clarity coaching¶
The separate communication-clarity worker uses the existing CoachingCard.
It shows a low-urgency reminder for an explicit customer request for slower or
clearer speech, or independently from the agent's transcript. One clearly
garbled final agent segment is sufficient, including essential missing words,
implausible substitutions or broken syntax that obscures meaning. No customer
comment or earlier agent segment is required. Borderline wording needs
corroboration in the eight-turn context window; ordinary understandable grammar
mistakes and clear corrections produce no new warning. The newest turn must
supply the evidence. The demo does not set a two-agent-segment runtime threshold.
Approved whole-utterance requests, such as “Could you slow down please?”, are
restricted to the customer channel and bypass the model. Longer requests and
agent wording use GPT-4.1-mini with structured applicability output.
clarity-coaching.json owns the prompt, thresholds and approved cards; the
installer and focused draft refinement share this definition.
The card has card_ttl_seconds: 15. A new qualifying turn refreshes the same
surface, even if the advice is identical. Unrelated or inapplicable dialogue
does not change its deadline. Expiry sends native deleteSurface; session end
cancels the timer. Empathy, claims knowledge and intake retain their own surfaces.
To demonstrate this beat, let Elena ask Jordan to slow down during the checklist. The card suggests: “Pause briefly. Use short sentences and check the caller can follow.” Jordan acknowledges the request and explains one step. Wait fifteen seconds to show the reminder disappearing. Repeat the customer request before expiry to show renewal without stacking another card.
For controlled transcript replay, use the positive and negative examples in
clarity-cases.json. A single unintelligible agent passage produces “Pause and
check understanding,” with a rationale that transcription may be the cause.
For an agent-only replay, send “Please the signed to and we will after it.”
as the first final agent transcript. The missing words should trigger the card
without any customer input. Ordinary fragments, understandable nonstandard grammar, minor mistakes,
customer grammar, clear corrections and resolved concerns should produce no
new card. These are supportive conversation cues, not measured speaking-rate,
pronunciation or employee-performance assessments. No nationality or accent
inference is used. Audio quality and STT errors can create the same text signal.
Scenario 2¶
Taylor runs gpt-live-1 with the Marin voice and managed Responses delegation to gpt-6-sol, with medium reasoning effort and a 6,000-token output limit. The voice layer handles turn-taking; the backend owns authentication, protected tools, workflow results and UI actions. The native managed-delegation test completed successfully. Earlier client-delegation tests produced speech and tool results but failed to finalize evaluation evidence; that configuration is not selected for this demo.
Native Customer Authentication must verify the caller before producer tools become available. The policy fixture also validates producer PR-004781, Chen Financial Group and policy VA-5830194.
In voice Preview, use the phone emulator's notification delivery mode and a caller number. Authentication creates a real conversation-bound link and delivers it to that phone tray. Opening it runs the native passkey ceremony; first-time callers save a passkey and then verify with it. Carrier SMS is a separate delivery path. The saved voice evaluation cases simulate authentication and therefore do not test link creation or the browser ceremony.
| Policy item | Synthetic value |
|---|---|
| Cash value | $184,320.14 |
| Surrender charge | $2,764.80 |
| Difference before tax and other adjustments | $181,555.34 |
| Displayed rate | 4.25% |
| Valuation date | September 28, 2026 |
| Commission | $1,248.60, displayed cycle October 15, 2026, subject to final processing |
The voice agent starts an illustration only when requested. It confirms the current premium schedule and no additional withdrawals, starts native supervisor transcript streaming, and invokes the review workflow. The producer can inspect or change assumptions using the IllustrationRequest component. UI confirmation is caller input; it is not specialist approval.
The human opens the Salesforce request and its connected supervisor chat. Both
that embed and the transcript relay resolve the same thread from agent 4445
and the call's related_conversation_id. The injection tool binds its destination
to that variable and forces wait_for_reply: false. Questions, explicit approvals,
rejections and completed-action updates return through chat injection. Caller
answers return through the ongoing transcript, without polling a workflow.
Workflow 1847 now contains only START → push_event_to_salesforce → END in
async mode, using published workflow 1324's platform_event.publish operation
on the existing SFDC DevSDO Action. It publishes HITL_Escalation__e with the
reason, related conversation and JSON-serialized record properties. The record
uses Allianz_Support in DevSDO, whose active Salesforce flow routes to Erik's
Allianz specialist queue. See the Connect and console deployment
for the number, record pages, presence status and verification fields. The
workflow has no business outputs, wait, Human Review, submission or injection
node. Its initial execution ID means queued; it does not prove a Case was created.
Taylor answers value/rate follow-ups before returning to illustration assumptions.
The voice instructions specify full dollars and cents, one combined assumption
confirmation, and a spoken recap after the final requested task. The recap covers
retrieved policy values, the actual illustration status, commission qualifications
and a successfully recorded note. Internal end_session.params.summary does not
count as the spoken recap. An explicit request to stop immediately takes precedence.
Approval alone means approved, with submission unconfirmed. The human specialist owns the actual submission and explicitly reports completion and its reference in chat. For the synthetic successful script, the human can say: “I submitted the illustration request using the current premium schedule and no additional withdrawals, reference IFR-291640. The document has not yet been generated.” Taylor attributes that status to the specialist. The example reference is never inferred from a queued workflow or approval alone. Pending/rejected calls close truthfully without a manufactured receipt.
The earlier /allianz-demo review desk was the wrong pattern for this capability.
It is now an instruction page; its authenticated /api/allianz-demo/reviews
endpoint returns HTTP 410 so stale tabs cannot resume legacy demo runs. Historical
pending runs and the old isolated submission fixture remain available as history;
the corrected workflow does not call them. No approval buttons or custom workflow
approval endpoint are required for this demonstration.
September 29 supervision correction and checks¶
refine-supervision.ts updates only 1847, 4445 and 4444, using native
revision checks, private snapshots and collaboration refresh. Optional target
IDs limit a later refinement. It preserves authentication, voice model, branding,
workflow 1324 and Copilot 4446. Publish the supporting workflow and supervisor
through run.ts publish-support so fresh voice drafts resolve the corrected
published dependencies. Existing calls/runs retain their original snapshots.
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/refine-supervision.ts
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/workflow-test.ts
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/supervision-model-test.ts
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/supervision-injection-preview-test.ts
The graph checks cover escaped reason text, optional ANI, changed withdrawals, completion without a wait and transport failure. They send no Salesforce event. The model checks use the saved supervisor and voice-backend prompts to distinguish approval, rejection and explicit completed submission, plus silent transcripts, caller role claims and ended calls. They isolate injection and do not establish live audio or Salesforce routing. Preserve the initial caller-role failure and its corrected rerun as separate evidence. The native support-wiring and silence gate regression suite also runs independently.
Final correction checks: 5 native graph cases, 10 real-model probes,
31 native support/prompt/silence-gate tests, and 12 focused TypeScript
checks passed. The skill validator and regenerated bundle parity passed.
Browser checks confirmed HTTP 410 on the retired review endpoint, no approval
buttons, working catalog actions/brand assets and no page errors. TypeScript
and lint had no errors; existing script typing/static-image advisories remain.
Supporting workflow 1847:v10 and supervisor 4445:v10 are published; the
voice draft revision is 2026-09-29T19:43:49.234Z.
A fresh native supervisor preview (20e3458d-0c7f-4326-a827-4c9507577523) stayed
silent during transcript monitoring and answered the human: 731 ms to first
text and 1,549 ms to completion, one sample. A separate connected GPT Live
preview resolved the same supervisor thread twice, delivered one chat injection
to the bound source call with wait_for_reply: false, and spoke the requested
connection-check question. These checks do not publish a Salesforce event;
record creation, queue routing and the full illustration journey through that
Salesforce workspace still require a connected rehearsal. The source workflow
1324 and Copilot 4446 were not changed by this correction.
Components and branding¶
The source catalogs live in allianz-producer and allianz-assist. The voice prompt includes native COMPONENT tokens so the backend receives the component schemas and owning catalog IDs.
The surface uses Allianz blue #003781, secondary blue #006192, a white logo header and Allianz Neo regular/bold. The SVG wordmark and WOFF2 assets are copies of the assets served by Allianz Life, stored under each application's public/assets/allianz directory. These are demonstration brand assets, not a newly designed logo.
Install and run checks¶
To add or refine only clarity coaching on the existing local Copilot draft:
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/refine-copilot.ts
bash .devcontainer/scripts/with-env.sh apps/ai-orchestrator/.venv/bin/python apps/ai-manager/scripts/allianz-demo/clarity-eval.py
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/clarity-preview-test.ts
apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/clarity-browser-test.ts
The refinement backs up the current draft, checks its revision, validates through
the native management save and resets its collaboration room. It preserves other
workers and settings. Start a new preview to load the updated draft and runtime.
The evaluation makes real model calls against controlled final text. The preview
test sends paced synthetic stereo audio through real STT and checks speaker
filtering, unchanged advice renewal, expiry and redisplay. Reports are private
under .local-data/allianz-demo/; neither test sends a carrier message.
September 29 clarity validation: 170 focused runtime tests passed, including native configuration validation and a regression proving that the first agent segment reaches the classifier without any customer input. All 30 controlled transcript cases passed after allowing one clearly garbled agent segment. These include missing essential words, implausible substitutions, minor grammar mistakes, unfinished phrases and clear corrections. Model evaluations took 733 ms median / 1,297 ms p95 from final text to card-or-skip, excluding STT and browser rendering. The two approved-phrase cases were excluded from those model statistics.
The earlier successful stereo preview (50a50d1e-8734-4cd4-9d53-8f88347674da) measured
418, 399 and 410 ms from detected speech end to publication for three
approved customer requests. It checked the representative-channel negative
case, timer preservation during an ordinary response, renewal by identical
advice, expiry after 15,001 ms and redisplay after expiry. Chromium replay
through the SDK, native A2UI renderer and catalog showed card counts
1 → 1 → 0 → 1, with no page errors. This verifies browser event handling,
not end-to-end browser latency.
Earlier failed runs remain in private evidence: the initial classifier over-triggered on isolated or resolved issues, and the first live worker was skipped because a zero debounce failed the runtime's 0.5-second minimum. The final configuration uses 0.5 seconds; debounce limits repeat triggers rather than introducing a fixed delay before each eligible extraction.
Run from the repository root with the normal local stack available. Existing OpenAI, Deepgram, messaging integrations and the native copilot-cards catalog must be configured. The installer carries integration references, not provider credentials.
python apps/ai-manager/scripts/allianz-demo/catalogs.py
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/install.ts
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/run.ts publish-support
catalogs.py prepares source workspaces. Publish those catalogs through Catalog Studio’s native validation and publish flow; catalog source compilation and publication are separate from installing agent drafts.
To apply the September 28 rehearsal wording to an existing local demo draft,
run refine-voice.ts with the same TypeScript command. It preserves the current
authentication, SMS, model and catalog settings, saves a private prior-config
snapshot, and updates only the backend/voice instructions and the relevant saved
tests. After committing, it refreshes the live editor's collaborative document
from the saved draft so an older editor snapshot cannot restore the old prompt.
The installer also includes those instructions and refreshes its affected rooms.
# Native saved voice conversations, including authentication and a supervisor participant
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/run.ts start
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/run.ts export
# Real speech recognition, Copilot cards and a synthetic trusted-CRM variant
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/preview-test.ts copilot
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/preview-test.ts copilot verified
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/preview-test.ts copilot latency
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/preview-test.ts copilot latency-warm
# Supervisor response timing; isolated Salesforce graph; authenticated Chromium UI
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/preview-test.ts supervisor
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/workflow-test.ts
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/browser-test.ts
# Real voice consent, auth link, emulator notification and browser passkey
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/auth-preview-test.ts
# Continue through values, assumptions, commission, note and pending-review spoken recap
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/auth-preview-test.ts wrapup
# Request the recap and hangup with the note, before the agent has given a recap
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/auth-preview-test.ts wrapup-close
# Historical September 28 review-workflow evidence only (not the corrected flow)
bash .devcontainer/scripts/with-env.sh apps/ai-manager/node_modules/.bin/tsx --tsconfig apps/ai-manager/tsconfig.json apps/ai-manager/scripts/allianz-demo/verify-wrapup.ts --legacy-review-workflow CONVERSATION_UUID
The prior saved voice suite used a wait-node-driven supervisor event. Those historical results do not certify the corrected single-action pattern: its simulation driver still needs an escalation-action event to schedule the human participant. Use the focused checks below and the connected Salesforce rehearsal until that evaluation adapter is extended. The graph check compiles the real LangGraph and executes the native tool-node renderer with a mocked Salesforce transport. Browser checks use a normal local test account, verify the retired endpoint and render the actual catalog React code.
auth-preview-test.ts uses the current 4444:draft without changing its configuration.
It speaks through GPT Live, asserts that nothing is delivered before permission,
then exercises the real auth API, share link and passkey UI. Delivery is explicitly
demo_notification, and Chromium supplies a virtual authenticator. It creates only
synthetic test identity/challenge records and never sends a carrier SMS.
Scenario steps wait for audible playout to finish; a final transcript fragment
alone is not the end of a Live response. The script writes its outcome before
transport teardown so a peer-ended stream cannot hide a failed assertion.
September 28 Preview authentication incident¶
Calls ce7ed3e8-1242-49b0-87a1-e122a3761945 and
7406d36a-a09f-4bca-9be5-ce40791388e3 failed before any challenge or message
was created. Taylor asked permission and the caller answered “Sure, go ahead,”
but the subsequent acknowledgement replaced the question in the consent
checker’s context. The tool returned message_consent_required, then
message_consent_evidence_unavailable. Changing the SMS integration could not
address that failure.
The publisher now retains the question present at caller speech start with the
final reply. Validation: 59 focused runtime tests passed. Live retest
f437a776-0948-47ce-85db-805126b70052 produced one Preview notification after
consent, opened its real share link, enrolled a Chromium virtual passkey, and
verified it successfully against the auth service. The current agent draft was
not modified. Carrier delivery and physical-device biometrics were not tested.
The earlier retest e337a329-4a5d-4c96-a15e-633f8d96bd97 ended the call before
authentication and produced no link; its failed result remains in the private
evidence directory. The successful focused retest establishes the repaired
consent/link/ceremony path, not a success-rate estimate for all voice conversations.
September 28 wrap-up rehearsal¶
Interactive call e26b8ebf-72fb-4263-89e4-ecd93c541034 completed authentication,
lookups, commission and a note, but repeatedly returned to the withdrawal
question and used abbreviated dollar pronunciation. The closing recap existed
only in end_session.params.summary; the caller heard “Okay.” Its illustration
review remained pending. No human had approved it and no submission reference
was available, so that part could not use the document's successful closing.
The revised business and voice prompts require full monetary pronunciation, preserve confirmed assumptions across follow-ups, interpret the commission-note correction in context, and give an audible recap with the actual workflow status. The saved complete-journey test now requires those spoken details; an additional follow-up case exercises percentage/estimate questions and the corrected note. The final recap targets 100–130 spoken words. It omits the repeated after-charge calculation and separate service-note reference while retaining the required policy, illustration, commission and note facts.
The editor's older collaborative document overwrote one saved refinement at
19:34 UTC, restoring the original prompt and low reasoning setting. The scripts
now use the native room reset after the database transaction commits. Repeated
GPT-6 Luna runs still showed omitted closing details and, in one run, a provider
Responses handoff incomplete error. The selected backend was changed to GPT-6
Sol with medium reasoning and more output headroom. The higher token limit is
not a diagnosis of that provider error: the recorded usage did not establish
token exhaustion.
The first follow-up regression, 933b2efa-a30a-4c92-94f7-12bcab59d201,
exposed a separate runtime cutoff: the thirty-second end-session fallback
disconnected a recap while Live was still speaking it. The handler now extends
the deadline only while closing audio is arriving, with an absolute two-minute
limit and the original bounded wait for a stalled backend. Native closing-turn
completion ends the wait immediately. 189 focused Python tests passed,
including long speech, a mid-recap stall and a missing completion event.
The actual voice/Preview/browser run
b1d2f523-7404-40e8-a96e-fc58c620ade8 passed authentication with a virtual
passkey and continued through an audible pending-review recap. It spoke the
cash value, surrender charge, rate, current premium/no-withdrawal assumptions,
commission and October 15 qualification. It accurately said no submission
reference existed, recorded one note and ended once without submitting.
This run used the first wording refinement, before the runtime cutoff fix.
The selected Sol configuration passed the native complete journey
(798686a1-e538-4d3c-8212-313f79fff934, review score 92%) and the follow-up/corrected
note journey (96215572-0405-4da2-90b1-743e1224e3d1, 100%). Both passed the
independent verify-wrapup.ts assertions: audible policy/illustration/commission
facts, a recorded note, one end action, and durable approval completion before
the submit node started. The complete case retained a minor timing defect: a
queued no-reference-yet statement played just before the submitted confirmation.
The factual final recap was correct. The report retains earlier failures.
In the real pending-review rehearsal
6c59b63d-db11-4b8d-a5c7-9140cb63d9bd, Sol spoke the requested facts but omitted
the end tool when asked to recap and finish in one request. The combined-request
instructions now explicitly order the note, end_session, then the spoken recap.
The tool schedules termination while the runtime waits for the final speech.
This avoids relying on a second backend turn after its final text.
The final real voice/browser test
7f2d1c96-e1e7-427d-a159-9d6afdd41282 passed with this ordering. It created
one Preview auth notification, enrolled and verified a virtual passkey, saved one
note, spoke the complete pending-review recap and farewell, and called
end_session once without submitting an illustration. The last spoken transcript
arrived 49,011 ms after the end-tool request, exercising the repaired
long-closing deadline. No carrier SMS was sent. The draft and live collaborative
document retained GPT-6 Sol, medium reasoning, the 6,000-token limit and the final
instructions.
The Sol runs recorded median delegation-to-first-completed-text-item times of 8.8 seconds (eight items) and 9.3 seconds (twelve items). These include the backend's tools and processing, and are not speech-end-to-first-audio latency or time to first token; Live acknowledgments can be earlier. They are small, different conversations, not a controlled model benchmark. The Copilot's separate sub-second approved-cue measurements are unchanged.
Keep evidence in .local-data/allianz-demo/. It includes all failed attempts as well as later runs. Distinguish server card-publication time, speech-recognition time, first chat token, browser acknowledgement and workflow completion. None measures a production PSTN/CCaaS path. A one-second result for an approved cue does not establish a one-second SLO for all model responses.
Run verify-evidence.ts with the same TypeScript command to assert the latest verified facts, hidden unverified data, retained intake values, all twelve phrase matches and silent supervisor monitoring. It writes preview-validation.json; latency thresholds are reported separately from content correctness.