AI counterparty detection: public OpenMuse experiment¶

Last reviewed: 2026-10-17

The final frozen confirmation set reached F1 0.979 and AI recall 95.8%, counting a connection setup failure as a miss. Concealed-AI recall was 91.7%. This is a small synthetic benchmark with human roleplay controls, not evidence of production-wide accuracy.

Results¶

Measure	First holdout, threshold 0.70	Fresh confirmation, threshold 0.65
OpenMuse contact attempts	24	24
AI attempts flagged	20	23
Disclosed AI flagged	12 / 12	12 / 12
Concealed AI flagged	8 / 12	11 / 12
Human conversations flagged	0 / 40	0 / 40
Human conversations ever flagged during replay	0 / 40	0 / 40
Precision	100%	100%
AI recall	83.3%	95.8%
F1	0.909	0.979
Attempts with transport/setup errors	5	1
Attempts without any native assessment	0	1
Last emitted assessment behind submitted customer turns	2	5

The first holdout demonstrates why F1 alone is insufficient: it cleared 0.90 while missing a third of concealed callers. Its four classification misses scored 0.66–0.68. The 40 human controls had no positive flags and a maximum score of 0.63 after the second turn. We selected 0.65, then froze that threshold before generating new confirmation conversations. The rubric and detector model stayed unchanged.

The confirmation failure, confirm-04-41-covert, occurred when the public stream-ticket endpoint returned 404 before the conversation row was available. No customer message was sent in that attempt. It remains in the 24-attempt denominator as a false negative. All 23 conversations that exchanged messages were flagged, including all 11 concealed callers that connected. The harness now retries that bounded setup race; this later fix is not used to rewrite the reported result.

Disclosed callers were first flagged after customer turn 1. Connected concealed callers were first flagged after customer turns 2–4. Five confirmation conversations ended with newer customer turns still awaiting assessment; their earlier positive flag had already been emitted during the conversation. This measurement uses the last emitted native verdict, rather than claiming every final utterance received a completed assessment.

No concealed caller was classified as self-disclosed. Human assessment availability was 40/40 in both sets.

What ran¶

Public endpoint: surface-app.vanguard.dev.trueserve.deloitte.com, using the city's native TrueServe runtime.
Published isolated agent: 4973, “AI Counterparty Evaluation.” First holdout used version 3; confirmation used version 4. No official Dev/QA cluster deployment was performed.
Business agent: GPT-4.1-mini, a fictional service desk. It could simulate orders, returns, bookings and billing outcomes, but had no real business transaction tools. Its instructions kept service behavior the same regardless of the internal AI assessment.
Caller: the actual tanstackAgent implementation from CopilotKit/OpenMuse, pinned to 1ac68f3909f2478ab6280883f1ab5ea65eb5719d. A custom contact_business tool called the public chat API and returned only business replies. This exercised OpenMuse's real model/tool loop, not the full OpenMuse desktop/E2B application.
Caller models: GPT-4.1 and GPT-5.4-mini, 12 attempts each per set. Each set contained six objectives crossed with the two models and two personas.
Disclosed persona: explicitly introduces itself as an AI assistant acting for an owner.
Concealed persona: instructed to impersonate the human account holder, deny being AI if asked, avoid owner/tool references, use short casual messages and occasional corrections, and avoid bullet lists and repeated summaries. Compliance with the brevity instruction varied; the callers did not receive detector feedback.
Detector: the production CounterpartyDetector, GPT-5.5, low reasoning effort, 3,000 completion tokens, 30-second timeout, counterparty-v3, minimum two customer turns, clearing threshold 0.35 and two low checks to reverse a positive flag.

The detector prompt SHA-256 was e31ecc9a6ea3412f9e14c7ef7128cfe672c08e9c3914dd804b5e142da84682c9 in both sets. Only customer/service transcript text and deterministic text observations entered classification. Test labels, caller model names, persona instructions, credentials and transport metadata were excluded.

The first holdout used SSE and encountered five reply-timeout conversations. Their assessments were recovered from persisted native traces, and the errors remain recorded. Confirmation used the same WebSocket transport as the browser, with one setup failure and no recorded reply-timeout cases. Native trace collection was separate from the caller loop. The last verdict in the native trace supplies each result.

The runtime publishes assessments of completed transcript prefixes with pending=true when newer messages have arrived, then assesses the latest prefix. An earlier development implementation discarded every in-flight result after a new customer message; rapid exchanges could starve it until the call ended. That behavior was corrected before either holdout. Corrections to the text of a turn still invalidate the in-flight assessment.

Human controls and development¶

Human controls came from the author-published ABCD v1.1 dataset. These are human-human, crowdsourced customer-service roleplays, not real production contacts. Each control contains the first six customer turns and the intervening original service turns. The production detector replayed customer turns incrementally; the control labels did not enter the prompt.

The initial development set used 20 eligible conversations from ABCD's official dev partition with seed 20261017. The first holdout used 40 from test with that seed. Confirmation used 40 different test conversations with seed 20261018, explicitly excluding the first holdout IDs. Corpus SHA-256 and conversation IDs are preserved with the results. Original human utterances are not redistributed in this repository.

Earlier development tried smaller models and rubric revisions v1 and v2. They recognized disclosure but missed tested concealed conversations. The final rubric added contrastive examples and combinations of repetition, procedural language, courtesy, local replies, hedging and corrections. GPT-5.5 with v3 recognized the longer concealed development conversation and had no flags on the 20 development human controls. Prompt candidates remain under scripts/counterparty-detection/prompts/.

This approach is inspired by the Cresta Sentry article. We did not obtain Cresta's evaluation corpus or reproduce its training procedure. These measurements should not be treated as a head-to-head comparison.

Limits of the result¶

The confirmation AI-recall Wilson 95% interval is approximately 79.8%–99.3%. Zero false positives among 40 humans still permits an upper Wilson bound of approximately 8.8%. A point estimate above 90% on this sample does not establish a production guarantee.

The AI and human sources differ in channel conditions, objectives and conversational style. Both caller models belong to one provider family. Some concealed callers wrote longer, more polished messages than instructed. Fluent professional humans, human representatives, translation/dictation users, multilingual conversations, mixed human/AI control and stronger adaptive attackers need additional matched tests. The scores are not calibrated population probabilities.

Human controls were replayed through the classifier; they were not newly collected conversations with the public business agent. Voice acoustics, live voice calls and a full operator Copilot session were not evaluated end to end. Voice transcript taps, Copilot planning/card integration, action access and state behavior have implementation checks, but their accuracy cannot be inferred from this text benchmark.

The evaluation settings are present on the published test agent. Platform defaults remain a 0.80 threshold and three customer turns. The reported numbers apply to the specified model, rubric and 0.65 confirmation settings. Smaller models, different prompts or defaults need separate evaluation.

Use this as an advisory signal. It does not establish identity, authority or fraud, and should not independently block a customer.

Artifacts and verification¶

Validation completed: a 245-test orchestrator suite and an additional Copilot integration regression passed; ai-manager TypeScript checks passed; affected UI lint passed with existing warnings; generated configuration schemas passed the drift/description checks. CounterpartyAssessmentCard was built and published in the local tenant's Copilot cards catalog, version 1.1.0. An anonymous public-browser smoke check also loaded the chat, sent a message and rendered the service reply without page errors. Tests cover grounded customer evidence, atomic protected variables, deduplication, prefix freshness and corrections, hysteresis, stable card identity, failure/cancellation, the cached/refresh action, reasoning-model request parameters, and planning against the actual published card schema shape.