AI counterparty detection¶

Status: Experimental | Last reviewed: 2026-10-19

Voice and Chat agents can continuously estimate whether the customer is an autonomous AI assistant. Copilots have a separate continuous monitor before transcript enhancement and the actionability gate. The assessment is advisory: it establishes neither identity nor authority, and does not imply fraud.

Authoring¶

On a native Voice/Chat model node, open Capabilities → Add capability → AI assistant detection. Its stored reference is builtin:counterparty_detection; configuration lives in config.builtin_tool_configs.counterparty_detection. It uses the owning agent's Logic SLM auxiliary model and tenant-scoped connection. Enabling it does not change business routing or verification rules.

Voice sessions must produce finalized customer transcripts. Enable input transcription on native audio models that do not emit it by default; without customer text, the result remains unavailable/unknown. The benchmark below exercises text chat, not acoustic detection.

See Guided channel selection for the proposed shared channel policy, in-queue declarations, separate synthetic-voice signal, and OpenMuse voice adapter. Those extensions are not part of the current detector; its evaluation harness remains chat-only.

On the Copilot canvas, select AI assistant detection, enable monitoring, choose a model connection, and select a published CounterpartyAssessmentCard. Configuration lives in copilot.counterparty_detection. The base copilot-cards catalog includes that component. Existing installations must publish the updated catalog before selecting it; the source seeder only repairs missing publications.

The Copilot monitor exposes OpenAI and AWS Bedrock connections supported by its auxiliary client. Its completion budget is 3,000 tokens; OpenAI reasoning models use low effort. OpenAI is the provider exercised by the live benchmark.

The monitor runs once per conversation, independently of gated suggestion workers. It reads original final customer transcripts; service turns provide context only. A stable copilot:counterparty-detection surface updates in place after the first positive result, including when the assessment later clears or becomes unavailable.

Runtime contract¶

Variable	Meaning
`counterparty_ai_likelihood`	Model score in [0,1], null before assessment. Not a calibrated population probability.
`counterparty_ai_detected`	Boolean with hysteresis. False does not verify a human.
`counterparty_ai_status`	`unknown`, `likely_human`, or `likely_ai`.
`counterparty_ai_available`	Whether the latest assessment attempt succeeded.
`counterparty_ai_pending`	New customer evidence awaits assessment; the previous score can be stale.
`counterparty_ai_assessed_through`	Last customer turn included in the successful assessment.
`counterparty_ai_updated_at`	UTC time of the last successful assessment.
`counterparty_ai_disclosure`	`none` or `self_disclosed`; a customer claim, not verified provenance.

These variables are capability-owned. Mutable or linked declarations cannot redirect their values. Updates commit atomically before notifying subscribers. Voice/Chat templated prompts refresh through the existing instruction refresher. Copilot prompt bindings are evaluated at worker invocation, rather than frozen at build time.

Voice/Chat models also receive get_counterparty_assessment({refresh?: boolean}). The default returns cached state immediately, with evidence, freshness, prompt version and SHA-256. refresh: true waits a bounded time for pending evidence. Copilot workers use the live variables; their monitor independently publishes the card.

A prompt may use:

Internal assessment: {{counterparty_ai_status}}.
Pending: {{counterparty_ai_pending}}.
{{#when counterparty_ai_detected}}
Use the organization's approved policy for personal assistants.
Continue normal verification and authorization checks.
{{/when}}

The default threshold is 0.8 to set the flag. Two successive assessments at or below 0.35 clear it. Behavioral decisions require three customer turns; an explicit first-person AI disclosure can trigger sooner. Ambiguous results preserve an existing positive flag. Configuration changes apply to new sessions.

Each completed customer turn schedules a debounced background assessment. Only one classification runs at a time. A completed assessment of an earlier prefix is published with pending=true when newer customer turns have arrived; the next request coalesces the latest context. This prevents starvation during rapid exchanges. Corrections to evidence already being assessed invalidate that result. Repeated delivery of the same turn is ignored. Text chat is observed at typed input, avoiding its synthetic STT duplicate. Repeated text in genuinely different turns is retained.

The context retains four opening turns and a rolling tail, bounded to 32 turns by default and 3,000 characters per turn. The classifier sees transcript evidence and deterministic observations; test labels, caller model names, transport metadata and credentials never enter its prompt. The rubric considers disclosure, repetition, request stacking, hedging, corrections and conversational structure, including counter-explanations and intentional concealment. Acoustic spoofing, voice timing and cryptographically attested agent identity are outside this implementation.

Quotes must occur in a customer turn. Invalid secondary quotes are omitted and counted; an unsupported decisive result or ungrounded disclosure is rejected. Model errors preserve the last score and mark availability false. Cancellation stops the observer. Internal results use PIPELINE_EVENT with frame_type=CounterpartyAssessment, rather than customer-visible chat events; the Copilot card is an explicit operator surface. Auxiliary usage is metered under counterparty_detection.

Models and evaluation¶

The live evaluation uses GPT-5.5 with low reasoning effort, a 3,000-token completion budget and a 30-second assessment timeout. The first holdout used a 0.7 threshold and two required customer turns; its results informed a 0.65 threshold for a separate confirmation set. The platform defaults remain 0.8 and three turns. These are evaluation settings, not silently imposed tenant defaults. Smaller models need separate validation. GPT-5-family auxiliary requests use max_completion_tokens and omit unsupported sampling fields.

The default rubric is counterparty-v3. The development experiment preserves earlier rubric candidates under scripts/counterparty-detection/prompts/. Custom rubrics get a content-derived version and SHA-256 in the result and usage metadata. They cannot override the fixed boundary that treats customer text as evidence rather than instructions.

See the evaluation report for observed results and limitations. F1 is 2TP / (2TP + FP + FN); AI recall is TP / (TP + FN). Report disclosed and concealed callers separately. A high combined F1 can hide poor concealed-agent recall. Human false positives must come from human-authored conversations, not LLM-generated people.

Reproducing the experiment¶

The harness uses the actual OpenMuse tanstackAgent implementation with a contact_business tool. That tool sends messages over the public TrueServe HTTPS surface and returns only the business reply. It does not return assessments or provenance. This exercises the OpenMuse agent loop and the normal public chat transport; it does not exercise OpenMuse's E2B desktop UI.

Clone and pin CopilotKit/openmuse at 1ac68f3909f2478ab6280883f1ab5ea65eb5719d, and install its locked dependencies.
Run apps/ai-manager/scripts/counterparty-eval/provision.ts through the environment wrapper and ai-manager's tsx --tsconfig apps/ai-manager/tsconfig.json. It uses an expiring tenant token, deletes that token on completion, and writes session state to a private local file. COUNTERPARTY_CONFIG supplies a complete agent config for validation and publication; COUNTERPARTY_MINT=1 mints a single-use embed token. COUNTERPARTY_SHARE=1 creates an expiring anonymous test-chat link.
Exchange the embed token through the public surface's /api/session/exchange, retaining the cookie only in a private local file. Copy scripts/counterparty-detection/openmuse-runner.ts to the pinned OpenMuse checkout as counterparty-runner.ts so it uses upstream dependencies.
Set TEST_SCENARIOS, TEST_OUTPUT, TEST_SURFACE_URL, TEST_STATE and TEST_COOKIE. Run the copied script through with-model-integration.ts, which passes the tenant's model key only in the child environment. SDK telemetry is disabled. The harness limits each caller to six business messages and bounds concurrency.
Download the author-published ABCD v1.1 corpus separately. prepare-human-controls.py selects disjoint, seeded development and test conversations. evaluate.py replays only original speaker-labeled turns through the production detector. ABCD consists of crowdsourced human-human roleplay, not production customer traffic; it is a useful control with a distribution mismatch.
Freeze the rubric, model and thresholds before running the holdout scenarios. Do not edit watched runtime files during a live batch: the dev auto-reloader terminates active chats. Keep setup failures and incomplete assessments visible rather than quietly removing them from the denominator.

All test accounts, orders and commitments are fictional. The public link grants access only to its test conversation and expires; no management token or model key is exposed to the caller.