Analytics Agents

Status: Current | Last reviewed: 2026-10-07

Analytics Agents use Studio identity, SSO, tenant switching, publication, encrypted integrations and PostgreSQL. The canvas owns chat behavior; immutable dataset releases own preparation. See the CCIQ assessment and user guide. The dataset input contract covers table imports, version carry-forward and the intent tree editor.

Analytics uses the shared Agents repository, filtered with /agents?type=analytics_agents. Its top-level sidebar item is retired. Cards open /agents/{id}/analytics?preview=1; builders can use the separate Edit action. &draft=1 requests a saved draft preview, with existing author checks. Without it, the preview resolves only a published agent available to the current tenant runner. AnalyticsPreview hosts one AnalyticsChat instance while resizing between a maximized surface and the editor-side panel (preview=panel), retaining history, current input and evidence. Copy link preserves mode, never another user's chat. Legacy /analytics?agent={id}&focus=1 redirects to the canonical preview; unselected /analytics redirects to the repository filter. Tenant switches remount previews.

Full analysis adds a scoped SQL/tool loop, operational warehouse snapshots, rich annotations, compiled report hierarchies, versioned memory pages and the ported CallSight composition. The same host serves Preview and the focused app. Built-in cards and selected published catalogues use the shared A2UI component registry; chat inspection actions remain nonterminal and independent of voice surfaces. The simpler metric/search mode remains available when full analysis is disabled.

Use the question and result parity contract for the acceptance scope. Successful requests, matching contact counts and a ported renderer do not alone establish question/result parity.

Enable and run

Apply migrations 068 through 072 through the normal forward migration process. The city migration runner includes preparation, immutable analysis snapshots, report jobs and briefing caches. Never reset a populated database for this feature.

FEATURE_ANALYTICS_AGENTS=true
ANALYTICS_WORKER_CONCURRENCY=4

The runtime flag defaults off and requires the parent Agents capability. It controls both the server and Studio's creation menus, filters and Home portfolio; disabled Analytics agents are omitted from lists/direct reads and their editor and API routes return 404. NEXT_PUBLIC_FEATURE_ANALYTICS_AGENTS is no longer used: build-time values cannot enable a feature in a destination environment. Restart ai-manager and the worker after changing their runtime flag; no frontend rebuild is needed. Scene agents use the separate FEATURE_SCENE_AGENTS opt-in. Worker concurrency is 1–16; the default is four leased jobs in parallel.

cd apps/ai-manager
npm run analytics:worker
# One job for an operational check:
npm run analytics:worker -- --once

The worker uses the normal database, encrypted-integration and assets-store environment (STORAGE_*, STORAGE_BUCKET_ASSETS). PostgreSQL needs pgvector. Set ANALYTICS_PYTHON to a Python environment with the shared libs/pii, libs/transcription, HTTP-client and provider dependencies. The dev launcher uses apps/async-agent/.venv/bin/python and installs the pinned dependencies in apps/ai-manager/analytics_runtime/requirements.txt when needed. Both the chat server and worker require this interpreter. The fixed bridge imports these shared libraries from this checkout; it never executes uploaded code.

Vanguard has the runtime flag and concurrency 16 enabled locally. The main Overmind analytics-worker service is running. Restart it after worker code changes; the worker does not hot reload. The chat server uses Next development reload. After changing shared configuration schemas, restart both ai-manager and analytics-worker: a stale worker can reject a briefing configuration queued by the newer server. Failed briefings show Retry briefing. That action retries only the current authorized failed cache entry; it preserves pending/ready work.

For container environments, build apps/ai-manager/Dockerfile with --target analytics-worker for the queue consumer and the normal runner for Studio. Both include Python 3.12, pinned runtime requirements and shared transcription/PII libraries. Set FEATURE_ANALYTICS_AGENTS=true at runtime in Studio and its Analytics worker. Leave it unset in other deployments to keep Analytics off. ANALYTICS_SHARED_ROOT points to packaged shared libraries; ANALYTICS_PYTHON points to the packaged interpreter. Supply the same DB, encryption, storage and integration environment to the worker. This change packages the services; it does not deploy other environments.

Prompts and model choices

The migration retains the original response-composition prompt and briefing addendum, with explicit adaptations for Studio. It does not retain every original model choice.

Stage	Original CCIQ	Vanguard Analytics configuration
Chat and briefing	Source default us.anthropic.claude-sonnet-4-20250514-v1:0	gpt-5.4, low reasoning; Analysis → Live analyst / Advanced reasoning effort
Native annotation	Imported v7 artifacts identify Sonnet 4.5 and convintel.annotation.v4.taxonomy.cciq.driver_taxonomy.v1	gpt-4.1-mini; Preparation → Annotations / Model / Full prompt
Native report and memory narrative	Source Bedrock compiler; imported artifact provenance is preserved	gpt-4.1-mini; Preparation → Prepared insights / Model / Full prompts
Transcript embeddings	Source default amazon.titan-embed-text-v2:0	text-embedding-3-small, 512 dimensions; Preparation → Semantic search / Model / Vector size

Imported reports and annotations are preserved outputs, not silently regenerated with these new models. Changing response settings takes effect after Save/Publish for new chats. Changing offline settings requires a new preparation/report build; an existing release keeps its models, dimensions and prompt provenance.

apps/ai-manager/analytics_runtime/system-prompt.txt retains the first 664 lines of the source prompt. analysis-chat.ts replaces the static AWS warehouse section with the authorized schema, domain vocabulary, exact comparison windows and metric conventions; it also adapts the introductory domain wording. analytics_runtime/briefing-prompt.txt is the original briefing addendum. The full analyst, briefing and review task prompts are editable in Analysis. GET /api/analytics/prompts serves the versioned defaults to authorized builders. Annotation defaults live in prompt-defaults.ts; the exact ported trend/memory templates live in analytics_runtime/{trend,memory}-prompt.txt. Preparation exposes those full templates plus separate trend and memory model bindings. Template placeholders are substituted from compiler data without evaluating code. Basic mode has independently configurable planner and grounded-answer models and prompts. Runtime scope, tool and output contracts remain mandatory.

Native report prompt/formatting functions are ported in analytics_runtime/compiler_format.py; dataset-wide memory synthesis is adapted in lib/analytics/report-compiler.ts. Annotation instructions in lib/analytics/compilers.ts are a Studio structured-output prompt implementing the richer annotation contract, not a verbatim copy of the source prompt.

Review answers against evidence adds one model iteration to check numbers, rankings, comparisons and causal wording. It is a quality/latency control, not a guarantee that every generated sentence is correct. Advanced composition settings expose budgets and tools. provider-client.ts adapts the shared Analytics loop to Bedrock Converse and Vertex GenerateContent; OpenAI uses the Responses SDK. Credentials resolve exclusively from the selected tenant integration. Tool-result history, Bedrock reasoning blocks and Gemini thought signatures survive provider turns. Structured tasks use a forced result tool; compiler outputs still undergo local Zod validation. Bedrock uses the SDK SigV4 signer for encoded model paths. Task models can differ for analyst, review, briefing, annotation, trend and memory synthesis.

Embeddings support OpenAI text-embedding-3, Bedrock Titan v2 and implemented Vertex text embedding families. model-config.ts is the adapter/size catalogue; profile validation and actual returned-vector checks enforce supported sizes. Vertex uses distinct RETRIEVAL_DOCUMENT and RETRIEVAL_QUERY tasks. Cloud embedding batches reserve one request per text; OpenAI reserves one batched request. Integration/model permissions remain prerequisites; selecting a model does not provision cloud access.

Preparation and serving

flowchart LR
  Upload[Bulk transcripts, workbook or audio] --> Objects[Encrypted Studio assets]
  Upload --> Manifest[Durable file manifest]
  Manifest --> Worker[Leased preparation worker]
  Objects --> Worker
  Worker --> STT[Shared transcription and PII]
  STT --> Turns[Private normalized turns]
  Turns --> Annotation[Annotation compiler]
  Annotation --> Embedding[Embedding compiler]
  Embedding --> Ready[Complete immutable release]
  Ready --> PG[PostgreSQL full-text and pgvector]
  Canvas[Studio canvas] --> Version[Agent version]
  Version --> Chat[Preview or focused Analytics app]
  Chat --> PG
  Chat --> Model[Tenant response model]
  Chat --> History[Private history and citations]

Each upload creates an idempotent manifest, then streams each file into the shared assets store. A random AES-256-GCM data key encrypts each source and is wrapped by Studio's managed encryption helper, including KMS when configured. Object keys contain tenant/release/file IDs, not original names. Sources have no public download API. Only authenticated, complete plaintext is handed to the fixed preprocessing bridge. Credentials travel through stdin and are not placed in command arguments, inherited cloud environment variables or provider logs.

A worker claims a file or conversation using short transactions, SKIP LOCKED, a lease token and heartbeats. Provider work happens outside claim transactions. File normalization is atomic inside its still-private release. Annotation and embedding batches checkpoint independently; retries preserve completed work. 429/5xx/connection failures and invalid annotation contracts retry with backoff, up to five job attempts. Permanent failures stop the release; Retry incomplete stages resumes it explicitly. Request budgets count attempts and never reset on retry. A new build is required to change the immutable profile or its budget.

Cancellation and checkpoint writes lock/recheck the release. A release becomes ready only after every file and compile item completes. Incomplete releases are never selected by chat. Existing sessions pin their release and published agent version. Rebuilds reuse retained encrypted source but clear normalization markers; copying a prior build's checkpoint would skip the new release's data.

Table	Contents
analytics_datasets	Tenant identity, external source ID and real/synthetic classification
analytics_releases	Immutable profile/input manifest, status, budget consumption and source digest
analytics_ingestion_files	Encrypted source reference, SHA-256, file lease and provenance
analytics_compile_items	Per-conversation annotation/embedding checkpoints and retries
analytics_conversations, analytics_turns	Prepared IDs, times, channels, roles, duration and full-text vectors
analytics_annotations	Model findings, evidence ordinals, model and profile hash
analytics_chunks	Prepared text chunks and model/dimension-specific pgvector vectors
analytics_sessions, analytics_answers	Private pinned sessions, applied filters, answers and citations
analytics_analysis_sets, analytics_relation_schemas, analytics_relation_rows	Immutable typed operational snapshots
analytics_artifact_bodies, analytics_session_state	Compiled report bodies and private tool/attachment history
analytics_report_jobs, analytics_briefings	Durable hierarchy jobs and scheduled briefing caches

New tables have forced tenant RLS and compound foreign keys. Studio's service role bypasses RLS, so every application query also has explicit tenant/release predicates. Workers establish tenant context before the shared secret resolver; a globally privileged database connection alone does not authorize credentials.

Configurable preparation

Setting	Effect
Canonical or Contact Lens JSON	Adapter for JSON/JSONL; XLSX and audio are detected separately
Transcription integration, language, diarization, role map	Shared Deepgram/AWS/Azure/Google adapters; unmapped speakers stay unknown
Rules, managed PII, or redacted-source attestation	Runs before searchable rows or model requests; tenant policy may tighten it
Annotation model, instructions and driver taxonomy	Evidence-grounded summaries, primary driver, sentiment and outcome
Embedding model, dimensions, chunk size and overlap	Covers every indexed turn, including the tail of long turns
Dictionary and speaker roles	English/simple full-text dictionary and retrieval scope
Request budget and source retention	Bounds provider requests; retained encrypted sources enable rebuilds

Rules detect structured identifiers such as email, phone and payment data; managed detection supports names and addresses. Audio cannot use attestation. Managed detection uses overlapping windows and unions overlapping spans so an identifier crossing a window or turn boundary is not partially retained. Integrations must enable the selected speech-to-text, PII or model capability. A required self-hosted tenant PII detector is currently unsupported and blocks ingestion instead of falling back. Raw source is encrypted transient input; warehouse/model boundaries receive prepared text. With retention off, original source references are removed after normalization; shared retained rebuild references prevent premature object deletion. There is no source-purge scheduler or user-facing dataset deletion yet.

Uploads produce immutable complete snapshots. A new manifest may include sourceReleaseId and removeFileIds: retained encrypted source references are copied into the new release, omissions are removed there, and uploaded files are added. Ownership, matching dataset, terminal source state, retained objects, unique filenames and combined limits are checked before creation. Old snapshots are untouched. At most 10,000 files, 100 MiB/file, 2 GiB/batch and ten active batches/tenant. One parsed file allows 20,000 conversations and 200,000 turns; a conversation allows 2,000 turns and each turn 16,000 characters. XLSX expansion is bounded. Uploads can resume after reselecting the same files; completed files are skipped. Concurrent writes to one file fail with 409. Source inventory/checkpoint changes are not silently merged.

The legacy /datasets POST still accepts its original 5 MiB pre-redacted pilot format. It cannot bypass a tenant policy requiring automated redaction; use /ingestions for those tenants or any new compiler features.

Source fidelity

Canonical input has id, optional channel/startedAt, and ordered turns containing role, text and optional startMs. Dates remain unknown when absent. Contact Lens imports preserve contact IDs, CCIQ started_at, original channel, duration and synthetic provenance. Empty non-speech events are counted in source_metadata.omittedEmptyTurns, not turned into invented speech.

Netomi workbook import uses the existing parser, including swapped transcript and summary columns. It does not import evaluation labels, infer dates, or use workbook summaries as model findings. Migrating existing Contact Lens exports preserves their original IDs and declared synthetic timing. Synthetic and real sources must use separate datasets; FNOL is real-source demo material despite its historical S3 prefix containing the word synthetic.

CSV maps one turn per row using a required conversation ID/text column and optional role, start time, channel and offset columns. Parsing supports quoted multiline fields and rejects duplicate headers, conflicting metadata and malformed rows. Conversation IDs are validated across files by the normal compiler. GET /datasets/{id}/conversations provides tenant-scoped, paged prepared transcript previews; the source inventory reports retained status and per-file conversation counts. Migration 072 permits CSV source manifests.

Chat, search and app experience

Author at /agents/{id}/analytics. Call Drivers is the first Analytics type; older agents default to it. The wizard and settings expose a Type dropdown. See types and canvas for the type registry, per-type configuration editors and fixed preset layout.

Dataset operations live in a modal. Preview flushes the collaborative draft and opens maximized; minimize changes layout without remounting the chat. Direct draft preview URLs load the saved configuration without mutating it. Published preview routes do not join an editing room or read a draft. The preview header retains chat controls, copy link, edit/back navigation, and tenant/account controls when maximized. Feature flags and all analytical API authorization remain unchanged.

AnalyticsEditor uses the shared collaboration room, bridge, presence and save indicator. Both pipeline converters use an Analytics Y.Map with recursive object fields, so edits to different settings merge independently; vocabulary arrays are atomic. Legacy Analytics values in the misc map are readable and migrate on reconciliation. The gateway hydrates a published Analytics config when its draft is initially empty. Drafts may contain incomplete settings; Preview/Publish validate them, flush collaboration and confirm the persisted content. Test through the city nginx origin, because :3001 bypasses the collaboration WebSocket.

Agent versions own the Analytics type, preparation drafts, data selection, live models/prompts, tools, composition and branding. Prepared releases freeze their compiler profiles. Every relative-date path uses today UTC; legacy referenceDate is accepted for compatibility but ignored. Explicit historical dates remain available. All tenant RunAgents users can consume published Analytics agents; ManageAgents controls authoring. Legacy audience lists are ignored. Branding copies existing surface templates and supports per-agent logo/icon/palette values.

Keyword counts cover all indexed matches, never a top-k estimate. Exact volume, annotation and outcome counts cover the displayed date/channel/annotation scope; keyword matching is identified separately. Unknown dates/outcomes remain unknown. Annotations are explicitly model findings, not verified operational facts. Evidence ordering favors different conversations before additional turns from the same one. Overview evidence prioritizes the annotation's supporting turns over introductory greetings. Explicit requests to show one conversation select one authorized hit and retrieve bounded, ordered excerpts from that contact; they cannot mix multiple calls into a purported single example. Opening a transcript shows all prepared roles; role filtering is a retrieval setting, not a data ACL.

Semantic queries use the released embedding model, profile and dimensions. Exact cosine search applies tenant, release, date, channel, role and annotation filters before ranking. Basic hybrid search fuses keyword/semantic ranks; full analysis can additionally constrain a semantic search to lexical topic matches inside the authorized candidate set. No global ANN top-k can exclude an authorized tenant's evidence. This is suited to demo-scale corpora; larger deployments should benchmark release-partitioned ANN indexes, warehouse partitioning and compiled rollups before scaling.

The selected provider supplies bounded structured planning/explanation; annotations use a taxonomy-specific schema and valid turn bounds. Basic mode does not accept model SQL. Full analysis accepts one guarded SELECT/CTE over registered typed relations; it cannot change data bindings, perform writes, call arbitrary functions or resolve credentials. Chat query embeddings have a four-second timeout; other model calls allow 20 seconds. Chat falls back visibly to keyword evidence or warehouse results when a provider is unavailable. Date output is constrained to calendar dates. Grouped citations are validated and expanded into clickable references. Answers use Studio's Markdown renderer, with remote images and model-generated external links disabled. Provider-reported annotation/chat/embedding token usage uses existing costing events (embedding product embedding). Unknown pricing is not a zero-cost estimate. Transcription and PII request counts are included in budgets; their provider costs are not yet metered by this worker. The existing tenant governance and model-deny rules apply.

Publication/tenant permission checks, session ownership and current data bindings gate history and evidence on every request. Tenant authors administer datasets; there are no additional row-level business grants beyond tenant/release isolation.

Migration and verification

npm run analytics:migrate-cciq -- SPEC.json CHECKPOINT.json [--inspect] is a trusted operator command using the normal Studio environment. Its specification contains tenant slug, an active authorized actor username, source integration ID, bucket/region and datasets (externalId, name, redacted prefix, synthetic, preparation, optional duplicatePolicy). Credentials remain in encrypted integrations. It paginates the entire source inventory, uploads six files concurrently through shared storage, records provenance and checkpoints resumably. Keep its checkpoint private. It writes only to Studio; original CCIQ objects remain unchanged. Duplicate contact IDs fail by default. Explicit duplicatePolicy: "latest" inventories contact IDs before upload, selects the newest source object for each ID (lexical key breaks timestamp ties), and reports superseded object counts. Changing the specification or source inventory requires a new checkpoint.

The migration covers FNOL, the original Delta seed, Netomi Delta and the newer CallSight corpus. The original Deloitte/Delta/FlightStar profiles share the CallSight data but become separately configurable Analytics Agents. Brand instructions do not rewrite original evidence. Full analysis also imports original operational/customer/survey tables, annotations, trend and memory bodies, and citation indexes through scripts/migrate-cciq-analysis.ts. This operator adapter reads the original AWS sources with an explicit tenant integration; serving uses PostgreSQL and has no AWS credential dependency. Historical generated reports remain identified as model findings. Original chat sessions are not imported.

The OpenAPI reference includes manifest creation, file upload, progress, start/retry/cancel/rebuild, authoring, published agents, private history, chat and evidence. Publication reuses the existing pipeline version endpoint. Every route uses Studio authentication and the matching agents:manage/agents:run scope.

cd apps/ai-manager
node --import tsx --test lib/analytics/analytics.test.ts lib/analytics/preparation.test.ts
# Only an isolated local validation database, with 068–071 applied:
ANALYTICS_TEST_DATABASE_URL=... node --import tsx --test lib/analytics/*.integration.test.ts
npm run typecheck
# Fixed bridge privacy/workbook regressions:
"$ANALYTICS_PYTHON" scripts/analytics-preprocess.test.py

Tests cover tenant isolation, authenticated encrypted storage, source redaction, idempotency, immutable rebuilds, failed-stage recovery, model/evidence contracts, request budgets, source retention and retrieval. Browser tests exercise normal Studio authentication, upload, canvas controls, preview, publication, evidence, private history and tenant-reader permissions. Vanguard additionally exercises real transcription, annotation and embedding integrations using synthetic fixtures.

Current validation and limits

See executed validation for source revision, dataset coverage, side-by-side results, browser checks and commands. The earlier transcript-only benchmark is superseded: its shorter answers were not equivalent work and must not be quoted as full-analysis performance.

Full analysis uses bounded read-only SQL translated from Trino to PostgreSQL. CTEs, joins and arrays are supported; writes, unregistered relations, arbitrary functions, recursion and lock clauses are rejected. Query time is capped at ten seconds and returned rows at 5,000. SQL errors use savepoints so the model can repair a query without aborting the entire chat transaction. Cost metering uses a small independent pool to avoid waiting for a connection held by the chat.

Snapshots support up to 50 typed relations and 100 MiB warehouse upload bodies. They remain a portable PostgreSQL implementation, not a replacement for a large columnar warehouse. At larger scales, implement a warehouse adapter and benchmark its joins and vector filters before changing those bounds. Runtime models support OpenAI, Bedrock and Vertex through tenant integrations; shared transcription/redaction retain their existing providers. GCP DLP byte offsets are converted to Unicode character offsets before redaction. Response components and briefings can use selected published catalogue versions. The focused app uses Studio SSO and tenant access, requiring no second identity system or standalone deployment.