Analytics Agents from CallSight¶

Implementation follow-up: the native Analytics runtime implements the canvas, preparation, operational warehouse, 16-tool analysis loop, report hierarchy, briefings and shared chat cards. See executed validation for migrated data, comparisons and remaining limits. The assessment below records the original source investigation and design recommendations, not current deployment status.

Status: Current | Last reviewed: 2026-10-02

Use this assessment to scope an Analytics Agent type in Agent Studio while preserving CallSight's chat, charts, briefings and evidence navigation. The recommended approach is to reuse CallSight's analytics behavior and renderer, give Studio ownership of configuration and access, and introduce a portable analytics service with a separate ingestion path.

Recommendation and scope¶

Build the authoring canvas and initial end-user Analytics workspace inside Agent Studio, behind an environment feature flag. Extract the CallSight experience into a shared React package so the same experience can run in Studio Preview and, later, a dedicated application using the same identity provider and APIs.

Keep analytics processing separate from the live voice runtime. Use the existing workflow infrastructure for durable ingestion and compilation, and reuse the platform's identity, versioning, integration credentials, object storage, telemetry and conversation services. Preserve Athena as a migration adapter; use an indexed PostgreSQL analytics serving database for the initial portable deployment. Make a larger warehouse an implementation choice behind the analytics API.

The native pilot is an intermediate release. Migration completion requires the question and result parity contract: all original analytical tools, data relationships, reports, recommendations, supported what-if scenarios and conversation/presentation behavior. The catalogue includes questions outside the recorded demo and requires comparable results before performance acceptance. Universal file interpretation, new analytical methods beyond CallSight, unrestricted SQL, automated operational actions and a separate branded portal remain additional scope.

This is a source assessment, not a deployed-environment audit:

CCIQ: fresh Deloitte-US/trueserve-cciq main, commit 720c2a48c3d7599ec2d987cdff0283e5fea1d83b, October 2, 2026.
Framework: local environment/vanguard, HEAD 583ed0c2578e2a5ec22140e9774c7eda2d697c73, with pre-existing working changes. This is not asserted to be the revision running in official Dev or QA.
At the time of this initial assessment, no production data, deployed configuration, cloud resources or live model calls had been examined. Performance numbers below come from repository comments/history; subsequent live demo measurements are in the validation report.
CCIQ paths in the evidence register are relative to that repository at the pinned commit. Framework links resolve within this repository.

What CallSight contains today¶

CallSight is a specialized analytics application with two execution paths: an interactive analyst and an offline compiler. Its useful behavior extends considerably beyond a system prompt and a transcript search tool.

Area	Observed implementation	Migration implication
Browser experience	React 18, Vite, Tailwind, Framer Motion and Recharts; chart/table/metric cards, executive briefings, follow-ups, citations, contact pickers and a breadcrumb detail drawer	Extract the components and interaction contract before redesigning anything
Interactive API	FastAPI `/api/chat`, `/api/briefing`, contact and citation routes; thread-to-async SSE bridge	Retain the behavior behind Studio authentication and a versioned runtime contract
Model loop	Bedrock Converse streaming, up to 20 tool rounds, system/tool/history cache points, parallel tools with up to five workers per round	Generalize provider messages, credentials and streaming without losing tool semantics
Tools	SQL queries, transcript search, contact retrieval, trends, executive summaries, citations, anomaly/intent-drift checks, charts, KPI cards and what-if scenarios	Separate data access, analytical functions and presentation functions
Warehouse	Athena over Glue-catalogued Iceberg tables and S3 objects	Keep as the first source adapter; remove Athena SQL and S3 URIs from the product contract
Transcript search	Titan embeddings, a local pickle index, NumPy similarity search; default build cap of 6,000 contacts and an 8,000-character embedding input truncation	Replace with complete, revisioned chunk indexing and a shared search store
Offline compiler	Lambda actions orchestrated by Step Functions; Glue loads staged JSONL to Iceberg; EventBridge schedules; optional Bedrock Knowledge Base sync	Extract transformations into reusable worker functions
Branding	File profiles with branding, copy, vocabulary and domain context; request profile resolves the dataset	Import as agent configuration and reusable tenant assets
Library	A hardcoded profile catalog, followed by `?profile=` navigation	Replace with an authorized published-agent listing
Identity	Deployment script expects an Authentik outpost and checks an SSO redirect; application routes do not bind that identity to Studio tenants/resources	Reuse the identity provider, add application authorization
Chat persistence	Backend process-local dictionary; browser messages held in React state	Introduce durable ownership, history, attachments, replay and retention

Evidence: C1–C7, C10–C12.

The repository documents an airline demo corpus of 999 synthetic contacts and an FNOL pilot of two real contacts. Those are documented dataset sizes, not counts checked against AWS. The thin FNOL corpus is useful for adapter validation but does not establish analytical quality or scaling behavior.

Brand profiles are partly generalized. FlightStar and Deloitte can present the airline data under different branding; FlightStar also rewrites source-brand strings in API output. That is a demonstration mechanism, not tenant isolation or an appropriate transformation of evidentiary quotations. Several remaining behaviors are still domain-specific: the what-if levers include Atlanta authentication changes and billing deflection, with a hardcoded illustrative handling cost. Universal ingestion is a scoping document, not a shipped upload product.

Current data flow¶

flowchart LR
    Sources["Contact Lens JSON / seed / FNOL adapter"] --> S3["S3 source objects"]
    S3 --> Compiler["Lambda compiler / Step Functions"]
    Compiler --> Glue["Staged JSONL / Glue loads"]
    Glue --> Warehouse["Iceberg / Glue catalog"]
    Compiler <--> Bedrock["Bedrock annotations and summaries"]
    Warehouse --> Athena["Athena"]
    Athena --> Compiler
    Compiler --> Artifacts["S3 summaries, trends and evidence"]
    Artifacts -.-> KB["Optional Bedrock Knowledge Base"]
    Browser["CallSight browser"] --> API["FastAPI / Bedrock tool loop"]
    API --> Athena
    API --> Artifacts
    API --> Search["Titan / local NumPy transcript index"]
    API -->|SSE text, tool status, attachments| Browser

Diagram source: current CallSight flow.

The app uses both cciq_* business tables and ci_* compiler tables. Business tables contain contacts, transcripts, surveys, customers, agents, queues and domain context such as reservations and operational changes. Compiler tables contain normalized conversations/utterances, annotations, annotation evidence, versioned summaries, trends and citations.

Compilation already includes content signatures, incremental watermarks, parallel annotation and active-artifact management. These are valuable foundations. The runtime reads summaries/trends directly through tools; its transcript search uses the local index. An optional Bedrock Knowledge Base exists in infrastructure, but it is not the core interactive retrieval path examined here.

The compiler hierarchy audit records the implemented rich multi-label annotations, configurable interval → daily → weekly reports, compiled memory pages and interactive anomaly tools. Parent reports consume child narratives and query parent-window facts. Runtime peer-outlier detection is separate from report synthesis.

Preserve the experience and its evidence contract¶

The October 2 rendering decision uses the shared A2UI registry for CallSight's composite cards and charts. The common host is apps/ai-manager/components/analytics/AnalyticsRichChat.tsx; Preview and the focused app share it. The catalogue contract records message length, expansion, ordering and drawer defaults, plus checkpoints from callsight-demo-v2.mov. Its agent-library → opening-briefing → contextual-chat → evidence flow is the acceptance journey.

Port CallSight's components before redesigning them. Replace global profile loading and direct API calls with injected configuration and an API client. Scope styles and adapt dependency versions to the Studio host. Reuse catalogue schemas, publication, versioning, previews and the shared registry without requiring voice-runtime orchestration.

Define an ordered, typed response-document schema with prose, chart, KPI, recommendation, follow-up and evidence-reference blocks. Translate the current [[kind:id]] markers and [cite:…] references at the compatibility boundary so captured CallSight answers can be replayed against both renderers. Models reference server-created result and attachment IDs; deterministic code supplies numbers, units and chart series.

Persist response blocks, attachments, evidence identifiers, catalogue versions and initial disclosure defaults. A replay reproduces the original chart and citation targets after a restart or model change. Give events stable turn/event IDs, replay cursors and explicit cancellation. Keep per-viewer presentation state separate. Adapt the shared renderer's historical-card locking and global component registration before applying it to inspectable Analytics history.

Treat the existing “ReasoningTrail” as tool activity, not a requirement to expose hidden model reasoning. Preserve progress labels, execution provenance and evidence.

Analytics canvas and configuration ownership¶

Add a proposed analytics_agents subtype under text_to_text, with an explicit analytics runtime discriminator such as agent_type: "analytics". These names are proposals. Current inference falls back to a voice agent for unknown types, so adding a label alone is insufficient.

Use a dedicated configuration canvas with connected sections:

flowchart LR
    Data --> Preparation --> Analysis --> Experience["Answer experience"]
    Models["Models and behavior"] --> Analysis

Diagram source: canvas sections.

These sections represent configuration and dependencies. A chat request should not rerun ingestion or annotation just because those sections appear on the canvas.

Section	Author-configurable inputs
Identity and branding	Agent name, description, logo, product label, theme tokens, chart palette, terminology, locale
Data	Dataset bindings, reference sources, available dimensions, default filters, permitted reporting windows
Preparation	Versioned adapter and preparation profile, transcript field mappings, taxonomies, extraction schema, quality thresholds, refresh policy
Models	Separate chat, annotation, summarization, embedding, transcription and optional OCR selections; tenant integration references; capability-appropriate settings
Behavior	Instructions, audience, response depth, evidence requirements, ambiguity handling, permitted tools, maximum rounds and runtime budget
Metrics and analysis	Metric definitions, numerators/denominators, units, time zone, comparison windows, anomaly thresholds, minimum cohort sizes, explicit what-if assumptions
Answer experience	Per-agent landing mode/layout, briefing content and refresh schedule, starter prompts, allowed charts/cards, transcript drawer, citation visibility, follow-up behavior, exports
Publication and access	Published audience, embedding policy, availability and data permissions; managed outside the versioned content where immediate revocation is needed

Keep operational settings out of the canvas: database URLs, object-store endpoints, queue names and secrets belong to deployment settings or encrypted tenant integrations.

Use the existing identity/version split:

agents: tenant ownership, identity, subtype and surface association.
agent_versions.config.analytics: the versioned analytics behavior, model references, dataset/preparation-version references and experience settings.
Dataset/preparation records: reusable sources, adapters, taxonomies, annotation schemas and data releases. Two agents can share one prepared dataset without paying to process it twice.
Access grants and tenant policy: immediate authorization, permitted integrations, export rights and resource ceilings. Publishing a new prompt must not restore revoked access.

Do not rely only on a mutable surface_config_id for reproducibility. Capture a version or content digest for branding, prompts, taxonomy, preparation configuration and data manifest in the published/runtime manifest. Data may follow a configured “latest approved release” policy, but each chat run records the exact release it used.

Preview uses an authorized draft configuration snapshot and the same renderer/runtime as the published workspace. Pin the snapshot for the session; expose the selected data release and effective settings. A Builder can preview unpublished work. The end-user library resolves published agents only. Preview uploads should use a separate dataset revision until promoted.

Implementation touches include pipeline types, agent labels, the creation wizard, editor routing, publish validation, import/export, templates and runtime dispatch. Extend the generated configuration schema and both collaboration converters: Studio converter and gateway. Unknown root sections are otherwise dropped during autosave. Add a round-trip check for the complete analytics configuration.

Studio workspace, standalone app and access¶

Option	Benefit	Additional work	Recommendation
Integrated Studio workspace	Existing login, tenant switcher, navigation, Preview and deployment	Analytics navigation, run-only layout and resource authorization	First release
Dedicated Analytics app	Independent branding and a focused business-user experience	OIDC client/BFF, session lifecycle, tenant-selection UI, deployment and the same authorization integration	Add when distribution requires it
Existing surface-app	Existing signed embed sessions, chat transport and A2UI support	Analytics renderer and runtime routing; it is currently an embed surface, not a tenant-aware SSO agent library	Reuse selected infrastructure; optional host

Proposed gates are FEATURE_ANALYTICS_AGENTS=false and NEXT_PUBLIC_FEATURE_ANALYTICS_AGENTS=false by default, following feature-flags.ts. The server gate is authoritative. Hide navigation and creation options, and gate direct page/API/runtime entry points. Return the effective server capability to the UI so a runtime disable cannot leave a working client entry point. Tenant enablement may narrow deployment availability.

Keep Analytics availability independent of SIP and real-time voice. If compilation uses async-agent workers, document that backend dependency rather than requiring workflow-authoring UI access for Analytics users.

Existing Studio roles are Admin, Builder and Viewer. RunAgents currently includes browsing/running every agent type and inspecting conversations within the tenant. The legacy pipeline-sharing helpers do not implement a per-agent consumption allowlist. “Select agents they have access to” therefore needs new resource policy, even though tenant roles and switching already exist.

Implement access as the intersection of:

Authenticated, active tenant membership and deployment/tenant feature availability.
Tenant run/manage capability.
Agent audience grant: tenant-wide, group or named user.
Dataset entitlement and any permitted row/field scope.
Operation permission: aggregated results, transcript detail, upload, export or administration.

Evaluate it on listings, chat creation, every tool query, citation resolution, downloads, saved-history reads and event reconnection. Bind the session to tenant, user, agent and version. In the initial design, keep chats private to their owner and make sharing explicit. Existing broad conversation-history access is not automatically appropriate for analytics containing employee/customer evidence.

Use the existing tenant switch endpoint. On a switch, abort the prior stream, reset drawers and client caches, and create a session in the new tenant. Bind concurrent requests to an explicit verified tenant context; a switch in another tab must never redirect an in-flight tool call to different data.

For a standalone app, use a separate OIDC client with the same Authentik issuer and its own secure session/BFF. “Sign in with Agent Studio” can be the user-facing entry, with Authentik providing SSO. Reuse Studio's membership and authorization APIs. Do not share its admin cookie across unrelated origins or use an identity token as an analytics API access token. Scope backend requests to the intended API audience and selected tenant.

Target service architecture and reuse¶

flowchart TB
    Studio["Studio canvas / Preview / Analytics workspace"] --> Access["Studio identity and authorization"]
    Portal["Optional dedicated app"] --> Access
    Access --> API["Analytics API and chat runtime"]
    API --> Config["Draft / published manifest resolver"]
    API --> Tools["Metric, query, evidence and search tools"]
    API --> Models["Model-provider adapters"]
    API --> Conversations["New conversation-api session/event adapter"]
    API --> Contract["Shared CallSight response contract"]
    Tools --> DB["Analytics PostgreSQL / vector and text indexes"]
    Tools --> Storage["Object store / Redis"]
    Inputs["Uploads / Studio transcripts / connectors"] --> Workers["Analytics functions on async-agent workers"]
    Workers --> DB
    Workers --> Storage

Diagram source: target architecture.

The Analytics API is a new, independently scalable FastAPI service/package. Initially it can wrap CCIQ's runtime and Athena tools behind a secure Studio boundary. Its long-term responsibilities are querying, evidence resolution, analytical chat and serving manifests—not processing bulk files inside a request.

Existing capability	Reuse	Work still required
ai-manager	Login, tenant context, encrypted integrations, agent versions, publishing, role checks, UI shell	New subtype/canvas/library, analytics policy, data management
collab-gateway and editor infrastructure	Presence, autosave, version flush, reusable layout/panel controls	Analytics config serialization and editor-specific validation
conversation-api	Sessions, messages, event persistence/streaming, attachments, recording/archive references	Explicit analytics runtime dispatch and durable typed-response/run records; current chat path starts ai-orchestrator
async-agent + RabbitMQ + Redis	Queued jobs, scheduler, checkpoints, durable iteration, provider batch jobs and concurrency controls	Ingestion/compilation functions, manifests and reconciliation
event-publisher	Existing database-event fan-out for triggering downstream work	Finalized-transcript and data-release event contracts with deduplication/reconciliation
Model/integration code	Tenant credential resolution and provider clients; async-agent has OpenAI-compatible, Anthropic and Gemini chat branches	Extract an appropriate shared interface; preserve/add Bedrock interactive support and embedding providers
agent-host/MCP	Reuse approved reference-data and operational tools; expose analytics tools to other agents later	Tenant/dataset context enforcement and narrow tool contracts
Shared storage	`libs/storage` and `shared/storage`: streams, reads, presigned GET, S3-compatible configuration and local test implementation	Analytics storage role/prefix, quotas, upload sessions; browser multipart/presigned PUT is not exposed by the current common interface
Shared transcription/OCR	`libs/transcription` provider registry and `libs/ocr`	Wire asynchronous media/document ingestion and quality reporting
Shared PII, encryption and telemetry	`libs/pii`, envelope encryption, tenant-aware OTLP and costing	Apply to new payloads, evidence access, deletion and worker spans
Tenant Knowledge Sources	Existing tenant-configured reference retrieval for policies, manuals and KB material	Preserve source/version provenance; it does not replace structured metric queries

Do not route analytics chat through telephony/media services. Do not treat the Studio coding Assistant runner as the analytics runtime. Its code execution/workspace permissions are unnecessary for query, evidence and chart tools.

Avoid overstating portability already present in shared libraries. libs/reasoning/llm_client.py has a narrower provider set than async-agent's executor, and neither is a complete replacement for CCIQ's Bedrock streaming loop. Existing credentials can fall back to process environment settings; Analytics should require an explicitly authorized tenant integration and fail closed when it is unavailable.

The current conversation storage implementation already uses trueserve_storage; older root guidance claiming a direct MinIO SDK dependency is stale. The seam supports S3-compatible access, including documented GCS interoperability. Native Azure Blob would require a new adapter or a compatible gateway.

Transcript ingestion and preparation¶

Adopt the compiler-neutral turn-list direction proposed in CCIQ's ingest-tooling-scoping.md. Extend it with server-assigned tenant/dataset identity, source provenance, revisions and quality fields. Contact Lens becomes one input adapter.

Canonical entity	Required information
Conversation	Internal ID; tenant/dataset/source IDs; external contact ID; channel; language; start/end if known; source content hash; schema and adapter versions
Turn	Stable turn ID; sequence; participant ID and role (`customer`, `human_agent`, `bot`, `system`, `unknown`); content; optional offsets; confidence and provenance
Source evidence	Object reference; source row/page/time span; redaction status/version; recording reference when available
Domain metadata	Validated typed fields plus extension metadata; map hub/claim/product/queue without making an airline schema mandatory
Derived annotations	Extraction/prompt/model/taxonomy versions, confidence, supporting turn IDs, processing status

Missing timestamps, speakers, CSAT and transfer outcomes remain unknown. Do not invent duration from text or mark missing outcomes as successful containment. Distinguish source facts, model-inferred labels and synthetic ground truth. The compiler currently permits ExpectedAnnotation to substitute for a model call; confine that behavior to explicit synthetic/evaluation workflows.

Recommended upload flow:

Select a dataset and create an authenticated upload session with size/type limits and a server-chosen object key.
Stream small files through an upload endpoint; support direct multipart object uploads for larger inputs by extending the storage interface. Verify size/hash and source ownership before completion.
Detect format and preview a bounded sample. Show conversation grouping, role mappings, timestamps, missing fields and redaction status.
Confirm/save a declarative adapter mapping. Use model assistance for ambiguous mapping proposals, not as the default parser for every row.
Queue normalization, deduplication and PII handling. Quarantine invalid records and provide a per-record error report.
Annotate and index accepted records using a versioned preparation profile. Display progress, completeness, estimates and retryable failures.
Build metrics/summaries/citations and atomically publish a complete data-release manifest. Queries continue using the previous release until the new one is ready.

Start with canonical JSON/JSONL, Contact Lens JSON, CSV/XLSX turn or conversation exports, and labeled plain-text transcripts. Add known vendor exports through adapters. Add audio via the existing transcription registry and PDF/DOCX through text extraction/OCR after structured ingest is reliable. Treat reference manuals and policies as reference documents, not conversations.

Consume Studio conversations through finalized transcript events or an incremental export adapter. Respect message_key, variant, revision, final/enhanced transcript selection and archived evidence; naïvely copying each streaming row will duplicate utterances. Keep the imported analytics corpus separate from the analyst's own chat sessions.

Make processing idempotent on (tenant, dataset, source, external_id, content_hash, preparation_version). Track a durable manifest per batch and retry only incomplete stages. Use an outbox/reconciliation mechanism around database commits and queue delivery. Handle corrected transcripts, late arrivals and deletes by publishing revisions and rebuilding affected aggregates/indexes.

Database and storage design¶

Start with a dedicated analytics PostgreSQL database and connection pools, using existing operational patterns. A separate database on the same server is convenient initially but does not isolate CPU/I/O; load measurements determine when to move it to its own instance. Keep interactive analytics off the core voice/runtime database as volume grows.

The core baseline already enables vector; this reduces deployment novelty but does not mean transcript chunk search already exists. Use indexed contact/evidence lookups, full-text search plus vector retrieval, and precomputed metric tables. Evaluate a columnar store or existing customer warehouse when scans, retention or concurrency exceed the measured PostgreSQL budget. Do not introduce several new database products before measuring representative workloads.

Store	Proposed contents
Studio control database	Agents/versions, analytics dataset/source registry, preparation versions, access grants, published bindings and integration references
Analytics serving database	Contacts, turns, annotations, evidence links, source dimensions, surveys/outcomes, metric aggregates, artifact/citation metadata and search chunks
Object storage	Original restricted uploads where retained, normalized/redacted JSONL or Parquet, recordings, compiled documents, exports and reproducible manifests
conversation-api storage	Analyst chat sessions/messages and lifecycle; analytics run/response extensions store structured output and manifest references
Redis	Query/result caches, shared admission limits, event delivery and short-lived state; never the only copy of evidence or history

Proposed table families are analytics_datasets, analytics_sources, analytics_preparation_versions, analytics_dataset_releases, analytics_agent_grants, analytics_contacts, analytics_turns, analytics_annotations, analytics_annotation_evidence, analytics_metric_rollups, analytics_artifacts, analytics_citations and analytics_chunks. Reuse existing workflow execution records; an ingestion manifest links to the workflow execution instead of creating a second workflow engine.

Use tenant/dataset/revision keys and matching composite relationships throughout. Internal contact IDs must include source scope; vendor contact IDs alone are insufficient. Version annotations rather than overwriting them. Publish an atomic manifest containing source/annotation/metric/index/artifact versions, coverage and freshness. A chat run pins that manifest so counts, prose and evidence describe the same dataset state.

Keep citations to historical artifacts for as long as saved answers retain them. Expiring a “current” pointer must not erase evidence needed to replay an old answer. Enforce deletion policy across source objects, normalized text, embeddings, derived artifacts, caches, exports and retained chat attachments; display a redacted/unavailable reference when evidence must be removed.

Query isolation and metric correctness¶

Use a restricted analytics database role with no ownership, superuser or BYPASSRLS privileges. Apply RLS and explicit tenant/dataset predicates, transaction-local context, read-only transactions, statement timeouts and an allowlist of views/functions. The framework baseline explicitly grants BYPASSRLS to tsf_app; copying that credential would defeat database-enforced isolation even with FORCE RLS.

Prefer semantic tools such as query_metrics(metric, dimensions, filters, window) and get_evidence(result_id) over arbitrary model SQL. During migration, retain a restricted SQL tool over approved analytics views only. Validate every table/catalog/function reference, not just known names, and bound scan cost as well as output rows. A LIMIT 5000 does not limit the work needed for an aggregate or join.

Define containment, escalation, AHT, CSAT and repeat contact in a versioned metric catalog. Preserve the distinction between actual transfers and annotator-detected escalation triggers, and between observed and inferred intent. Return units, numerator, denominator, source count, time zone/window, coverage and freshness with every metric. Label what-if output as a scenario with assumptions; it is not evidence of a causal effect.

Concrete gaps that affect migration¶

These are source findings and design risks, not claims of exploitation or measured production incidents.

Finding	Evidence	Required treatment
Profiles select data without application user/tenant authorization	C2, C3, C10	Resolve datasets from an authorized agent binding; profile IDs never grant access
History keyed only by caller-supplied conversation ID in process memory	C2	Server-created owned sessions, durable writes, per-turn concurrency control
Known-table SQL rewriting is not a complete allowlist; unknown table names pass through	C4	Restricted database role and closed relation/function scope
Dataset propagation is incomplete: `run_whatif` calls omit `ctx.dataset_id`; small reference caches load the server-default dataset	C7	Require scope in every query/cache API and test reused external IDs across datasets
Compiler IDs hash only `contact_id`; several MERGE keys lack dataset scope	C8	Migrate to scoped internal IDs and explicit tenant/dataset relationships
Discovery watermark advances during normalization, before downstream annotation/load completes	C8, C9	Durable stage manifests and reconciliation; restarting discovery alone can skip unfinished work
Cost acknowledgement is checked by the compiler but omitted by the main state-machine payloads	C8, C9	Persist budget authorization and propagate it through each cost-incurring stage
In-process body cache ignores requested per-key TTL; contact/annotation cache assumes seeded data is immutable	C6, C7	Correct TTL behavior and revision-based keys with invalidation
Briefing cache uses profile/dataset/manual prompt version and 24-hour TTL, not a compiled-artifact manifest	C2	Key on authorized scope, agent/config and data release; invalidate on publication
Local search can fall back to a legacy index and does not validate the loaded dataset/model metadata	C5	No cross-dataset fallback; verify embedding model/dimensions and indexed revision
Recompilation overwrites annotations and deletes citations for inactive summary/trend versions	C8	Preserve versioned evidence required by saved answers
Redaction admission checks source shape/path, not actual PII removal	C8	Run shared PII policy and record redaction provenance

Performance priorities¶

CallSight already parallelizes tool calls and contact queries, caches compiled bodies/listings, prefetches contacts and uses Bedrock prompt caching. Preserve those improvements. The biggest next gain is changing what work sits on the interactive path.

Priority	Change	Why it matters here
1	Serve contact, citation and artifact lookups from indexed PostgreSQL/object metadata	CCIQ comments attribute seconds of cold latency to individual Athena lookups
1	Cache briefings/results by data manifest and authorized scope; rebuild after publication	Avoids repeated agent work without returning stale or differently scoped results
1	Deterministic metric tools and chart construction from result references	Reduces model rounds and arithmetic/rendering errors
2	Narrow prompt/schema context to the selected tools/domain; compact old tool results	Base system prompt is 742 lines, plus tool descriptions and growing history
2	Stream validated final text and completed blocks while keeping internal planning hidden	`/api/chat` currently enables deferred text and emits assembled prose after tool work
2	Shared request limits, provider concurrency budgets, cancellation and backpressure	Current request threads and per-turn pools multiply with simultaneous users; SSE queue is unbounded
2	Chunked, complete hybrid search with dataset filters	Avoids one-vector truncation and sampled-corpus blind spots
3	Incremental per-stage processing, batch inference and bounded bulk database loads	Avoids repeated full annotation and multiple Glue jobs for small batches
3	Recompute only affected windows; compact analytical files if retaining Iceberg	Keeps preparation cost tied to new or corrected data

Cache keys should include tenant, effective data-access scope, agent/preparation versions, data manifest and normalized query. Include user identity where output is personalized. Revocation must affect cached results and event/download access. Use single-flight generation to avoid duplicate briefing work on concurrent cold requests.

Do not assume Athena result reuse is configured from comments alone: the inspected query wrapper does not pass explicit result-reuse/cache settings. Verify it if retaining Athena. Its warm behavior is not a substitute for measuring cold queries and concurrency.

Proposed acceptance targets, to validate against an agreed workload: p95 contact/citation reads below 500 ms, cached briefing below 1 second, standard aggregate tools below 1 second, simple grounded answers in roughly 3–8 seconds and complex investigations with useful progress within 1 second. These are engineering targets, not guarantees.

Benchmark at 1, 10 and 50 concurrent chats and at approximately 10,000, 100,000 and 1 million contacts, including long transcripts and mixed data scopes. Measure cold/warm latency, time to first useful content, total answer time, provider/SQL rounds, tokens, scan bytes, ingest throughput, completeness, failure/retry rate and cost per processed contact/answer. Use the results to choose deployment size and any warehouse expansion.

Phased migration¶

Phase	Deliverable	Exit condition
0 — Establish contracts	Pin CallSight response fixtures, dataset mappings, metric definitions and baseline questions; measure current behavior	Exact numeric/evidence expectations and visual fixtures exist
1 — Studio vertical slice	Flagged agent type/canvas, versioned config, shared renderer, SSO/tenant/resource policy, Preview and integrated workspace; secure Athena adapter	One published and one draft agent behave correctly; cross-tenant/agent access is rejected; chat survives restart
2 — Portable data preparation	Canonical uploads, Studio transcript adapter, durable jobs, analytics PostgreSQL/search/object storage, manifest publication	Repeat upload is idempotent; partial failure resumes; counts and citations reconcile
3 — Portable model/compiler runtime	Provider-neutral messages/tools, separate chat/preparation models, extracted compiler functions, shared provider limits and telemetry	At least one non-Bedrock configuration passes the same quality suite
4 — Controlled cutover	Historical backfill, shadow comparisons, tenant-by-tenant data binding changes and operational runbooks	Agreed parity/performance/access tests pass; old binding remains a rollback option
5 — Optional distribution	Dedicated branded SSO application and optional embeds/A2UI exposure	Same agents, versions, policies and response contract work through both hosts

Preserve original dataset IDs as external aliases and original evidence IDs in a migration map. Import existing annotations, summaries, trends and citations before re-annotating. Reuse embedding vectors only when model/dimensions/chunking match; otherwise rebuild explicitly. Brand profiles become templates/configuration, not newly invented tenants. Sharing the same synthetic corpus across demo brands requires an explicit authorized binding.

Backfill through immutable exports or a pinned source snapshot. Compare contact/turn counts, metric denominators, time windows, taxonomies, artifact bodies and evidence resolution. Record missing historical citation links as a source limitation rather than silently recreating “original” evidence. Switch each agent to a prepared data release; preserve its old adapter configuration for rollback. New uploads need a catch-up/export plan if rollback must restore the old runtime.

Indicative planning range for two backend/data engineers and one frontend engineer, with QA/design support: 3–5 weeks for a secure Athena-backed Studio pilot; another 4–7 weeks for structured uploads, portable serving/compiler work and hardening, with some overlap; roughly 1–3 additional weeks for a separate portal after the shared contracts are stable. These are estimates, not repository-derived delivery commitments. Universal media ingestion, complex entitlements and large historical volumes need separate sizing.

Verification and decisions before implementation¶

Use synthetic fixtures for initial replay, UI and tenant-isolation tests. Cover numeric accuracy, missing-data behavior, units/windows, contact/utterance provenance, chart rendering, nested drill-down, reconnect/restart, simultaneous turns, cancellation, expired access, changed data permissions and duplicate external IDs across sources/tenants.

Preparation tests need correction/deletion/re-upload, late data, empty transcripts, missing roles/times, interrupted stages, budget rejection, exhausted retries and publication of only complete manifests. Provider parity means grounded answer and UI/evidence quality, not identical wording. Add model/prompt/taxonomy changes to a maintained analytics evaluation set.

The decisions that materially change scope are expected volume/retention/concurrency, initial input formats, required data providers, whether users can share chats, how transcript/detail/export access differs from aggregate access, whether a dedicated portal is needed at launch, and whether datasets follow live or frozen reporting windows. The recommended defaults above allow the vertical slice to proceed before every later connector is specified.

Verification performed for this assessment: static inspection of both source trees and three passing existing CCIQ prompt-composition regression tests. The complete CCIQ application, cloud compiler, browser parity and load tests were not run. No model calls, dataset writes or service deployments occurred.

Evidence register¶

Unqualified backend/, frontend/ and tests/ paths in this register are under CCIQ's contact-center-iq/ directory.

ID	CCIQ source at the pinned commit
C1	`README.md`; `contact-center-iq/frontend/package.json`; `frontend/src/types.ts:1`; `frontend/src/components/MessageBubble.tsx:98`; `frontend/src/App.tsx:26` (frontend paths after the first are under `contact-center-iq/`)
C2	`contact-center-iq/backend/main.py:83,367,471`; `backend/store.py:22`; `backend/sse.py:26`
C3	`contact-center-iq/backend/agent.py:102,132,167,224,297`; `backend/tools/base.py:17`; `backend/profiles.py:94`; `profiles/airline-flightstar/profile.yaml`
C4	`contact-center-iq/backend/data_lake.py:45,72,111,178`; `backend/tools/query_data_lake.py:58`
C5	`contact-center-iq/backend/transcript_index.py:31,45,77,106,133,175`
C6	`contact-center-iq/backend/cache.py:50`; `backend/_caches.py:37`; `backend/_warmer.py`
C7	`contact-center-iq/backend/tools/get_contact.py:37,59,88`; `backend/tools/run_whatif.py:12,41,45`
C8	`aws-conversation-intelligence/lambda_src/compiler_handler.py:249,301,364,458,2644,2662`; `glue_scripts/load_staged_to_iceberg.py:215,445` under the same compiler directory
C9	`aws-conversation-intelligence/step_functions.tf:10,49,98,181,304`; `aws-conversation-intelligence/bedrock.tf:59`
C10	`contact-center-iq/frontend/src/components/AgentLibrary.tsx:24`; `frontend/src/profile/types.ts`; `contact-center-iq/scripts/deploy-demo.sh:162`
C11	`ingest-tooling-scoping.md:22`; `contact-center-iq/scripts/ingest_fnol.py:89,143`; `brand-agnostic-plan.md`; `profiles/README.md`
C12	`contact-center-iq/backend/prompts/system.md`; `contact-center-iq/tests/test_system_prompt_composition.py`; `merge-plan-current.md` (historical performance observations only)

Framework source anchors: