Skip to content

Analytics dataset inputs

Status: Current | Last reviewed: 2026-10-02

Call Drivers datasets combine canonical transcripts with supplied operational tables and generated analysis tables. Transcript ingestion does not infer handling time, survey scores, actual transfers or business events.

Authoring contract

GET /api/analytics/datasets/{datasetId}/tables returns supplied input metadata: display names, descriptions, typed columns, keys, relationships, source filenames, row counts, the current input revision, and available prepared snapshots. With releaseId, coverage includes conversations, known dates, annotations, semantic-indexed conversations and supplied handling-time records.

snapshotId selects a dataset-owned immutable version. table, page and search return 50-row pages; search applies to the whole relation. format=csv or json exports a table. format=bundle includes typed tables and stored report bodies, with table metadata under source.inputTables. All exported values have already passed the dataset redaction policy.

POST accepts:

{
  "releaseId": "ready-transcript-release-uuid",
  "baseRevisionId": "current-input-revision-uuid-or-null",
  "changes": [{
    "table": {
      "name": "business_events",
      "columns": [
        {"name": "event_id", "type": "text"},
        {"name": "description", "type": "text"}
      ],
      "rows": [{"event_id": "event-1", "description": "Refund policy changed"}]
    },
    "details": {
      "label": "Business events",
      "description": "Operational changes relevant to customer contact",
      "key": "event_id",
      "relationships": [],
      "mapping": {}
    },
    "mode": "replace"
  }]
}

baseRevisionId is JSON null for the first input import. The request body is bounded at 100 MiB, with up to 40 changed tables, 100 columns and 200,000 rows per table. All changed tables commit together. A dataset row lock and revision check return 409 for a stale import rather than overwriting another author's work.

The UI parses CSV/JSON and previews five rows before sending typed data; the server independently validates types, names, unique keys and relationships. Built-in table schemas preserve the original Call Drivers tool contract. Custom tables declare their columns, unique key and optional relationships. Generated table names and internal prefixes cannot be supplied as inputs.

Modes:

  • Replace replaces this table's records and keeps other tables.
  • Append adds records and rejects duplicate unique keys.
  • Merge updates matching keys and inserts new keys. Unmapped columns are preserved when the request includes a column mapping; explicitly mapped empty cells clear the corresponding values to unknown.

Contact IDs must belong to the selected transcript version. Canonical contact dates and channels take precedence over operational imports. Contacts without supplied metrics remain in the input table with unknown metrics. Foreign keys into a populated table must resolve; an absent/empty reference table produces a visible warning so authors can import related tables in stages. Built-in keys and relationships cannot be removed by changing upload metadata.

All endpoints require the Analytics feature and tenant author permission. Tenant predicates and dataset ownership apply to metadata, row reads, exports, imports and source-version selection. Free text is re-redacted before storage.

Versioning and preparation

Operational input revisions reuse analytics_analysis_sets, analytics_relation_schemas and analytics_relation_rows. They carry configuration.purpose = operational_inputs, table metadata, warnings and the previous revision. No additional physical tables or schema migration are needed.

Input-only revisions are excluded from runtime snapshot resolution and publish-time analysis readiness. They cannot displace the last usable chat snapshot before preparation finishes.

queueReports resolves the latest input revision when sourceAnalysisId is omitted. Before the first native input revision, it adopts existing migrated snapshots containing operational data. Explicit JSON null retains the transcript-only build option; an explicit snapshot freezes that source. The resolved ID is part of the job's content hash and is stored on the job.

The report loader projects supplied facts onto the new transcript membership. Existing contact metrics survive; new contacts have unknown metrics. Rows linked to removed contacts are excluded from serving and recorded in configuration.inputCoverage; their source revision remains unchanged. Prepared snapshots carry purpose = analysis and inputRevisionId. Existing sessions and pinned agents retain their original evidence.

Save inputs stages the new revision. Save inputs & prepare for chat also queues the configured reports using Studio annotations. The Prepared insights tab shows progress and selects the completed snapshot for new chats. Original imported annotations remain available through explicit prepared-source selection, but are rejected for a source that only contains operational inputs.

The UI identifies whether inputs have a matching prepared result. Presence counts are coverage indicators, not a guarantee of answer quality: temporal questions need dates, annotation questions need classifications, and handling time or surveys need their supplied measures.

Intent hierarchy

Intent vocabularies support 10,000 stable leaf IDs and paths of up to ten category levels. path is ordered root to immediate parent. Older category/subcategory entries are displayed as equivalent paths. The compiler retains path metadata and projects deeper levels into the legacy taxonomy's subcategory display for the existing tools.

The wide vocabulary modal stages all edits until Apply. Search retains matching ancestors, and the tree renders only visible rows. Category moves update all descendant paths while preserving intent IDs. Import/export supports an intent array, {intents: [...]}, or the full vocabulary export. Duplicate IDs and invalid paths fail validation. If collaborative annotation settings changed while the modal was open, Apply requires reopening against the current values. Analytics saves and collaboration flushes permit configurations up to 10 MiB; the collaboration endpoint checks the stored agent subtype and tenant before raising its ordinary 1 MB cap.

Larger vocabularies increase annotation prompt size. Model context limits and the configured preparation request budget still apply; the editor's entry capacity does not override provider limits.

The annotation adapter compacts oversized outgoing enum schemas into bounded string fields, while retaining the original Zod parser's exact allowed-ID validation. This avoids provider enum limits without accepting invented labels. Small vocabulary schemas remain unchanged.

Validation

lib/analytics/dataset-inputs.test.ts covers CSV types, duplicate keys, relationships, 10,000-entry tree search/moves/round trips, tenant isolation, stale imports, exclusion of input-only snapshots from chat, and report builds carrying metrics while excluding rows for removed conversations.

Run its database cases only with an isolated ANALYTICS_TEST_DATABASE_URL. Also run the existing analysis integration, authoring and report tests, TypeScript and ESLint. Browser verification should exercise a CSV import, a completed build, exports, a large hierarchy edit, and collaborative save/reopen through the city nginx origin.

Verified on Vanguard on 2026-10-02:

  • All 20 dataset-input, analysis-integration, authoring and report tests passed against an isolated database, with no skipped cases.
  • The browser imported two contact metrics and one operational change, prepared a ready snapshot and exported its 19 tables. A real-model chat correctly reported two contacts and 180 seconds average handling time from 120/240 inputs.
  • A 10,000-intent hierarchy rendered 36 tree rows, supported search and editing, and persisted through collaborative save and page reload without browser errors.

This validates the new import-to-chat path and hierarchy scale. It is not a fresh exhaustive comparison of all three original demo query suites.