Question discovery from transcripts¶
Status: Draft | Last reviewed: 2026-10-09
This proposal makes transcripts a fourth starting point for Knowledge Management Agents. Use it to connect Call Drivers Analytics with question discovery, knowledge-gap prioritization, and evaluation without repeating transcript preparation.
Workflow¶
Start with the information people actually request:
- Select a prepared Analytics transcript release or upload transcript files.
- Extract explicit questions, contextual follow-ups, and implied information needs.
- Group equivalent requests while preserving product, policy, date, and audience distinctions.
- Review representative questions with their original conversation context and available Call Drivers findings.
- Locate supporting knowledge, definitions, conditions, and exceptions in the selected collection; record missing content and unresolved terminology.
- Promote reviewed requirements into the shared evidence suite, then compare preparation and retrieval changes against them.
Transcripts establish demand and observed interaction behavior. The selected knowledge sources establish support for an answer. A spoken answer can be retained as an observation to check; neither that answer nor a “resolved” classification automatically becomes a verified reference answer.
This path works before the collection has content. Missing support produces a reviewable acquisition requirement rather than invented replacement content.
Inputs and reuse¶
| Entry | Proposed behavior |
|---|---|
| Prepared Analytics dataset | Bind an authorized, ready release and reuse its redacted turns |
| Call Drivers selection | Carry dataset/release, optional annotation snapshot, taxonomy version, filters, and sampling scope |
| Transcript upload | Reuse canonical/Contact Lens normalization and CSV mappings through shared preparation interfaces |
| Later refresh | Discover questions from a new immutable source release and show the changes for review |
A Call Drivers agent is optional. Plain transcript imports can run through the Knowledge Agent's own gated workflow. Reading existing Analytics data retains its feature and access checks; a new Knowledge route must not bypass those checks. Missing optional driver annotations do not block question extraction.
The inspected implementation provides these concrete reuse points:
| Existing code | Reuse | New work |
|---|---|---|
| Normalization and CSV mapping | Conversation IDs, ordered roles/turns, dates, offsets, source provenance | Discovery input adapter and manifest |
| Preparation worker | Redaction before model/index work, complete releases, leases/checkpoints | Shared service boundary; reuse a ready release without re-ingesting it |
| Annotation contract | Driver, intent, root cause, resolution, friction, escalation, evidence | Multiple question/need records per conversation |
| Annotation compiler | Validated structured results and quotes copied from canonical turns | Question extraction schema, full-context coverage, contextual reconstruction |
| Intent hierarchy | Stable vocabulary IDs and editable hierarchy | Question grouping, merge/split review, mappings to knowledge topics |
| Dataset inputs | Versioned operational facts and their coverage | Optional demand-priority signals with explicit provenance |
The current annotation schema contains a conversation summary and classification, not a reusable list of every question. Its existing prompt cannot produce the new contract without a dedicated extraction task. Driver/intent labels provide context and candidate grouping; a call driver is not interchangeable with a knowledge question or policy concept.
The conversation preview endpoint returns at most 100 turns per conversation. Build a bounded, complete turn reader for discovery rather than consuming that preview as the full source. The current annotation compiler also rejects input over its context ceiling. Qualify complete windowing/context reconstruction for long transcripts and report unprocessed ranges; never publish silent prefix-only extraction.
Question and occurrence contract¶
Keep a canonical question separate from its source occurrences. One conversation can contain several needs, and many conversations can express the same need.
| Record | Retained fields |
|---|---|
| Discovery run | Input manifests/releases, filters, time window, sampling method, processing coverage, model/prompt/schema versions |
| Occurrence | Source/release/conversation identity, speaker role, turn ordinals, available timestamps, exact redacted utterance and context |
| Candidate | Normalized question, explicit versus inferred origin, context assumptions, classification, extraction/review state |
| Context | Product/policy/version/date/audience constraints, with observed and inferred values distinguished |
| Observed response | Separate answer-turn references, stated resolution, annotation provenance, unknown or conflicting status |
| Group | Stable question ID, versioned membership, representative variants, merge/split/rejection history |
| Knowledge match | Required facts, source evidence, terminology assumptions, missing support, selected collection version |
| Evaluation case | Approved question/context, optional verified expectation, required evidence, origin and development/held-out assignment |
Use (source, release, conversation, turn) identity for evidence. Turn ordinals
are valid within their normalized release; rebuilding or changing redaction can
change them. Preserve a mapping or explicitly invalidate affected references.
Do not use an LLM-generated quotation as occurrence evidence.
Extract customer requests by default. Human or bot clarifying questions remain context unless the author explicitly selects another role/purpose. Unknown speaker roles and unclear antecedents remain visible uncertainties.
Reconstruct follow-ups such as “Does that include contractors?” using the preceding discussion. Keep the original wording, the proposed standalone question, and the context turns that support the reconstruction. Mark implied needs separately from literal questions and let the reviewer edit or reject them.
Classify each need as knowledge-answerable, account/state-dependent, action request, mixed, or unclear. A mixed request may yield a policy question plus a separate action requirement. Missing account data or a failed transaction is not automatically a missing knowledge article.
Grouping and priority¶
Use existing driver/intent assignments as optional grouping hints. Combine wording, meaning, topic, and applicability context to propose equivalent questions. Keep exceptions, negation, thresholds, and different policy versions distinguishable. An abbreviation match does not establish cross-repository equivalence.
Reviewers merge, split, rename, or reject groups while retaining source occurrences. Connect reviewed groups to the shared terminology/topic model. Call Drivers vocabularies and knowledge vocabularies keep their own IDs and explicit mappings.
Show priority components separately:
- Unique conversations and occurrences, source selection, date range, and volume trend where dates are available.
- Available unresolved/friction/escalation findings, labeled as model findings or supplied operational facts.
- Verified knowledge coverage, retrieval failures, and unresolved mappings.
- Authored business importance and severity, including infrequent policy exceptions.
Pin the source population and show sampled/excluded/unprocessed counts. Repeated wording inside one conversation must not inflate a unique-conversation count. Overlapping imports need declared source identity and deduplication rules. Do not describe a selected sample as the entire contact population.
These signals prioritize investigation. An unresolved interaction alone does not prove a knowledge failure, and fewer calls after publication do not establish a causal effect without a separate measurement design.
Review and evaluation¶
Within Test, Question discovery shows groups and representative transcript evidence beside the normalized question and proposed knowledge requirements. Actions include Keep separate, Merge, Edit question, Find support, Create content gap, and Add to evidence suite.
Preserve a trace from the original request through the reviewed group to the source passages, graph dependencies, retrieved chunks, and answer claims. For the CDC/internal-policy scenario, a transcript can reveal a local term or missing applicability question; the terminology review still needs definition evidence from the relevant repositories.
Reviewing a question can establish a requirement without establishing an answer. Only supported, reviewed expectations contribute to correctness metrics. Keep observed agent responses out of answer-generation context when evaluating whether the knowledge pipeline can independently answer the question.
Measure discovery fidelity separately from RAG quality: missed information needs, invented questions, lost follow-up context, incorrect grouping, and attribution errors. Use labeled transcript fixtures covering the complete conversation, including late turns and multiple requests.
Show both demand-weighted results and results across distinct questions/topics. Declare whether weights count conversations or needs; include the denominator, reviewed coverage, and unmeasured share. Common requests must not hide failures on rare conditions and exceptions.
Split development and held-out cases by related question group and conversation, including near-duplicate wording. A forward time window can additionally test new demand. Held-out questions/answers and their occurrence transcripts must stay out of enrichment and optimization inputs for that experiment.
Refresh, access, and initial scope¶
New transcript releases create new discovery runs. Reuse unchanged extraction artifacts by source content and configuration hash, preserve review decisions, and propose new or changed groups. Existing evaluation suites stay pinned until their author accepts a revision; a refresh does not rewrite an experiment.
Retain Analytics redaction and authorization on transcripts, candidate text, group exemplars, and derived exports. Aggregate counts must respect the selected source scope. A source revocation affects derived evidence and drill-down as well as the original transcript. Pin real/synthetic provenance and report them separately.
Text transcript uploads and reuse of a prepared Analytics release belong in the first useful Vanguard release. Shared transcription adapters can extend that path to supported audio after qualification. Full video analysis remains in the media milestone.