Knowledge Management Agents¶

Status: Draft | Last reviewed: 2026-10-09

This proposal defines a Knowledge Management Agent that connects source content, terminology, topics, and policy evidence, measures retrieval and reasoning quality, and publishes tested knowledge releases. Read it to agree on the product and implementation boundaries before building the first Vanguard release. The APIs, schemas, screens, and flags described here are proposed.

Direction¶

Add Knowledge Management to the existing Agents repository. Give it an editable, forward-only preparation canvas, a retrieval-and-answer canvas, and shared Test and Health workspaces. Start with native RAGFlow, as confirmed for this plan, including its built-in graph capabilities. Qualify those capabilities before considering a separate graph database. Keep prepared content independent of its destination.

The agent should help you answer five practical questions:

What can the available content answer?
What content and evidence are missing for the questions you need to answer?
Which processing or retrieval change improves those answers?
Which definitions, conditions, exceptions, and source versions apply together?
Is the combined index/graph effective, current, and good enough to publish?

Treat this as a knowledge preparation and evaluation product. A prompt chain alone does not account for source refresh, document structure, retrieval behavior, or the evidence needed to explain a result.

Requirements and working decisions¶

Area	Direction
Availability	Server runtime opt-in, enabled only in Vanguard for the initial rollout
First destination	Native RAGFlow; use its native graph/compilation engine first
Sources	Multiple sources feed one corpus and one logical index
Preparation	Editable steps, branches, explicit merges, and no execution cycles
Question inputs	Content-generated Q&A, supplied Q&A, questions only, and requests extracted from transcripts
Semantics	Topic modeling, terminology crosswalks, and AI-populated evidence graphs
Evaluation	Multi-source synthesis, policy reasoning, evidence retention, and operational cost
Health	Coverage, mappings, graph support, version consistency, and measured task effectiveness
Query behavior	Configurable rewrite, filtering, retrieval, reranking, and answer models/prompts
Formats	Structural HTML/Markdown, document parsing, tables, and selective OCR
Media	Preserve a path to image, audio, and video processing and citations
External targets	Bedrock, Google data stores, and Salesforce assessed through separate adapters
Publication	An immutable tested release; incomplete work stays out of live retrieval

The first rollout flag is proposed as FEATURE_KNOWLEDGE_AGENTS, default off. Use the runtime-only Scene/Analytics pattern; keep the opt-in out of shared environment templates. This is separate from the existing per-tenant Knowledge Sources connection and must not revive its retired environment-variable fallback.

The product model¶

Object	What you control
Knowledge Agent	Identity, versions, preparation graph, query graph, test policy
Knowledge collection	Reusable sources, document identities, metadata vocabulary, access rules
Build	Frozen source snapshot and processing configuration, with inspectable outputs
Knowledge model	Versioned topic hierarchy, terminology mappings, and source-backed assertions
Index release	A destination's vector, keyword, or graph representation of a completed build
Publication	Agent, knowledge model, index/graph releases, query behavior, and evaluation evidence
Evidence suite	Versioned questions, scenarios, comparison tasks, required evidence, and assertions

Use Collection in the UI and corpus in internal contracts. A collection can serve several agents without repeating source acquisition. Agent versions retain authored settings; large documents, run outputs, and test results live outside configuration JSONB.

Two canvases keep the different execution lifecycles understandable:

Prepare: source snapshot → parse → route → transform/split → topics, terminology, assertions and metadata → validate → candidate indexes/graph.
Answer: request → term/context resolution → allowed filters → hybrid retrieval and graph expansion → evidence assembly → supported synthesis.

Test compares them together. Evaluation may propose an edit, but applying that edit creates another build or query revision. It never introduces a backward execution edge into the preparation graph.

flowchart LR
  Sources["Files, websites, buckets, Salesforce"] --> Snapshot["Versioned source snapshot"]
  Snapshot --> Parse["Parse and preserve source locations"]
  Parse --> Route{"Document strategy"}
  Route --> Structure["HTML and Markdown structure"]
  Route --> Layout["PDF, tables, and OCR"]
  Route --> Media["Transcript and visual evidence"]
  Structure --> Prepare["Transform, split, enrich"]
  Layout --> Prepare
  Media --> Prepare
  Parse --> Semantics["Topics, terminology, and supported assertions"]
  Prepare --> Bundle["Canonical knowledge bundle"]
  Semantics --> Bundle
  Bundle --> Candidate["RAGFlow vector and keyword index"]
  Bundle --> Graph["RAGFlow native knowledge graph"]
  Candidate --> Query["Retrieval, graph expansion, and synthesis"]
  Graph --> Query
  Transcripts["Transcripts or Call Drivers release"] --> Discover["Extract, group, and review questions"]
  Discover --> Questions
  Questions["Questions, scenarios, and comparison tasks"] --> Evaluation["Pinned evaluation run"]
  Snapshot --> Evaluation
  Bundle --> Evaluation
  Query --> Evaluation
  Evaluation --> Release["Publish tested version and index"]

The diagram source is kept alongside this proposal. The index and knowledge graph are logical outputs of the same native RAGFlow backend. Media and external adapters are later milestones, not first-release claims.

Four ways to begin¶

Start from content. Connect or upload content, inspect the parsed sources, generate candidate questions with answers and source evidence, and review the useful cases. Compare processing strategies against those cases.

Start from questions and answers. Import the questions and reference answers. Find supporting source passages and identify contradictions or gaps. Describe the required retrieval unit: for example, a table row together with its headers, product version, qualification notes, and effective date. Test that requirement against actual retrieved results.

Start from questions only. Record the required facts and search the current collection for evidence. Show “answer not established” until evidence or a reviewed reference answer exists. An apparently plausible model answer does not establish correctness.

Start from transcripts. Import conversations or select a prepared Call Drivers dataset release. Extract explicit questions and contextual information needs, group variants, and prioritize them using observed demand and available resolution/friction signals. Find supporting knowledge or create a content gap, then add reviewed requirements to the evidence suite. A transcript answer remains an observation to verify. See question discovery.

All four paths converge on the same source-linked evidence suite. Also support scenario interpretation, policy comparison, terminology alignment, and change impact tasks without requiring a single short reference answer. Generated cases are candidates; reviewed reference cases and held-out cases retain distinct status.

Main acceptance scenario¶

Connect selected live CDC material and an organization's internal document repository. Build a reviewed terminology crosswalk and source-backed graph so that an analysis can follow definitions, conditions, exceptions, and citations across both collections. Track which internal provisions may be affected when an external source changes.

The connected policy knowledge design defines this scenario, source-authority handling, topic modeling, graph extraction, and health metrics. Similar language is a mapping candidate; the system must preserve differences in meaning, scope, and authority.

For the first public benchmark, use the travel and sponsored-project policy corpus. It combines real organizational policies with federal references, with 12 core sources and four optional fixtures. It exercises these contracts before private content is available; retain the CDC scenario for domain qualification.

What already exists¶

The repository assessment records inspected files and revisions. The most consequential findings are:

The Analytics editor supplies the visual shell, but its canvas is currently a fixed non-editable sequence. The new graph needs its own typed model and editor.
Workflow Decisions already have visual rules, optional AI assessment, and validated routing results. Reuse that authoring and assessment behavior.
Analytics preparation already demonstrates immutable releases, leased work, encrypted retained sources, and separate serving snapshots.
Call Drivers supplies normalized transcripts, intent/driver findings, and turn-linked evidence. Multiple-question discovery adds a dedicated extraction task and review contract around those inputs.
Tooling has crawl4ai acquisition, RAGFlow source/connector management, GraphRAG generation and query settings, and Gallery media processing. These are reuse seams.
Existing source operations also initiate indexing. They need an acquisition boundary before they can feed an arbitrary TrueServe preparation graph.
Tooling's knowledge-core implements wiki/CRM ingestion patterns. It is not an existing general-purpose knowledge DAG executor.

SCM inspection succeeded. The Tooling deployment revision was not verified: the current AWS role accepted a read-only SSM command but denied access to its result. The proposal does not treat repository HEAD as deployment evidence.

Decisions that make the design work¶

Preserve original evidence. Keep raw source bytes and a structured document representation with page, element, table-cell, or media-time references. Summaries, alternate phrasings, generated questions, and extracted metadata are derived material with their own provenance.

Separate retrieval text from evidence. You may improve discovery using context headers, summaries, or alternate questions. The answer still needs support from the source. If a destination cannot preserve that distinction, the adapter exposes the limitation and the pipeline chooses a supported mode.

Preserve connected policy meaning. Model definitions, applicability, conditions, exceptions, and cross-references with source support. Graph fields and terminology mappings are versioned, inspectable extraction/review artifacts. Topic summaries help discovery; they do not replace clause-level evidence.

Use the native graph engine first. Reuse RAGFlow's graph extraction, retrieval, and compilation where the selected version meets the contract. TrueServe adds authoring, review, evaluation, health, and publication controls. The provider assessment defines the checks before introducing another graph store.

Make branching precise. A route sends an item down one matching path; a fork creates multiple derived views. A merge declares how those views combine. Iteration over documents or chunks is worker behavior, not a canvas loop.

Measure more than answer similarity. A failed answer may come from absent source material, damaged extraction, a bad split, a filter, ranking, or answer generation. The Test workspace should show the earliest evidenced failure and the artifacts needed to investigate it.

Give each output one owner. A pipeline-owned RAGFlow dataset must not also have an independent connector reparsing or replacing its chunks. Existing provider-managed sources remain available through their existing tools.

Publish complete generations. Indexing runs against a candidate generation. Only a verified, evaluated generation becomes active. Refresh, deletion, and access changes are part of this lifecycle.

First useful release¶

Deliver one complete Vanguard path with a small policy schema:

Combine uploaded files/folders and a website into a collection.
Parse HTML, Markdown, text, and PDFs; route scans through an explicit OCR strategy and retain table structure.
Edit a small graph with deterministic routing, structural splitting, a prompt transform, typed metadata extraction, and an explicit merge.
Assign/discover topics, propose terminology mappings, and extract supported definitions, cross-references, conditions, and exceptions using native RAGFlow graph/compilation capabilities qualified in milestone 0.
Inspect the outputs and publish prepared units with their verified native graph in one RAGFlow release.
Author/import questions and rich evidence tasks; generate candidate Q&A and discover questions from text transcripts or an existing Analytics release.
Test hybrid retrieval, graph expansion, and synthesis with selectable models, prompts, filters, and visible evidence paths.
Compare two versions and inspect Health by topic/source. Demonstrate source refresh, terminology ambiguity, change impact, and removal without exposing a partial or mismatched index/graph.

Start with the public policy corpus, then qualify selected CDC pages with synthetic or authorized internal policies. Include differing terms, an indirect definition, an exception in another document, an older source version, and a deliberately ambiguous mapping. These are proposed acceptance fixtures, not measured results.

Delivery boundaries¶

Milestone	Result
0. Contract proofs	Native RAGFlow graph/index capability proofs and public-policy evidence fixtures
1. Authoring and execution	Gated subtype, typed graph, semantic schema, inspection, durable sample runs
2. First useful release	Topics, terminology, graph-assisted retrieval, tests/health, publication and refresh
3. Broader ingestion	S3/GCS, Salesforce acquisition, bulk scale qualification, richer document strategies
4. Destination adapters	Bedrock first, then Google and Salesforce with documented capability limits
5. Media and optimization	Video/audio pipelines, visual evidence, bounded parameter experiments

Evaluation starts with milestone 0 fixtures and ships in milestone 2. It is not deferred until every connector or modality is available. Work packages, acceptance criteria, and dependencies are in the delivery plan.

Open decisions¶

The following need evidence or product input during implementation, but do not prevent planning the first RAGFlow release:

The exact RAGFlow version and extension boundary in the native migration.
The first policy ontology, terminology-review ownership, and native graph controls.
Representative corpus sizes, largest files, change rate, and concurrent users.
Which tenant access rules the first connected sources need to preserve.
The acceptable quality thresholds and who reviews reference answers.
Which external provider should follow Bedrock and whether preview features are acceptable for that provider.

Start with manual publication and explicit refresh schedules. Add automatic promotion only after quality thresholds, completeness rules, and rollback have been demonstrated on the selected collection.