Public policy test corpus¶
Status: Draft | Last reviewed: 2026-10-09
Use University of Minnesota travel, expenses, and sponsored-project policies, paired with GSA references, as the first public Knowledge Agent test collection. This document identifies a small downloadable corpus and the reasoning tasks it can test before you have an authorized internal document repository.
Recommended collection¶
Start with 12 core items. Add four optional items for long-PDF parsing, historical rates, and question discovery. The organizational policies are real policies published on the university's public website; they stand in for the internal side of the CDC/internal-repository scenario.
The collection contains governing policies, appendices, procedures, worked examples, federal guidance, and structured rate tables. Its explicit references give you useful graph relationships without inventing connections between unrelated policy collections.
The source manifest records exact URLs, observed content types, retrieval times, byte sizes, SHA-256 hashes, source roles, and edition labels. It is a research seed manifest, not an implemented connector configuration or a reviewed answer key.
Core sources¶
| ID | Document | Useful test |
|---|---|---|
| T01 | Traveling on University Business | Governing rules, definitions, cross-references |
| T02 | Reimbursable / Non-reimbursable Travel-related Expenses | Expense table, qualifications, exceptions |
| T03 | Mileage, Lodging, and Meals & Incidentals Examples | Worked scenarios versus general rules |
| T04 | Business Expenses | Allowability and reimbursement context |
| T05 | Hospitality and Entertainment of Non-Employees | Meal purpose, participants, funding |
| T06 | Using Vehicles for University Business | Driver, vehicle, and trip distinctions |
| T07 | Direct Costs on Sponsored Projects | Funding-specific conditions and authority |
| T08 | Charging Direct Costs to Sponsored Projects | Policy-to-procedure references and cost categories |
| T09 | GSA Fly America Act guidance | Carrier terminology and conditional exceptions |
| T10 | GSA FY2027 per diem rates | Spreadsheet rows, locations, seasonal dates |
| T11 | GSA M&IE breakdown | DOCX table, headers, explanatory notes |
| T12 | GSA Per Diem Bulletin FTR 27-01 | Rate-year scope and effective interval |
T01–T09 and T12 are HTML. T10 is XLSX; T11 is DOCX. All 12 returned the expected public content during direct HTTP checks on October 9, 2026. These checks do not establish that the production crawler, parsers, or RAGFlow adapter can ingest them correctly.
The GSA download page lists T11 as “FY 25-Present”; the document itself is labeled FY2025. Retain both observations. T12 states the FY2027 travel interval as October 1, 2026 through September 30, 2027. Do not infer a university policy's effective date from its download time or the website's “Printed on” banner.
Optional extensions¶
| ID | Document | Role |
|---|---|---|
| P01 | 2 CFR Part 200, 2025 annual edition | Long, cross-referenced federal rules PDF |
| V01 | GSA FY2026 per diem rates | Historical rate and schema-change fixture |
| A01 | Traveling on University Business FAQ | Candidate question discovery and source comparison |
| A02 | Hospitality and Entertainment FAQ | Candidate question discovery and boundary cases |
All four optional downloads also returned the expected content.
P01 is an explicitly dated 2025 edition, not a claim about current law. Use its travel-cost provisions, definitions, and referenced conditions in a separate version-pinned experiment. Validate current applicability before promoting any answer to a current-policy reference set.
V01 is useful even before comparing rate values: the downloaded FY2026 workbook
has an ID column and a Master sheet; FY2027 uses Sheet1 and has no ID
column. Both include location and season fields. A refresh must detect changed
headers and preserve row meaning.
Keep FAQ-derived cases identifiable. If an FAQ answer remains in the searchable collection, the resulting test measures retrieval of published answers. Use independently authored scenarios for synthesis evaluation, and keep held-out questions and reference answers out of enrichment and tuning inputs.
Authority and terminology¶
Preserve the distinction between the following source roles:
- University policies establish organizational rules within their stated scope.
- Appendices and procedures have explicit governing-policy relationships.
- Worked examples illustrate scenarios; their numbers are not a current rate feed.
- Federal references apply when the scenario and organizational rules make them relevant. A GSA rule for federal employees does not automatically govern a university employee.
- Sponsor-specific terms may be required to answer a grant question. Public general policies cannot establish an absent award's conditions.
Do not combine the corpus into one undifferentiated authority level. A passage's scope, source relationship, and version are part of the evidence for an answer.
Use these as terminology-review candidates, not preapproved equivalences:
| Terms | Review task |
|---|---|
| “M&IE,” “meals and incidentals,” “meal allowance” | Match the relevant concept while retaining calculation and scope |
| “Per diem,” “actual expenses,” “reimbursement” | Preserve distinctions between allowance, incurred cost, and payment |
| “Allowable,” “allocable,” “reasonable” | Keep separate funding conditions |
| “Travel meal,” “hospitality,” “entertainment” | Resolve purpose and participants before selecting rules |
| “Sponsored driver,” “sponsored project” | Reject a misleading shared word |
| Carrier, operator, ticket flight code | Preserve the details needed for a Fly America question |
Begin with topics such as travel booking, meals, vehicles, funding restrictions, documentation, and exceptions. Extract explicit policy-to-appendix/procedure links first; add supported conditions and exceptions with source anchors and review state. Assess native RAGFlow graph retrieval against the same hybrid-only baseline instead of assuming a graph improves every question.
Draft evaluation scenarios¶
The cases below are questions and evidence requirements, not verified answers. Review the exact source passages and freeze the relevant versions before turning a case into a reference test. Add dates, location, traveler role, funding, and other missing context deliberately rather than having a model invent them.
| Case | Request | Required test |
|---|---|---|
| Q01 | “My hotel includes breakfast and the conference provides lunch. How do I calculate meals on arrival day?” | T01/T02/T10/T11; distinguish the applicable organizational rule, table notes, and first-day condition |
| Q02 | “I am adding two personal days to a work conference. Which expenses need separate treatment?” | T01/T02/T04; assemble conditions across booking, business purpose, and expenses |
| Q03 | “Can I charge this foreign-airline ticket to my federal grant if a U.S. airline operates the flight?” | T01/T07/T09; request ticket/carrier and funding details before applying an exception |
| Q04 | “Is dinner with a visiting research collaborator a travel meal or hospitality, and can the grant pay?” | T01/T05/T07/T08; resolve terminology and identify missing sponsor terms |
| Q05 | “Can I claim the entire drive from home to a business event?” | T01/T03/T06; preserve scenario details and avoid generalizing from one worked example |
| Q06 | “Which rates apply to a trip spanning September 30 and October 1, 2026?” | T10/T12/V01; select the rate year for each date and preserve location/season columns |
| Q07 | “GSA mentions an actual-expense exception. Does that give me the same approval at the university?” | T01/T02/T12; test source applicability instead of importing an external permission |
| Q08 | “The expense is allowable. Does that mean it can be allocated to any of my grants?” | T07/T08; retain distinct funding conditions and identify the missing award context |
| Q09 | “Which travel procedures and guidance should be reviewed if the sponsored-travel rule changes?” | T01/T02/T07/T08 and optional A01; return supported dependencies, with inferred impact labeled |
| Q10 | “What does the 2025 federal travel-cost provision require for this scenario?” | P01 plus the explicitly selected university snapshots; cite section and edition, and avoid a current-law claim |
Add a deliberate insufficient-evidence case: a named sponsor's private award conditions are absent from the collection. The answer should identify the missing source rather than manufacture an approval.
Create a small, separately labeled synthetic transcript pack from several of these scenarios to exercise question discovery. Include a follow-up, two paraphrases, and an incorrect observed agent answer. Synthetic occurrence counts do not establish real customer demand.
For health, measure required-evidence coverage, condition/exception retention, terminology mistakes, version and authority mistakes, and unsupported claims. Compare the same pinned scenarios across hybrid retrieval, term expansion, and graph expansion. Report failures by topic and document strategy.
Acquisition and parsing checks¶
Use the exact manifest URLs as an allowlist; do not start with an unrestricted crawl of the university or GSA domains. Retain original bytes and source URLs alongside parsed output.
| Input | Inspection requirement |
|---|---|
| Policy HTML | Separate navigation and repeated banners from policy body; retain definitions, history, tables, and related links |
| Expense appendix | Keep each row with column headers and applicable notes |
| Rate XLSX | Preserve sheet/row coordinates, location fields, season bounds, and fiscal year |
| M&IE DOCX | Keep table headers, amounts, and explanatory qualifications together |
| Federal PDF | Preserve section/page anchors and inspect both early and late sections |
| Converted Markdown/HTML | Compare retained evidence against the original; conversion must not erase conditions |
Generate a small OCR fixture later by rasterizing selected pages of P01. Label it as a synthetic scan derived from the original, retain the exact page mapping, and keep it separate from the PDF retrieval baseline. The selected originals do not themselves demonstrate OCR, video, or large-volume ingestion support.
Store source_id, organization, source role, governing-policy links, edition,
reviewed effective interval, retrieved_at, original URL, raw hash, content hash,
and source locators. Compute the content hash from a controlled extraction so
that a changed print banner does not look like a changed policy. Preserve the
raw hash independently.
A refresh creates a new candidate snapshot and a visible diff. It must not silently replace the corpus used by an existing evaluation. Keep original acquisition failures in the manifest; HTTP 200 alone is insufficient.
The following candidates are excluded from the verified collection:
- The current eCFR travel-cost page redirected to a request-access page from this environment, despite returning HTTP 200. Qualify authorized acquisition separately; do not ingest the access page.
- A candidate GovInfo 2026 Part 200 URL returned its error page with HTTP 200. This does not prove that a 2026 edition is unavailable elsewhere.
- A University of California travel-policy URL encountered a redirect loop. It is unnecessary for the initial collection.
Raw downloads are research artifacts outside Git. The checked-in manifest contains URLs and provenance, not redistributed copies of the source documents. Before preparing a shared immutable dataset, record the applicable reuse terms for the third-party material.
Place in the delivery plan¶
Use this collection for the first M0 acquisition, parsing, terminology, graph, and evidence-suite proofs. It does not establish performance at large volume or fitness for insurance and health policy reasoning.
Retain the CDC/internal-policy scenario as a domain qualification case. Once the public travel baseline works, replace the organizational side with approved internal material and test the same authority, terminology, version, and multi-document evidence contracts.