Skip to content

Cost estimation and usage attribution

Status: Current | Last reviewed: 2026-09-11

TrueServe forecasts agent and workflow cost before publish, then prices durable usage evidence after execution. Forecasts and observed totals share selectors, meters, price books, and decimal arithmetic, but they are deliberately separate: an estimate changing must never rewrite the history of a completed run.

User surfaces

GPT Live forecasts a speech_engine/session_second meter for the voice stage and separate llm token meters for <stage-id>:delegation. Runtime evidence uses Live's cumulative session seconds and each managed backend response's actual model, response ID, tokens, cache usage, and tier; client workers use their own service telemetry. The public seed includes the Live session rate verified on 2026-09-11. Hosted web-search fees remain outside this forecast. See GPT Live.

  • Agent Settings → Cost estimate shows low / typical / high cost per chat conversation, or per voice/translation minute and hour.
  • Workflow Settings → Cost estimate shows cost per execution. Batch workflows also show per batch and per item.
  • Conversation transcripts and workflow run details show observed provider cost.
  • Settings → Tenant Settings → Costing imports and publishes price books, reports selector coverage, publishes workload defaults, proposes measured-data calibration changes, and reconciles usage with provider billing exports.

All values are provider-cost estimates, not customer prices or invoices. Partial, unsupported, missing, and stale coverage are explicit states; the UI never turns an unknown meter into zero cost.

Forecast model

The pure calculator lives in apps/ai-manager/lib/costing/. It accepts a versioned configuration, an assumption set, and price-book context. It returns decimal strings, components, warnings, coverage, and the IDs required to replay the calculation.

Chat uses 15 / 22 / 30 turns by default. Input tokens include cumulative history: for n turns the calculator sums the system prompt on each call, user messages through the current turn, prior assistant messages, and expected tool results. Cached input is split from ordinary input. Prompt text without a tokenizer uses the documented four-characters-per-token approximation and raises a warning.

Voice is normalized to one connected minute. Speech ratios first determine the active caller and agent speech time within that minute. Realtime token-priced audio then uses:

audio tokens = active audio seconds × provider/model audio tokens per second

Audio-token cadence is a shared, editable provider/product/model profile rather than a low/typical/high workload variable. Exact selectors override wildcard profiles. OpenAI Realtime defaults to 10 input and 20 output audio tokens per second: 600 input or 1,200 output tokens per active audio minute. This follows OpenAI's official audio-token-to-minute conversion. Nova Sonic uses a separate 25 input and 50 output speech tokens per second profile. The input cadence is inferred from AWS's published $0.27/hour input-audio example at $3/million input speech tokens. AWS does not publish the output codec cadence; 50 output tokens/second is an explicit planning assumption and exact usageEvent counts replace it after execution.

Caller and agent speaking rates remain independent. The low / typical / high defaults are 120 / 150 / 180 caller words per spoken minute and 130 / 160 / 180 agent words per spoken minute, but WPM now affects transcript text, TTS characters, and translation characters only. It never derives audio tokens.

Nova Sonic does not publish a cached-speech price dimension, so its forecast does not split speech into cached and uncached meters. At the typical 40% caller and 35% agent speech ratios, the model projects 600 input and 1,050 output speech tokens per connected minute. At Nova 2 Sonic's current US East rates, that is $0.864/hour for speech. Continuous speech in both directions would be $2.43/hour under the same coefficients; normal half-duplex conversations are lower. This lands inside AWS's own approximately $0.30–$0.60 per 30-minute active-session example.

Nova text projection is itemized separately because AWS also charges text tokens for transcripts, tool activity, grounding, and context. The model follows AWS's tool configuration, chat history, and usage event contracts. It includes:

  • configured prompt text at four characters per token;
  • enabled tool schemas at 100 / 200 / 400 tokens per runtime tool. Multi-tool built-in capabilities are expanded to their real schema count; capabilities such as message injection that register no LLM-callable function add none;
  • expected tool calls per turn at 0.05 / 0.2 / 0.6, with separately editable argument and result sizes;
  • caller and assistant speech transcripts at 1.2 / 1.33 / 1.5 text tokens per spoken word, including two default billable assistant transcript passes in the typical and high cases; and
  • cumulative transcript and tool history replayed when a provider session is continued. Defaults project a 5 / 15 / 60 minute session and a six-minute continuation interval.

Each value is editable at platform, tenant, or individual-agent scope. Exact Nova usageEvent.details.delta speech/text counts replace the forecast after execution.

STT and time-priced speech products continue to use speech seconds. Character- priced TTS and translation use the applicable caller/agent words per minute multiplied by the configurable six-character-per-word default. Translation traverses incoming and outgoing flows independently.

Workflow reach probabilities are propagated through the DAG. Parallel branches all execute; router branches are equal-probability until observed branch data is available. Retry probability uses 1 / (1 - p). For Each bodies, reasoning-chain steps, tool iterations, nested inline workflows, input-item counts, and batch occupancy multiply the reachable call count. Cycles are bounded by the loop-count assumption and warned. max_total_llm_calls caps token projections.

Assumptions are immutable published defaults plus an optimistic-locking cost_subject_profiles overlay. Authors edit only the selected agent/workflow; new platform defaults continue to flow to subjects that have no override. Tenant and platform administrators create effective-dated assumption drafts in the Costing screen and publish them after review. Publishing never mutates an older set or a subject's explicit pinned profile.

Price books

cost_price_books has public-retail or tenant-contract scope. Immutable cost_price_book_versions carry effective dates, source checksum, verification date, review deadline, and lifecycle. cost_rates resolves:

provider + product + model + region + service tier + meter + unit

Tenant contract rates win over public rates; exact model, region, and tier win over *. Billing minimums and increments apply before price_per_unit / unit_quantity. Mixed currencies are excluded from one estimate. Exactly one version is active per public or tenant scope at an instant; publishing is serialized and chronological, and switches the whole scope even when the new version uses a different book name.

The versioned reviewed catalog in public-pricing.ts covers the enabled OpenAI APIs/models, direct Claude models, AWS Bedrock models, Nova 2 Sonic, AWS Transcribe, and ElevenLabs speech products. It was verified on 2026-08-27 against the providers' official pricing pages and the AWS Price List API. npm run costing:sync-prices loads this complete catalog by default.

Manual imports use normalized JSON. The AWS importer uses the Price List API in us-east-1, maps provider display names back to runtime model IDs, separates standard/batch/flex/priority dimensions, preserves Bedrock speech-token dimensions as audio-token meters, and creates a reviewed draft. Run npm run costing:sync-aws for that AWS-only refresh. Use COST_PRICE_SOURCE=normalized_json plus COST_PRICE_RATES_FILE for a separately reviewed provider or contract snapshot. Because publishing switches the complete scope, do not auto-publish an AWS-only import over the multi-provider public catalog.

COST_PRICE_AUTO_PUBLISH=false is the safe default. Unchanged imports re-verify the existing version instead of creating duplicates; changed imports return added/changed/removed counts for review. Publishing retires the preceding effective version.

Reviewed pricing conversions

  • OpenAI text meters use the official per-million-token rates. The same model prices support Chat Completions, Responses, Assistants, and other endpoints for which OpenAI does not add an API-specific fee.
  • OpenAI Realtime audio retains the exact input, cached-input, and output per-million-audio-token prices. Workload assumptions project token quantity; they are not baked into the price rate.
  • gpt-4o-mini-tts combines its exact text/audio token prices into a character planning rate calibrated at 160 spoken words/minute and six characters/word. The configurable agent speaking rate determines projected characters; observed provider tokens supersede the planning conversion.
  • ElevenLabs uses $0.05/1K characters for Flash/Turbo and eleven_v3_conversational, $0.10/1K for Multilingual v2 and eleven_v3, $0.39/hour for realtime Scribe, $0.08/connected minute for Speech Engine, and $0.12/source minute for Voice Changer. The v3 Conversational selector is the low-latency Text to Dialogue model; it remains distinct from the standard v3 rate. The legacy eleven_monolingual_v1 option uses the current standard TTS rate and is explicitly marked as a fallback.
  • Nova 2 Sonic text rates are exact regional AWS Price List dimensions. In us-east-1, input is $0.33/million tokens and output is $2.75/million tokens. AWS publishes no cache-read dimension, so cached input deliberately uses the ordinary input rate rather than becoming unpriced.
  • AWS prices Nova 2 Sonic speech at $3/million input speech tokens and $12/million output speech tokens in us-east-1. TrueServe retains those exact rates and projects quantities from active audio duration and its configurable provider/model tokens-per-second profile. Nova's separately priced text meters include prompt/tool/history traffic as described above. Observed provider usage deltas supersede every projection.
  • The default Textract TABLES + FORMS projection adds the two first-tier feature prices ($0.065/page). Managed Bedrock Agents are marked unsupported for forecast pricing because the underlying foundation model is configured only in AWS; execution can still record its provider-reported model usage.

Coverage compares every model option in the service-definition registry with active public plus tenant rates. A newly enabled model therefore becomes an operationally visible gap rather than silently pricing at zero.

Durable observed usage

libs/costing/usage.py is shared by the voice/chat and workflow runtimes. It creates one positive row per meter with a stable SHA-256 idempotency key. Each row records tenant, conversation or workflow run, agent or node/stage, tool-loop iteration, provider request ID, measurement source, selector, quantity, and the active public/tenant price-book version IDs.

The voice runtime measures even when OTEL export is disabled. It persists service summaries in tracked background tasks so database latency is outside the media path. Nova Sonic's incremental usage events are captured as exact input/output speech and text token meters. The workflow runtime records every successful LLM iteration, including cached-token metadata, in the same way. Provider-reported values take precedence; runtime-measured audio/characters are explicitly labelled.

Observed APIs reprice each aggregate only with the versions stamped on its rows. That makes a completed conversation reproducible after catalog updates. They also compare the observed total with the matching immutable publish snapshot, including absolute/percentage variance and whether the result falls inside the saved low/high range.

Set FEATURE_COST_USAGE_LEDGER=false to stop runtime quantity writes without changing OTEL. Set both FEATURE_COST_ESTIMATES=false and NEXT_PUBLIC_FEATURE_COST_ESTIMATES=false to gate the APIs and authoring UI.

Publish snapshots and reconciliation

Agent and workflow publish transactions also insert a forecast_publish snapshot containing config hash, calculator version, assumptions/profile revision, price versions, components, totals, coverage, and warnings. The unique publish key makes retries idempotent.

Reconciliation selects the best historical rate for every usage event in a provider/time window, applies billing increments, and compares that estimate with an uploaded invoice/CUR total. Store provider allocation keys in event metadata. For Amazon Bedrock, application inference profiles and cost-allocation tags are the preferred correlation mechanism; CUR 2.0 is the provider-cost input.

Investigate variance in this order: missing selector coverage, wrong region/tier, cached-token mapping, request-level minimum/increment semantics, unattributed shared endpoints, then delayed provider adjustments.

APIs

Route Purpose
GET/POST /api/pipelines/:id/cost-estimate Saved or in-editor agent forecast
GET/POST /api/workflows/:id/cost-estimate Saved or in-editor workflow forecast
GET/PUT /api/costing/subjects/:type/:id/assumptions Resolved assumptions and subject override
GET /api/conversations/:id/cost Observed conversation cost
GET /api/workflows/:id/runs/:runId/cost Observed workflow-run cost
GET/POST /api/admin/costing/price-books List/import catalog versions
GET /api/admin/costing/price-books/:versionId Paginated normalized-rate review
POST /api/admin/costing/price-books/:versionId/publish Publish reviewed prices
GET/POST /api/admin/costing/assumptions List/create default assumption sets
POST /api/admin/costing/assumptions/:id/publish Publish reviewed assumptions
GET /api/admin/costing/calibration Measured distributions and draft proposal
GET /api/admin/costing/coverage Active selector coverage
GET/POST /api/admin/costing/reconciliations Reconciliation history/run

Agent, workflow, and configuration permissions gate the matching routes. Every database operation runs in tenant context; only platform administrators can write the public-retail book.

Tests and operations

Run:

cd apps/ai-manager
node --import tsx --test lib/costing/costing.test.ts
npm run typecheck

PYTHONPATH=libs python -m pytest -q libs/costing/test_usage.py

Schema is migration 044_add_cost_estimation.sql. The schema drift job, migration idempotency, route-auth audit, pricing unit tests, and runtime meter tests are release gates. Alert when coverage drops, a review date passes, background usage writes fail, observed cost stays empty on active traffic, or reconciliation variance exceeds the agreed threshold.

The Costing screen's calibration report groups measured quantities by agent or workflow, provider, model, and meter. It computes p10/median/p90 values over the last 30 days and proposes low/typical/high changes only after the minimum sample count is reached. The proposal shows sample size and quantity impact, loads into the normal draft editor, and always requires human publish approval.