Object storage — the provider-agnostic seam¶

Last reviewed: 2026-09-07

All object storage rides one seam per language, speaking the S3 API. There is no MinIO SDK anywhere anymore; portability to another provider (GCS today, anything S3-compatible tomorrow) is configuration, not code.

Language	Package	Backed by
TypeScript	`shared/storage/` → `@trueserve/storage` (packed tarball, `file:` dep)	`@aws-sdk/client-s3` + `s3-request-presigner` + `lib-storage`
Python	`libs/storage/` → `trueserve_storage` (sys.path bootstrap per app, like `libs/knowledge`)	`boto3`
Go	`apps/media-proxy/internal/storage/`	`aws-sdk-go-v2`

Roles and buckets¶

A service never names a bucket in code — it asks for a role:

Role	Env var	Contents	Writers
`recordings`	`STORAGE_BUCKET_RECORDINGS`	`conversations/<id>/{caller,agent,mixed}.wav`, conversation image/document attachments, `whisker-traces/`, `transcriptions/` staging, `workflow-datasets/`	ai-orchestrator, media-proxy, conversation-api, async-agent
`catalogs`	`STORAGE_BUCKET_CATALOGS`	`catalogs/<scope>/<slug>/…` bundles, `plugins/`, `previews/`	catalog-builder, ai-manager (import)
`assets`	`STORAGE_BUCKET_ASSETS`	`surface-configs/images/`, Teams agent icons	ai-manager
`scale-testing`	`STORAGE_BUCKET_SCALE_TESTING`	scale-testing job files + recordings	ai-manager

Buckets are never created at runtime: the seam has no ensureBucket, and the runtime IAM policy deliberately omits s3:CreateBucket/s3:ListAllMyBuckets (a CloudTrail CreateBucket event from a service is a regression).

There are two naming schemes, and which one applies depends on who provisioned the bucket — the name is not a convention you may derive.

Where	Actual bucket names	Provisioned by
`local` — devcontainers + city EC2s (shared buckets, separated by `STORAGE_KEY_PREFIX`) and Kind	`agent-studio-local-<role>`	`scripts/storage/create-buckets.sh`
KMS `dev` / `qa` / `prod` (AWS EKS)	`trusrv-<env>-orch-<role>`	Terraform — `local.resource_prefix = "${customer}-${environment}-${application}"` in `infra/stacks/aws-kms/<env>/main.tf`

create-buckets.sh documents the agent-studio-<env>-<role> pattern for all four envs, but it was never run against the KMS accounts — those buckets predate the seam and live under the Terraform names. On KMS the bucket name is therefore constrained twice over: the bucket must exist under the Terraform-derived name, and that exact ARN must appear in the env's IRSA policy (S3AppBucketsAccess in infra/stacks/aws-kms/<env>/main.tf), which enumerates the app buckets rather than wildcarding a prefix. Editing STORAGE_BUCKET_* in a values-kms-<env>.yaml to a name Terraform doesn't produce breaks that env — twice: the bucket 404s and it is outside the grant. A genuine rename is a three-sided change (values + Terraform + IAM), and it must be verified per env, because these values files are per-env and a fix to one does not touch the others.

This has bitten twice. PR #646 renamed STORAGE_BUCKET_* to agent-studio-<env>-* in all three KMS values files; the follow-up reverted only values-kms-dev.yaml on the assumption that qa/prod were already correct, leaving both pointing at nonexistent buckets. QA surfaced it as NoSuchBucket on every catalog publish, which failed ArgoCD's PostSync catalog-builder-seed-builtins hook — an ArgoCD "Sync failed" on an App that was otherwise Healthy and Synced. grep -rn "agent-studio-" charts/ is the check; only values-kind.yaml's agent-studio-local-* should match. See KMS_DEPLOYMENT_CHANGELOG.md Sessions 112 and 115.

The assets bucket is the newest of the four and was missing everywhere until 2026-09-07: STORAGE_BUCKET_ASSETS had named trusrv-<env>-orch-assets in all three KMS values files since the seam landed, but no stack ever provisioned it and the role was absent from the IRSA grant, so every surface-config image upload and Teams agent icon write failed with NoSuchBucket on dev, qa and prod. module "s3_assets" now provisions it per env alongside the ARNs in S3AppBucketsAccess. It deliberately carries no CORS rules (nothing reaches those objects from the browser — ai-manager streams them through /api/surface-configs/images/<key>) and no lifecycle expiry (an icon must outlive the agent version referencing it).

Object keys are unchanged from the MinIO era and must stay byte-identical: cloned A2UI catalog versions share bundle objects by absolute key, so re-keying silently detaches clones from their bundles.

Configuration surface¶

Every MINIO_*/S3_* alias is gone (CI gate: scripts/storage/check-retired-storage-vocabulary.sh). The canonical vars:

Var	Required	Meaning
`STORAGE_PROVIDER`	no	`s3` (default) \| `gcs` \| `local` (filesystem — tests)
`STORAGE_BUCKET_<ROLE>`	per role used	no default — a missing bucket is a loud failure, never a silent wrong-bucket write
`STORAGE_REGION`	yes (s3/gcs)	always passed explicitly to the client
`STORAGE_ENDPOINT`	no	empty ⇒ AWS S3; `https://storage.googleapis.com` for GCS interop; `http://localhost:9000` for the CI MinIO
`STORAGE_FORCE_PATH_STYLE`	no	`true` for any custom endpoint (MinIO, GCS)
`STORAGE_ACCESS_KEY` / `STORAGE_SECRET_KEY` / `STORAGE_SESSION_TOKEN`	no — keep it that way	absent ⇒ SDK default credential chain (IRSA web-identity on EKS, instance profile, GCS HMAC via secret). Never wrap these in a required-env helper: the absent-key path IS how KMS pods authenticate
`STORAGE_KEY_PREFIX`	no	prepended to every key — separates shared-`local`-bucket users
`LOCAL_STORAGE_PATH`	no	base dir for the `local` provider

Cross-cloud (GCS)¶

GCS is reached through its S3-interop XML API: STORAGE_ENDPOINT=https://storage.googleapis.com, STORAGE_FORCE_PATH_STYLE=true, HMAC keys as STORAGE_ACCESS_KEY/STORAGE_SECRET_KEY (provisioned by charts/trueserve-stack/templates/external-secrets/storage.yaml). charts/trueserve-stack/values-cloud-gcp.yaml carries the wiring.

Because daily dev runs against real AWS S3, the custom-endpoint + path-style path — the only route to GCS — is exercised by the storage-conformance CI job (.github/workflows/test.yml): the same TS + Python conformance suites run against a MinIO service container with STORAGE_TEST_ENDPOINT + path style. That job is the standing guard on the cross-cloud claim; don't remove it to "speed up CI".

Presigned URLs and IRSA¶

A presign signed with STS temporary credentials dies when the credential expires (~1 h) regardless of the window requested. Rules of thumb: attachment presigns are 1 h (conversation-api/storage.py); the catalog-bundle 12 h window survives only on static-credential environments (documented caveat in catalog_manifest.py); never store a presigned URL.

Per-tenant archive sinks¶

Conversation archive sinks carry their own S3-compatible config (endpoint/keys/bucket, tenant data with legacy alias key names — those alias keys are DATA compatibility and stay). Both seams expose a constructor for it (createStoreFromS3CompatibleConfig / create_store_from_s3_compatible_config), and the archive copy is streamed across stores with a server-side CopyObject fast path only when source and destination resolve to the same client.

Legacy bucket names in data¶

DB rows (conversation_attachments.storage_bucket, archive manifests) still carry pre-migration bucket names. roleForStoredBucket/role_for_stored_bucket maps conversations → recordings, trueserve-catalogs → catalogs, trueserve → assets (the objects were synced key-for-key into the role buckets); unknown names pass through as explicit buckets (tenant sinks).