Skip to content

catalog-builder

Status: Current | Last reviewed: 2026-10-01

Compiles A2UI component TypeScript into deployable bundles and publishes them to object storage.

The devcontainer launcher waits for PostgreSQL, without requiring local MinIO. City defaults use S3 through STORAGE_BUCKET_CATALOGS and STORAGE_REGION; configured object operations establish whether that bucket is accessible. The conversation archive worker follows the same storage convention and no longer spends a minute waiting for an unused MinIO endpoint.

Published object paths include <semver>/<publication-uuid>/ for bundles, CSS, source archives and plugins. Different city databases may share a bucket and publish the same tenant/slug/version; the UUID keeps those builds from overwriting each other's integrity-checked bytes. Existing stored paths and cloned version references remain valid. A CSS-only repair also changes the conversation manifest ETag through its CSS URL and integrity fields. The separate Auto Club Group catalog contains ACGAddDriverForm and ACGAddVehicleForm. It uses the standard catalog registration and show-component tool contracts, with scoped ACG tokens from the supplied brand assets. It is a tenant-owned multimodal catalog, not an automatically cloned baseline. Its field definitions, CSS, and logo also generate the standalone Agentforce demo's assets.

Source

apps/catalog-builder

Runtime

Aspect Value
Language TypeScript (Node)
Bundler ESBuild
Port 8090

What it does

  1. Receives a /publish command from ai-manager (or code-server terminal).
  2. Archives the catalog source as source.tar.gz.
  3. Compiles *.tsx components to a single bundle.js.
  4. Generates a manifest.json of components, props, and defaults.
  5. Uploads source + bundle + manifest to object storage.
  6. Writes a2ui_catalogs, a2ui_catalog_versions, a2ui_tools rows.

/restore reverses a publish — pulls the source archive back into the workspace. The source archive was added 2026-05-04 after a healthcare-tenant PVC wipe destroyed a customer's catalog source.

Catalog Tailwind is styles.css-only — no tailwind.config.*, no @plugin/@config. The build runs Tailwind v4 (@tailwindcss/cli) against the workspace's styles.css; content detection comes from its @source directives and theming from @theme — a JS/TS config file is never read. Because a JS config can execute arbitrary code inside the builder, assertSafeTailwindInputs (src/build/tailwind.ts) hard-rejects any tailwind.config.{js,cjs,mjs,ts,cts,mts} in the source dir and any @plugin/@config directive in styles.css. The scaffold template used to ship an inert tailwind.config.ts, so every scaffolded catalog failed this guard on its first rebuild/preview (HTTP 500: JavaScript Tailwind config files are not allowed) — fixed 2026-07-24 by dropping the file from templates/. Older workspaces (and /restores of pre-fix source.tar.gz archives) may still carry the file: delete it and rebuild — it configures nothing. Don't re-add the file to the template or relax the guard. The built-in catalog sources (apps/catalog-builder/catalogs/shared/<slug>/, baked into the image and copied onto the PVC by the startup seeder) carried the same inert file until 2026-08-28 — every built-in /publish (including the seed-builtins Job's) 500'd on this guard, blocking a scaffold-clobber recovery. Dropped from all seven then; because the seeder copies the baked source verbatim, a pre-2026-08-28 image re-adds it on every pod restart, so the seed Job keeps failing until the image is rebuilt.

Dependencies

Service Purpose
Postgres a2ui_* tables
Object storage (S3 / MinIO) Source + bundle storage, via the shared @trueserve/storage seam (getStore('catalogs'))
ai-manager Tenant + catalog metadata
code-server Source workspace mount

Vanguard

overmind restart catalog-builder

Production

Helm chart at charts/catalog-builder.

The Docker build context is the REPO ROOT, not apps/catalog-builder/. catalog-builder consumes the shared @trueserve/storage seam as a file:../../shared/storage/trueserve-storage-1.0.0.tgz dependency, which npm can only resolve if that tarball is inside the build context. So Dockerfile COPYs are apps/catalog-builder/-prefixed and copy shared/storage/*.tgz into /shared/ before each npm ci, and the CI matrix (branch-build-deploy.yml / pr-build-scan.yml) sets context: . with shared/storage/** in the change filter — mirroring ai-manager, the other consumer of the seam. Reverting the context to apps/catalog-builder breaks npm ci (the file: path escapes the context) and no image builds. Storage config is env-driven: STORAGE_BUCKET_CATALOGS + STORAGE_REGION are required (an unset bucket/region throws at first store use), credentials come from IRSA on AWS (no STORAGE_ACCESS_KEY/STORAGE_SECRET_KEY). The retired MINIO_* vars are gone — don't re-add them.

Seeding the built-in catalogs (KMS) — automatic in-process self-heal, always version: "auto"

Bundle objects in the catalogs bucket are the source of truth the browser fetches, and they do not travel with the Postgres snapshot restore that seeds the KMS EKS environments — so something has to (re)publish the seven built-ins into each env's bucket. As of 2026-08-31 that happens automatically and in-process, replacing the fragile fire-and-forget curl Job that left buckets silently empty:

  • Startup self-heal (primary). On boot — after the source-seed startup below — catalog-builder runs ensureBuiltInCatalogsPublished in the background. For each built-in it checks whether the catalog's current ready bundle actually exists in this env's bucket (storage.objectExists against the stored bundle_url) and re-publishes only the missing ones via the in-process runPublish (version: "auto"). It is idempotent — a healthy bucket does zero work and no version churn — and it needs no token and races nothing, because it runs inside the pod with direct DB + S3 access. This is why a bare pod restart (or snapshot restore without a Helm sync) now reconciles the bucket. Controlled by SEED_BUILTINS_ON_STARTUP / SEED_BUILTINS_TENANT_ID / SEED_BUILTINS_CATALOGS (values.yaml env block — the single source of truth for the slug set).
  • Second pass — cloned tenants (reconcileClonedBuiltins). The primary pass only reconciles the source tenant (SEED_BUILTINS_TENANT_ID / TrueServe). Every other tenant gets its baselines from provisionTenantBaselines → cloneCatalogRows, which copies the source version rows' bundle_url verbatim — clones share the source's bundle object by reference, they never re-upload (see Cloning and tenant provisioning). So a tenant cloned before the shared→tenant rescope still advertises a dead catalogs/shared/<slug>/… key, and its multimodal surface 404s even though the source tenant is fully healed — this is exactly why PR #636's source-only self-heal left QA's orbit tenant broken. After the primary pass, reconcileClonedBuiltins walks every non-source tenant's copy of each built-in and, only when that clone's own bundle object is absent from the bucket (objectExists === false), re-points its current version row's bundle_url/css_url/integrity/css_integrity onto the source's now-healed pointer — the shared reference it was always meant to hold (SRI still matches because we copy the source's integrity too). It is a pure DB re-point (no wasteful per-clone re-publish), safe against tenant customization (an edited clone publishes to its own catalogs/t/<clone-tenant>/… key, so its object exists → skipped-healthy, never overwritten), idempotent, and best-effort — it is wrapped in its own try/catch and never propagates, so a clone-reconcile failure can never regress the primary pass. It only re-points when the source itself is healed (skipped-no-source otherwise). Diagnosed 2026-09-01 on QA (orbit still pointing at catalogs/shared/standard-primitives/1.0.2/bundle.js after PR #636).
  • Idempotent endpoint. The same reconcile is exposed as POST /seed-builtins (behind the shared Bearer-token hook, routes/seed.ts): 200 when every built-in is healthy, 500 when any publish failed.
  • The Helm Job is now a fail-loud verifier. The post-install,post-upgrade Job (seed-builtins-job.yaml) waits for /health, then curl -fs POST /seed-builtins with an empty body (so the app's own SEED_BUILTINS_* config decides the set). Because the endpoint 500s on any failure and curl -f propagates that as a non-zero exit, the Job now fails visibly in ArgoCD instead of the old design's silent success. Set seedBuiltIns.enabled: false to rely on the startup self-heal alone.

The old Job swallowed every failure and deleted itself. Until 2026-08-31 the Job POSTed /publish?bySlug=true per slug with || echo "publish failed … (continuing)", so a single transient publish failure (source-seed vs /health race, or a snapshot-restore version collision) left the bucket empty, the Job exited 0, and hook-delete-policy: hook-succeeded deleted the evidence — the bucket then stayed empty until a values change re-triggered the hook. An empty bucket means every catalog-manifest presigned URL 404s → blank multimodal surfaces ("Multimodal connection failed"). The self-heal + fail-loud endpoint above is the permanent fix; recover-clobbered-builtin-catalog.md's "blank / empty-bucket" case is now auto-healed on the next deploy or pod restart.

Historically the Job published by POSTing /publish?bySlug=true with version = .Values.seedBuiltIns.version; the notes below on scope and "auto" still apply because runPublish resolves the catalog the same way (bySlug) and takes the same auto-version path.

The Job publishes by (slug, scope='tenant', tenantId=<TrueServe>), NOT scope='shared'. init-scripts/183 rescoped every built-in to the TrueServe platform tenant (see Catalog scope — there are zero scope='shared' rows), but the Job kept POSTing scope:"shared" until 2026-08-27. A shared lookup matches no row in publish.ts's bySlug query (WHERE c.scope=$2 AND c.tenant_id IS NOT DISTINCT FROM $3), the route 404s, the Job swallows it (|| echo "publish failed … continuing"), and the bucket stays empty — the same blank-surface 404 symptom as the version-collision case above, but for a different reason and unfixed by "auto" (the scope 404 happens before the version path runs). seedBuiltIns.scope ("tenant") and seedBuiltIns.tenantId (the well-known TrueServe id 00000000-0000-0000-0000-000000000001) are values in values.yaml; the Job passes both in the publish body and omits sourceDir (publish.ts ignores the body value and derives it from the resolved catalog). Diagnosed 2026-08-27 against an empty trusrv-dev-orch-catalogs on dev/qa/prod while richmond (Overmind, non-Helm) was unaffected.

seedBuiltIns.version is "auto" (values.yaml), and must stay that way. A pinned semver cannot reconcile a snapshot-restored DB:

  • KMS dev/qa/prod restore Postgres from another environment. The snapshot carries a2ui_catalog_versions rows whose current_version_id points at a version (e.g. 1.0.2) that only ever had bundle objects uploaded to the source env's bucket. The target env's catalogs bucket starts empty.
  • The explicit-version publish path in publish.ts (autoRequested === false) has no collision-avoidance — it inserts the requested (catalog_id, version) tuple verbatim. A pinned "1.0.0" therefore collides with the snapshot's existing 1.0.0 row on the unique key, the route 500s, and the Job swallows the error (|| echo "publish failed … continuing"). Net: the bucket stays empty while the DB still advertises 1.0.2, so every catalog-manifest presigned URL 404s and surface-app logs [SurfaceApp] catalog manifest load failed / catalog fetch failed: 404.
  • "auto" takes the auto path instead: it loads the previous ready version, bumps past every already-taken tuple (while (taken.has(...))), uploads the bundle, and re-points current_version_id at the version it actually wrote — so DB and bucket converge no matter what the restore carried. Diagnosed 2026-08-24 (surface-app chat 404 on catalogs/shared/standard-primitives/1.0.2/bundle.css against an empty trusrv-dev-orch-catalogs).

Also required for the presigned URLs to point at the right bucket: conversation-api must set MINIO_CATALOG_BUCKET (it defaults to the nonexistent trueserve-catalogs otherwise), and the catalogs bucket CORS must allow surface-app's origin — both set in values-kms-*.yaml / infra/stacks/aws-kms.

The /workspaces build source is seeded by the app container's startup, not an initContainer

The seed Job publishes by building from the resolved catalog's defaultSourceDir — and because the built-ins are TrueServe-tenant-owned, that is CATALOG_SOURCE_ROOT/catalog-tenant-<tenant_slug>-<slug> (i.e. catalog-tenant-trueserve-<slug>, default under /workspaces/…), not the legacy catalog-shared-<slug>. publish.ts / assertCatalogSourceDir reject any other sourceDir — so those source dirs must exist on the shared PVC before the Job runs, or every /publish build throws, the Job swallows it, and the bucket stays empty (same 404 symptom as above). The startup seeder (below) therefore copies the baked-in sources into catalog-tenant-trueserve-<slug> targets (the image path /app/catalogs/shared/<slug> is historical and unchanged).

The built-in sources are baked into the image at /app/catalogs/shared/<slug>. They are copied onto the PVC by the app container's command/args (charts/catalog-builder/values.yaml) at startup — which then execs node dist/index.js. This deliberately replaced a seed-workspaces initContainer: base-app renders initContainers images verbatim, without the global.imageRegistry prefix that the main app container gets — so on ECR the initContainer was unpullable, and all three KMS envs had pinned initContainers: [] to dodge the ImagePull failure. That override silently disabled source-seeding on dev/qa/prod and was the deeper cause of the empty buckets (diagnosed 2026-08-24, QA agent 4062). The startup seeder uses the correctly-prefixed app image and behaves identically in dev/kind/KMS; the KMS initContainers: [] overrides were removed. The seed is idempotent (skips existing target dirs) and tolerant (a chown/cp hiccup never blocks pod startup).

The scaffold-clobber guard — a stale empty scaffold must never publish over a real catalog

A /publish whose source is the untouched starter scaffold (only ExampleCard + ex_show_example_card) is REJECTED (409) when the catalog already has a real published version. publish.ts loads the previous ready version and, before inserting any version row, compares manifests via isPristineScaffoldManifest: real → scaffold is refused with a message pointing at the missing source. allowScaffoldReset: true in the publish body is the deliberate-reset escape hatch; a genuine first publish has no previous version so scaffolding-then-publishing a brand-new catalog is unaffected.

This closes the 2026-08-27 built-in incident. A stale empty scaffold had been left in the built-in workspaces on the PVC (components/ holding only ExampleCard.tsx); the source seeder's skip-if-exists guard preserved it instead of the baked real source; and the seed-builtins Job published that scaffold via the "auto" path as a MAJOR bump that removed every real component — financial-services (8 components) and standard-primitives (11 primitives) were overwritten with the lone ExampleCard at 2.0.0. The other five built-ins were spared only by luck: the seed burst landed during an ArgoCD rollout, their publishes were load-balanced onto the terminating pod, failed mid-shutdown, and the Job's || echo "…continuing" swallowed each failure — leaving their good current_version_id untouched (but their PVC source is the same scaffold, so a clean re-seed would have clobbered them too). Two independent guards now prevent recurrence: (1) the publish-layer guard above (environment-independent, authoritative), and (2) the source seeder (values.yaml) now heals a built-in workspace — if components/ is scaffold-only while the baked source has real components, it removes the stale dir and re-copies the baked source (best-effort; a non-root container that can't rm uid-1000 PVC files still has guard (1) as backstop). Recovery for an already-clobbered catalog: runbook.

Workspace path format

Tenant directories are named catalog-tenant-<tenant_slug>-<catalog_slug>. Both ai-manager and catalog-builder join tenants.slug into the path; renaming tenants.slug is blocked by init-scripts/152 while a tenant owns catalogs. See the catalog-workspace-path-format memory note.

Repository-authored editor workspaces

The Allianz producer catalog ships one file per registered component under components/, shared helpers under components/shared/, and its own package, lockfile, TypeScript config and editing guides. The Studio extension matches a component filename to its catalog.json name for right-click preview, watches component/helper saves, and installs the workspace's development dependencies for editor types.

The demo's catalogs.py allianz-producer copies that complete source tree into catalog-tenant-trueserve-allianz-producer, excluding dependencies, Git history and build output. It overwrites matching editor files; preserve editor changes before rerunning setup. The repository and editor copies are separate. An existing index.ts makes /scaffold a no-op, so reopening an older workspace does not add missing project files. Copy the catalog's own project files instead of replacing its manifest or entry with the generic starter scaffold.

Catalog scope: effectively tenant-only

Catalogs carry a scope of shared or tenant. As of init-scripts/183 (2026-06-04) all built-in catalogs were rescoped to the TrueServe platform tenant, so in practice every catalog is tenant-owned and independently editable. scope='shared' remains a legal-but-unused enum value (the CHECK constraint and shared-read RLS policies are untouched — they're dead-but-safe branches).

The seven former shared catalogs — standard-primitives, common-extended ("Cross-Industry"), financial-services, healthcare, insurance, retail, utilities — now belong to the TrueServe tenant (00000000-0000-0000-0000-000000000001).

There must be ZERO scope='shared' rows — an empty one is a phantom catalog every tenant sees. listCatalogs' read predicate is (is_platform_admin OR c.scope = 'shared' OR c.tenant_id = <current>), so a shared row appears in every tenant's list regardless of owner. The 2026-06-20 squash re-materialised all seven slugs as empty shared rows (0 versions / 0 components / 0 tools), which (a) duplicated "Cross-Industry" and "Standard Primitives" in every tenant's catalog list — one husk beside the real tenant copy — and (b) silently broke any agent still holding the trueserve://catalogs/v1/<slug> URI, since that form resolves against scope='shared': the agent bound to an empty catalog and its A2UI surface rendered blank with no error (agent 4201 "Reference Multimodal Voice Agent", created after 183 so its rewrite never saw it, held v1/insurance@latest). 022_a2ui_drop_shared_husks.sql repoints straggler v1/ URIs to the tenant form and then deletes the empty husks (it refuses to delete a shared row that has real content, warning instead); it's idempotent and folded into migrate-city-db.sh. Don't re-introduce shared rows as a seeding shortcut, and don't "fix" a duplicate baseline by deleting the tenant-owned copy — the TrueServe-tenant copies are the platform originals that provisionTenantBaselines clones from, not custom forks.

Catalog types: multimodal vs agent_assist

Beside scope, a catalog carries a catalog_type discriminator added by 028_a2ui_catalog_type.sql: multimodal (the default) or agent_assist. It decides whether publish blocks on a component's props_schema.

On a multimodal catalog a props_schema is documentation, and lintCatalogSource's findings are advisory. On an agent_assist catalog it is the literal json_schema handed to the provider's structured-output call by a copilot worker — there is no mapping layer, the model's JSON is the card's props (see Copilots). A schema a provider rejects therefore doesn't render oddly; it makes that copilot type silently produce nothing at runtime. So checkAgentAssistManifest findings are publish-blocking errors, returned by POST /publish and shown non-fatally in /preview.

The rules are the intersection of what OpenAI strict mode, Google (Cloud + Vertex) response_schema, and Bedrock tool-input schemas all accept — the floor across every provider the workers dispatch to:

  • props_schema present, type: "object", at least one property
  • additionalProperties: false on every object
  • required naming every property — strict mode has no optional keys, so "optional" is a nullable type (["string","null"]), never omission from required
  • a description on every property (it is the only prompt the model gets for that field)
  • object nesting ≤ 2 levels
  • none of oneOf / anyOf / allOf / not / $ref / pattern / patternProperties

The type must be on the ROW before the first publish. POST /publish?bySlug=true only resolves an existing catalog row — it never creates one — and createCatalog defaults catalog_type to multimodal. A row created any other way (the UI, a hand-written insert) therefore publishes through the non-blocking path: the gate silently doesn't apply and a card whose schema a provider rejects ships looking fine. That is why the platform copilot-cards catalog row is created by a schema script, 029_copilot_cards_catalog.sql, which also repairs an existing row by forcing the type — otherwise the gate stays off on exactly the environment where somebody already made the row by hand. copilot-cards is a baseline catalog, so it clones into each tenant; a tenant editing one of its cards redefines what that copilot type's model must produce, for that tenant only.

Cloning and tenant provisioning

A catalog can be deep-copied into another tenant via cloneCatalog (apps/ai-manager/lib/catalog-clone.ts), which wraps the pure-DB cloneCatalogRows (apps/ai-manager/lib/schema-catalogs.ts). It copies the catalog row, all versions (re-pointing current_version_id), components, tools, and plugins (plugins reset to is_platform_approved=FALSE). MinIO bundle objects are shared by reference — cloned versions reuse the source's absolute bundle_url/source_url keys (immutable, content-addressed), so there's nothing to copy and no re-publish. Ephemeral catalog_presence/catalog_commits and per-agent pins are not copied. The clone is idempotent (skip-if-exists by (tenant_id, slug)).

After the DB copy, cloneCatalog fires a non-fatal POST /restore to rebuild the new tenant's workspace folder from the shared source.tar.gz (a missing workspace only affects "Open in editor", not runtime).

New tenants automatically receive editable clones of the two baselines (common-extended + standard-primitives) via provisionTenantBaselines (apps/ai-manager/lib/provisioning/provision-tenant-baselines.ts), wired into both tenant-creation paths (the admin POST /api/admin/tenants route and the SSO-JIT path in local-rbac-service.ts). The SSO path is self-healing — a pre-existing tenant lacking baselines gets them on next login. To backfill existing tenants in one shot: cd apps/ai-manager && npx tsx scripts/backfill-tenant-baselines.ts.

provisionTenantBaselines and cloneCatalog run as system operations — they use withTenantContext(..., isPlatformAdmin=true) to read the TrueServe-owned source and write into the target tenant, neither of which normal RLS permits. This is a deliberate privilege escalation for provisioning, not a user-auth path.

Export / import as a portable bundle

GET /api/catalogs/:id/export produces an ed25519-signed bundle of a catalog version (manifest.json + manifest.sig + source/catalog.json + dist/ artefacts). Add ?format=zip for a portable, human-inspectable .zip container (default stays .tgz for back-compat); the signed manifest.json bytes are byte-identical across containers, so verification is container-agnostic.

POST /api/catalogs/import accepts either container (auto-detected by magic bytes), verifies the signature + per-artefact integrity + signer-env trust, then upserts the version. By default the target catalog must already exist; pass ?createIfMissing=true to materialise it on the fly as a tenant-scoped catalog in the importer's active tenant (auto-promoted), enabling cross-tenant/instance sharing. Known limitation: the export's source/ is only catalog.json (not the full TSX tree), so an imported catalog has a runnable bundle but no editable source until re-published.

See also