Assessments architecture and technical guide¶

Status: Current implementation documented; production target proposed | Last reviewed: 2026-10-20

Use this guide to understand the unified Behavioral and English Proficiency app, continue its implementation, or review a production proposal. The app supports demonstrations with completed AI analyses and fresh attempts. Production release still needs reproducible source delivery, narrower access controls, safe service replication, recoverable storage and independent assessment validation.

This guide describes the implementation inspected on October 20, 2026. Read the production work packages for changes and acceptance criteria, and the operations and validation plan for testing, migration and running the service. The AI handoff retains the product decisions and recovery history.

Documentation format: Archify was explicitly requested, so its editable JSON and checked exports accompany the diagram. Source paths are repository-relative code references because the ignored implementation is available in the working checkout but is not published by the documentation site.

Interactive architecture¶

Open the standalone Archify diagram. It includes light/dark themes, zoom, node search, relationship exploration and export controls.

Use the interactive viewer on desktop. The PNG preview and SVG download are convenient for slides and narrow screens; the interactive viewer requires horizontal exploration on a phone.

Current Assessments architecture: browser, nginx, Fastify, voice and scoring, Authentik, PostgreSQL, English evidence files and OpenAI

The main path is browser → TLS ingress → application API → voice/scoring → OpenAI. The diagram separates responsibilities visually; the API, voice gateway and both scoring loops currently execute in one Node.js process in one container. Database and evidence-file branches show persistent dependencies. Arrows summarize interactions, not every HTTP response or storage read.

The editable specification, delivery receipt and browser verification accompany the HTML. The diagram describes the existing deployment, not the proposed AWS target.

Scope and evidence¶

The product name is Assessments. The source directory, database, SSO application/client and some API terminology retain proficiency, langtest and bpaas. This service is separate from the framework's core conversation/voice applications.

Item	Inspected or recorded state
Source	`apps/proficiency`; implementation files and hashes are in the source evidence manifest
Deployment identity rechecked October 20	Container `tsf-proficiency`; image `trueserve-proficiency:login-brand-20261019`
Image ID	`sha256:48258c057bd2a2a1bbb690c697ad924ce5bc9c20b7cd989ea3ee20d186276b09`
Listener	Dedicated host deployment with application listener `127.0.0.1:3050`; nginx publishes HTTPS and WebSocket traffic
Primary host	`assessments.dev.trueserve.deloitte.com`; the legacy `langtest` hostname redirects
Persistence	Dedicated `proficiency` PostgreSQL database plus local encrypted English evidence files
Source reproducibility	Only `apps/proficiency/.env.example` is tracked under the app directory; most app source and the older English technical guide are explicitly ignored
Verification scope of this documentation update	Source inspection, container image identity, official reference review, diagram validation and documentation/browser checks

The source manifest records exact file bytes because the repository commit alone cannot identify ignored application source. Container identity does not prove that every local source file equals the deployed image. Earlier rollout verification is recorded in the handoff; this documentation update does not rerun a live candidate assessment, corporate sign-in, provider scoring or disaster recovery.

No deployment inventory, cloud-account control review, penetration test or psychometric study is implied. Where a production control is absent from inspected code, the roadmap says so. Controls that might exist outside the app, such as organization-level data agreements or platform monitoring, require separate evidence.

Technology stack¶

Version families below come from the application's package declarations and Dockerfile. Resolve the lockfile and built image before a release; a declared range is not an exact installed version.

Layer	Technology	Responsibility
Browser	React 19, React Router 7, TypeScript 5.9, Vite 7	Login, assessment journeys, history, reviewer tools, invitations and configuration
Audio capture	Browser microphone APIs, AudioWorklet, Web Audio	Mono PCM16 at 24 kHz, capture activity, timed responses and playback
HTTP service	Node.js 22; Fastify 5	Authentication, authorization, CSRF checks, validation, assessment commands and static SPA delivery
Live transport	Fastify WebSocket and `ws`	Browser-to-server and server-to-OpenAI live connections for English speaking
Validation and identity	Zod 4; `jose` 6; Authentik OIDC	Request/result schemas, token verification, corporate claims
Database	PostgreSQL through `pg` 8	Users, sessions, invitations, attempts, configuration versions, jobs, reviews and audit records
Evidence storage	Local/S3 adapters from `@trueserve/storage`; application AES-256-GCM	English media/artifacts outside SQL; encrypted Behavioral media/artifacts inside SQL
AI integration	Direct server-side HTTP and WebSocket calls	Spoken instruction generation, live English interviews, audio observation, structured rubric analysis and optional coaching
Delivery	Multi-stage Docker image and dedicated Compose service; nginx	Process isolation, host routing and TLS
Existing verification	TypeScript, ESLint, Node tests and Playwright	Domain/API regressions, browser journeys and optional real-provider smoke tests

The service does not use an autonomous agent framework, a vector database or a retrieval pipeline for assessment scoring. Its main AI work is an explicitly programmed sequence of provider calls against versioned questions and rubrics. Dataset exports identify eligible evidence; they do not train or deploy a model.

Request and process boundaries¶

The server entry point (apps/proficiency/server/main.ts) initializes configuration, the PostgreSQL pool, Fastify security plugins, authentication, routes, evidence storage, the English voice gateway and both job loops. It serves the built SPA from the same process and origin. Application state and session records are server-owned.

Fastify limits ordinary request bodies and WebSocket messages to 128 KiB, sets a 30-second request timeout, redacts several sensitive logging fields and disables routine request logging. Helmet establishes a restrictive content security policy; API responses use Cache-Control: no-store. Cookie-based writes require the expected Origin and a CSRF token. Rate limiting is in process, with a default 600 requests per minute per client key; code login has a lower limit. See apps/proficiency/server/main.ts and apps/proficiency/server/auth.ts.

Those settings need revalidation behind a new proxy. trustProxy currently trusts loopback, and the default bind address is loopback. A distributed limiter and tested proxy trust rules are necessary before relying on rate limits across replicas or behind a corporate NAT. Moving to a private service network requires explicit listener, forwarded-header, cookie and Origin tests.

The Compose configuration (apps/proficiency/deploy/compose.yml) uses a non-root user, a read-only container filesystem, dropped capabilities, restricted temporary storage and read-only secret mounts. It uses host networking, local paths, a two-CPU/two-GiB resource ceiling, and a 40-second stop grace. These are useful local controls, not a production availability design.

Identity, invitations and authorization¶

Name, email and code¶

An administrator creates an invitation for Behavioral, English or both and chooses published configurations and expiry. The server generates a 96-bit random code, returns it once and stores its SHA-256 hash. The invitation stores the expected name/email and authorization to the selected assessments. Administrators can list invitations, replace a code or revoke access; email delivery is not implemented.

Login normalizes name, email and code and matches all three against a live invitation. A valid code establishes a candidate-only session for up to eight hours, bounded by invitation expiry. This is possession of an administrator-issued credential plus matching supplied details. It is not verification of control of the email inbox. See invitation routes (apps/proficiency/server/invitations.ts) and schema (apps/proficiency/server/invitations.sql).

Each invitation creates a separate application subject. Matching an SSO email never merges the accounts or grants corporate roles. Every request checks invitation status and the assigned assessment scope. Revocation/expiry rejects subsequent authenticated requests; an already connected English voice session is periodically rechecked, rather than every audio frame.

One invitation authorizes one persistent attempt per assigned assessment type. Replacing a code invalidates prior invitation sessions and restores access to the same attempts. It does not clear a consumed attempt reference. An administrator issues a new invitation for a retake.

Deloitte SSO¶

The browser is redirected through Authentik to the corporate sign-in flow. The server verifies the OIDC flow using state, PKCE, nonce, issuer/audience checks and matching token/user-info subjects. It stores encrypted provider tokens behind an opaque, hashed application session identifier and refreshes identity/entitlements server-side.

SSO identity is keyed by the provider subject. Preserve the existing provider/client and subject mapping when migrating. Recreating the application with a different subject mapping can orphan ownership even if displayed emails remain the same.

The current demonstration admits all authorized SSO users as reviewers through ASSESSMENTS_ALL_USERS_REVIEWERS=true; upstream reviewer claims also affect access. Administration uses assessments_admin, with a legacy fallback to langtest_reviewer when the administrator claim is absent. Explicit assessments_admin=false remains false. Production role changes therefore require both app and Authentik mapping changes, not just one environment flag.

Review visibility and conflicts¶

Code users remain candidates and do not receive AI scores through reviewer APIs. SSO reviewers can explicitly reveal AI analysis, including their own completed result. A reviewer cannot save a human review/release for an attempt owned by the same application user. This supports the requested demonstration.

The current self-review check uses the application user ID for modern attempts. A person using an invitation and SSO has different IDs; a production conflict-of-interest rule needs a separately verified person/assignment relationship. Do not solve this by silently merging users with the same unverified email. Dataset split logic based on user ID needs the same review before claiming that one physical person's retakes cannot cross evaluation partitions.

Reviewer queues are broad, without a tenant/requisition/campaign assignment model. Production must specify who can see each person's recording, who can release results and who can export reusable evidence. See English routes (apps/proficiency/server/routes.ts), Behavioral routes (apps/proficiency/server/bpaas/routes.ts) and production package P1.

English Proficiency lifecycle¶

Select or resume. The server enrolls against a published blueprint, freezes its configuration and hash, and preserves previous completed attempts. It permits one active English attempt per user. Explicit fresh-attempt commands and operation receipts distinguish retries from retakes.
Consent and readiness. The candidate accepts the applicable notice and makes separate optional reuse/coaching choices. The existing local microphone check still needs manual audibility confirmation; automated readiness is a separately recorded user follow-up.
Start a timed task. The server owns task deadlines, attempt state and reading-answer acceptance. Secure reading material is released at the appropriate step, not preloaded with answer keys into the home page.
Collect speaking evidence. Browser PCM reaches the application over an authenticated WebSocket. The server maintains sequence/epoch information, records candidate and assistant audio and interaction events, and mediates the OpenAI connection. Provider credentials never reach the browser.
Guide the conversation. Authored instruction text can be rendered to speech. Live follow-up logic seeks substantive exchanges; short acknowledgements do not satisfy the substantive-response rule. Original prompts, responses and recording boundaries remain evidence.
Seal the submission. The application finalizes the task records and creates a manifest of the snapshot, answers and original evidence. The scoring job consumes that sealed evidence.
Analyze asynchronously. A durable job assembles eligible speaking audio, performs audio observations, then runs two structured synthesis passes. Evidence coverage and disagreements can withhold a result or require review. Reading marks are deterministic; a CEFR reading level also requires configured calibration.
Review and release. A reviewer can listen, examine evidence, explicitly reveal AI, record a separate judgment and release a reviewed report. Original AI runs and human revisions remain distinguishable. Candidate coaching requires a released result and the relevant optional permission.

The seed blueprint has four speaking tasks and three reading tasks, with twelve reading questions. These are configuration facts, not a fixed product limit. Speaking coverage and task-duration rules are implemented in the worker. The seed reading calibration is unestablished; do not interpret a raw reading mark as an established CEFR level.

English state includes ready, in_progress, interrupted, processing, review_pending, released, ungradable, abandoned and deleted. Interruption is a recovery condition, not permission to reset the original timer or rewrite accepted answers. Evidence already saved can survive a reconnect; continuity of the upstream AI conversation is not guaranteed.

See apps/proficiency/server/domain.ts, apps/proficiency/server/voice.ts, apps/proficiency/server/interview.ts, apps/proficiency/server/worker.ts and the English technical reference.

Behavioral assessment lifecycle¶

Enroll and consent. The server freezes five questions, rubric, model choices, required coverage and retention in an attempt snapshot. SSO users can start a fresh attempt after submission; an invitation retains its assigned attempt.
Check the microphone. The current flow records a ten-second local check, displays input activity and requires playback acknowledgement. Server checks enforce elapsed time, but a checked box is not a measured intelligibility result.
Answer five timed prompts. Each response has a 30-second preparation/start window and up to five minutes of recording. Early submission is allowed; missed responses remain explicitly missing. A short upload grace handles the end of capture.
Upload immutable audio chunks. The browser sends PCM chunks with ordinal, sequence and writer ownership. The server hashes and encrypts accepted chunks into PostgreSQL. Reopening rotates writer ownership so an old tab cannot overwrite the recovered attempt.
Seal and queue. Final submission seals the snapshot, consent and ordered answer/chunk evidence. Original deadlines, accepted chunks and missing-answer states are retained.
Analyze recorded answers. The worker observes/transcribes each nonempty answer, then performs structured transcript-based competency scoring where there is sufficient meaningful evidence. Behavioral recording does not use the English live interviewer.
Validate and aggregate. The server checks structured output, exact contiguous quotations and coverage. Criterion scores are 0–4 or null. Zero requires supported counterproductive evidence; missing evidence stays null. A 0–100 aggregate is withheld when required answer/competency coverage is missing.
Review. A reviewer listens, reads the transcript, chooses whether to reveal AI and records notes, a reference score and a disposition. Re-analysis appends provenance and preserves review history.

Behavioral transcripts support assessment of the content of a response. Speech observations must not become personality, emotion, cultural-fit or protected-trait predictions. The current generic customer-service rubric still needs role-specific policy and job analysis.

The candidate completion experience promises later feedback, but the inspected Behavioral routes do not implement a complete candidate-facing reviewed-feedback delivery workflow. English report release and Behavioral reviewer disposition are different capabilities. Production must finish and test the latter's notification/delivery, correction and appeal path.

See Behavioral domain (apps/proficiency/server/bpaas/domain.ts), scoring (apps/proficiency/server/bpaas/scoring.ts), shared types (apps/proficiency/shared/bpaas.ts) and the Behavioral technical reference.

AI calls and their limitations¶

Use	Current integration	Evidence/control
English authored instructions	`POST /v1/audio/speech`, `gpt-4o-mini-tts` in the helper	Original authored text and generated audio are distinguishable
English live conversation	WebSocket `/v1/live/sessions`, configured live model; current implementation uses GPT Live	Server-mediated connection, persisted original audio/events, `store: false`
Recorded audio observation	`POST /v1/chat/completions` with audio input	Structured observations/transcript and provider output feed subsequent scoring
English synthesis	`POST /v1/responses`, two passes over evidence	Versioned rubric, structured result validation, evidence references and disagreements
Behavioral competence	Structured Responses calls after audio observation	Transcript quotations and coverage checks; null for insufficient evidence
Optional English coaching	Responses call using the released result	Separate candidate permission; does not rewrite original evidence or released scores

Model IDs are stored with configuration/attempt snapshots; the earlier verified Behavioral demo used gpt-audio-1.5 and gpt-5.6-terra. Do not treat those historical calls as proof of current provider entitlement, regional availability or stable behavior. Record the exact model identifier, prompt/schema version, request hash, usage, attempt manifest and release version for each production analysis.

Two passes using the same provider and rubric can share errors. Schema conformance proves that output has an expected shape; an existing citation ID or matching quotation does not establish that the reasoning is valid, fair or job-relevant. Human agreement is also not, by itself, independent criterion validity.

OpenAI distinguishes application state from abuse-monitoring data. Its current documentation says API inputs are not used for model training unless the customer opts in, and default abuse-monitoring retention can include customer content for up to 30 days, subject to stated exceptions. store: false is not the same as approved Zero Data Retention. Live-session storage, endpoint eligibility, prompt caching and regional controls have separate conditions. Confirm the organization's actual project settings and agreement before production; those settings were not inspected. See V1 in the operations reference list.

The helpers currently use the global api.openai.com host. Selecting a regional project alone is insufficient if the application continues calling the wrong endpoint. Production configuration must cover both HTTP and WebSocket hosts and test every enabled model/feature in the chosen region.

Data and integrity design¶

Data family	Current storage	Production implication
Users, sessions, invitations	PostgreSQL; session tokens hashed and OIDC token payloads encrypted	Name/email and invitation metadata need explicit access and retention rules
English attempts/tasks	PostgreSQL snapshots, answers, state and timing	Preserve identifiers, accepted answers and configuration hashes through migration
English original media and AI artifacts	Encrypted local files through the evidence adapter; metadata/hash in PostgreSQL	S3 adapter exists, but migration, reconciliation and backup consistency still need implementation/testing
English interaction events	PostgreSQL JSON payloads, including transcript-related content	File encryption does not encrypt all sensitive application columns
Behavioral original audio and model artifacts	Encrypted `bytea` in PostgreSQL	`STORAGE_PROVIDER=s3` does not migrate Behavioral data; DB size/restore cost grows with audio
AI results, human reviews and dispositions	PostgreSQL structured records/revisions	Separate AI provenance, human accountability and candidate delivery
Jobs and operation receipts	PostgreSQL rows, leases, attempt references and request hashes	Durable queueing is present; execution and provider billing remain at least once
Audit and manifests	Per-aggregate hash chain; content hashes and HMAC signatures	Tamper detection under a key trust model, not independently immutable storage

Evidence encryption uses AES-256-GCM with random nonces and context-bound authenticated data. The current LT01 envelope contains no key version. One configured encryption key also protects OIDC token payloads; the session secret is used for CSRF derivation and manifest signing. Replacing either secret without a compatibility/migration plan can make historical evidence unreadable or unverifiable.

Original media is hashed before encryption and verified on retrieval. Submission manifests bind the frozen assessment to its evidence. Audit events are serialized per aggregate with sequence/previous-hash linkage. These controls detect several accidental or unauthorized changes, but a sufficiently privileged actor with database and signing-key access can rewrite the chain. Independently retained audit anchors and separation of duties are production work.

English retention removes evidence objects and interaction events, eventually expires reports, and handles explicit deletion requests and some appeal holds. Behavioral expiry removes chunks, clears stored provider content/results/reviews, scrubs copied attempt identity and revokes legacy invitations. The newer shared assessment_invitations and users identity records are not covered by that Behavioral scrub. Backup expiry, object versions, provider retention and all derived copies require a common retention policy.

See apps/proficiency/server/crypto.ts, apps/proficiency/server/audit.ts, apps/proficiency/server/evidence.ts, English schema (apps/proficiency/server/schema.sql) and Behavioral schema (apps/proficiency/server/bpaas/schema.sql).

Background work and scaling constraints¶

English jobs are claimed with FOR UPDATE SKIP LOCKED, a ten-minute lease and periodic heartbeats. Behavioral jobs use a similar queue with a five-minute lease and heartbeat. Each loop currently processes one job at a time per process. Recoverable leases support another attempt after a crash; they do not make a provider call execute exactly once.

There are three immediate constraints:

Replica startup is unsafe for live English sessions. voice.recover() selects every open voice_run, without an owning instance or expired-owner lease, and marks it interrupted. Starting a second process can therefore invalidate sessions still handled by the first.
Shutdown does not drain all work. The entry point stops scheduling, closes live sockets, waits seventeen seconds and closes Fastify/database resources. The worker stop methods do not await every in-flight scoring operation.
Maintenance has limited visibility. Some worker/retention failures are caught without an operational signal. Health checks test database connectivity, not queue progress, evidence availability or retention success.

Retain the durable SQL queues initially if they meet measured load. Separate web/voice, scoring and maintenance responsibilities, add owner/fencing semantics, bounded retries and observable recovery before adding replicas. A message broker is an option after measurement; it does not remove database transaction or duplicate-provider-call problems.