Assessments app and English proficiency¶

Status: Current | Last reviewed: 2026-10-20

Assessments combines Behavioral and English Proficiency with Name/Email/Code login, Deloitte SSO and a shared home screen. This page covers the common access layer and English implementation. The live primary hostname is https://assessments.dev.trueserve.deloitte.com, with a redirect from the former langtest hostname. The unified build and additive migration were deployed on 2026-10-14; DNS, TLS and the app’s Authentik configuration were applied through their APIs. The app keeps its dedicated data and permissions. See the deployment and verification details below.

For the interactive system diagram and detailed cross-app implementation, read the architecture guide. The linked production plan and operations/validation guide distinguish existing controls from work still required for production.

Assessment behavior¶

The BPaaS candidate screener provides Behavioral assessment at /screener, evidence review at /hiring, and elevated administration at /hiring/admin. Both types are available to admitted SSO users. Code users see only the types assigned by their administrator. SSO derives identity from verified claims; invitation sign-in matches the issued name, email and code. The Behavioral rubric does not reuse or alter English/CEFR scoring.

For SSO repeat demonstrations, each home card offers View AI review for the user's latest completed AI result alongside Start new assessment or Continue assessment. /api/home provides result-availability flags and reviewer permission without disclosing AI scores. English review links use /review/:id?tab=ai to open the AI tab while preserving explicit reveal. History links retain access to earlier submissions. English keeps its existing one-unfinished-attempt rule and configured daily limit; Behavioral now permits a separate attempt after submission. See the demo walkthrough.

The original seed contains four speaking tasks and three original workplace readings with twelve keyed questions. Speaking moves from a familiar project through explanation, negotiation and a wider issue. The form takes approximately 30 minutes including preparation and device checks. It does not copy IELTS, TOEFL or ACTFL items or claim equivalence to their scores.

Setting	Initial value
Speaking and reading requirement	B2 each; configurable to C1 or C2
Reported speaking bands	Below B2, B2, C1, C2, or insufficient evidence
Speaking criteria	Range, accuracy, fluency, coherence, interaction, phonology
Reading result	Deterministic keyed marks; CEFR initially not determined
AI conversation	`gpt-live-1`, Marin
Audio observation	`gpt-audio-1.5`
Structured assessment	`gpt-5.6-terra`; `gpt-6-astra` selectable
Recording retention	90 days
Report retention	365 days
Attempts per candidate	Three per rolling day; one collecting attempt
Concurrent voice sessions	12 per server process

A human reviewer can release a speaking and reading profile with cited evidence and a reason. The configured requirement is met only when both reviewed skill levels meet their respective threshold; a missing level remains undetermined. Levels are ordinal and are never averaged. No writing assessment or independent listening score is produced.

Candidate pages withhold AI output and say that feedback will follow later. All admitted Deloitte users can open English reviews and explicitly reveal the experimental estimate, including for their own submission. Viewing one’s own AI is allowed; self-rating and self-release remain forbidden. A human-reviewed result is a separate revision released to the candidate. Neither is a validated CEFR certificate. This pilot needs empirical validation and role-specific standard setting before use as an employment screening instrument.

Architecture¶

Component	Responsibility
React 19 / Vite browser	Invitation/SSO entry, consent, local device check, audio capture/playback, readings, result and reviewer screens
Fastify 5 / Node 22	OIDC session, authorization, task deadlines, WebSocket gateway, API and static files
Dedicated PostgreSQL database	Versions, attempts, task state, transcripts/events, jobs, reviews, audit and consent
Shared object-store abstraction	Encrypted recordings and scoring artifacts; local provider initially, S3 adapter available
In-process durable worker	Lease-based scoring/deletion jobs backed by PostgreSQL
Host nginx	Dedicated TLS vhost and WebSocket proxy to loopback 3050
Authentik	Corporate sign-in, first-time enrollment, profile mapping and elevated administrator membership

The browser captures mono audio in an AudioWorklet and resamples to signed PCM16 at 24 kHz. It sends 40 ms binary frames over a same-origin WebSocket. The first eight bytes hold a sequence number and browser capture time. The gateway validates sequence, live input rate and queue bounds, records the audio, and forwards it over the OpenAI primary Live WebSocket.

This implementation uses a relayed WebSocket media path. It does not use direct browser-to-OpenAI WebRTC or the core TrueServe media proxy. This keeps evidence capture in the app's gateway and needs only HTTPS/WSS on the existing development host. It trades WebRTC's network adaptation for a simpler auditable pilot deployment. Test real headsets and corporate network conditions before expanding the pilot.

The app generates and records the opening instruction audio, displays the authoritative text, waits for playback and preparation, then explicitly opens the microphone. It does not trust the browser clock to start capture. Live follows the current task brief and permitted follow-ups. Provider generation is probabilistic: prompt constraints do not establish that every delivered question followed the rubric. Reviewers inspect administration deviations.

The gateway records candidate and generated examiner audio separately in approximately five-second encrypted chunks. It logs sent/suppressed audio ranges and browser-reported playback. Browser playback reports are telemetry, not proof of what a person heard. Reconnection creates a new voice epoch, preserves the deadline and records an interruption. A network break may lose the final unflushed chunk.

Guided speaking exchanges¶

The guided-exchanges-v2 runtime protocol requests follow-ups with a trusted session.instructions.append behavior update followed by the question in session.commentary.append, both with delegation_id: null. Instructions alone do not reliably elicit spoken questions. After sufficient candidate speech, Answer complete or a three-second acoustic pause ends the answer. The default requires two question–answer exchanges per speaking section; minimumFollowupExchanges optionally configures one to four in a reviewed blueprint. Older frozen forms use the runtime default without changing their snapshots.

The gateway accumulates examiner transcript fragments and looks for a substantive, complete question; acknowledgements alone do not count. If no such question is observed after twelve seconds, it generates an authored TTS follow-up, persists that audio, resets queued playback and suppresses competing Live audio during its delivery. The candidate also sees the question text. Each counted exchange needs recorded speech after the question. These controls establish administration coverage, not a CEFR interaction rating: the scorer and reviewer still judge whether the candidate responded relevantly and effectively.

Normal Finish section becomes available after the exchanges and total speech minimum. End section early, a deadline, or a disconnect still saves available evidence and records incomplete coverage. A reconnect starts a fresh exchange sequence inside the original deadline. interview.* events preserve the protocol, requested/generated question, answer completion, early ending and fallback provenance. Final voice.finished records exchange coverage. Transcript detection and the energy threshold are heuristics; they do not prove playback or adequate language performance.

The browser sends answer.complete or stop (early: true for an early exit). The server emits interview.progress, interview.question and, when fallback playback is needed, audio.reset. These controls supplement the existing recorded audio protocol.

Scoring and review¶

The server finishes every task and checks that recording has stopped. It snapshots media IDs and hashes, task answers, timestamps and a digest of the interaction events, then signs the manifest with HMAC.
The worker claims a PostgreSQL job with a lease, verifies the manifest and reads/decrypts candidate recordings. It creates a task-level audio observation with conservative transcription and acoustic observations.
At least three speaking tasks must have the configured minimum recorded speech before a skill estimate is attempted. Otherwise the result explicitly reports insufficient evidence.
Two separate structured scoring requests use the same frozen rubric and evidence. Responses must satisfy the six-criterion schema and reference existing evidence IDs. If the model proposes an overall level while leaving a criterion unassessed, the server withholds the overall level and explains the missing evidence. Raw scoring responses remain separate audit artifacts. These are repeated AI judgments, not independent human validation.
The app stores model identifiers, prompt version, inputs, outputs, usage, evidence references and disagreement. Keyed reading marks remain separate from speaking.
The reviewer replays clips, examines transcripts and answers, adds non-destructive annotations, and records a rating. The AI panel starts hidden; revealing it is audited and prevents subsequent ratings being labeled independent. Raw scoring artifact downloads and validation comparison access also record score exposure, so those paths cannot bypass the independent-rating check.
Release requires another person, valid evidence, a reason, the current state version and a valid manifest. Rescoring creates new artifacts and preserves any released human result.

The cefr-research-evidence-v2-passages scorer additionally returns a reviewer-only draft and labeled evidence highlights. A provisional speaking suggestion may use the observed criteria when a full speaking estimate is withheld. The app displays the missing criteria and both draft suggestions; it does not fill missing interaction evidence or map uncalibrated reading marks to CEFR. Candidate pages hide both the complete AI assessment and the provisional draft; both are available only after explicit reveal in the reviewer workspace. Use AI draft in my assessment copies editable fields; it does not save, release or select citations. The reviewer must provide their reason. Saved reviews record adoptedScoringRunId, final levels and cited passage ranges, retaining the original scoring artifacts for comparison.

Evidence passages and transcript presentation¶

Storage chunks are not presented as meaningful clips. The worker creates fixed candidate-track passage ranges, preferring acoustic pauses between 18 and 55 seconds and capping long passages at 60 seconds. These are conservative navigation windows, not verified sentence or topic boundaries. The audio observer receives the exact passage map; structured scoring can select only existing passage IDs, assign a descriptive label, explain a criterion and flag supporting evidence, counterevidence or limitations. The server supplies their original chunk references and validates all ranges.

Reviewers listen to the full section, Cite this passage, Hear surrounding context, or Adjust excerpt. They can independently mark custom start/end positions and add their own label and explanation. GET /api/attempts/:id/audio accepts optional startSeconds and endSeconds, validates bounds and returns a WAV excerpt without modifying source media. Review requests accept evidenceSelections; the server verifies each range's original source IDs and stores those selections in the additive review record. All audio access remains authorized and audited.

The AI panel links each suggested passage directly to its section. If the reviewer has not chosen a section, the evidence view opens the first highlighted section; an explicit section choice is preserved.

AI highlights remain hidden until AI reveal, preserving independent review. The transcript display joins each speaker's fragments verbatim, including split words and repairs, then groups them using a 2.5-second transcript gap. Overlapping examiner backchannels remain separate. Provider times describe the voice-session timeline, not exact word alignments or joined-track seek positions. The original event sequence remains available in history; grouping is a display aid and does not revise the evidence.

Recorded authored fallback questions are labeled separately in the transcript. Their gateway delivery offsets are not displayed as provider transcript timestamps or merged into Live text.

Rescoring an earlier monologue-only attempt can add passages and a provisional draft, but cannot establish interaction that was never elicited. It preserves the original recording, frozen configuration, manifest and earlier scoring runs.

Attempt responses include scoringStatus from the latest score job. Reviewer pages show queued/running rescoring separately from the last completed AI result and disable duplicate requests while one is active. Candidate pages retain their completion/feedback message until a human result is released.

The speech-activity minimum uses a simple audio-energy threshold; it is an evidence sufficiency check, not validated voice activity detection. Noise, whispering or microphone processing can affect it. Technical gaps are reported separately and must not be interpreted automatically as language weakness.

Human reviewers provide overall skill ratings, narrative strengths and improvement priorities. Six-criterion analytic scores are currently generated by AI; collecting independent human analytic scores is a future validation extension. Validation shows descriptive AI/human agreement, not a claim of test validity or calibrated confidence.

Data and audit model¶

Tables	Purpose
`users`, `sessions`, `oauth_transactions`	Corporate identity, encrypted provider tokens, expiring login state
`blueprints`	Draft/published/retired configuration versions and authorship
`attempts`, `task_runs`, `voice_runs`	Frozen form, state transitions, deadlines, voice epochs, final provider usage
`evidence`, `interaction_events`	Object metadata, original media, transcripts and delivery events
`jobs`, `scoring_runs`	Durable leases, retries and assessment provenance
`reviews`, `review_views`, `appeals`	Human revisions, AI reveal history and review requests
`datasets`, `dataset_members`	Consent-scoped reusable evidence references
`audit_events`, `command_receipts`	Hash-linked audit history and idempotent mutations

Evidence objects use AES-256-GCM, object-bound authenticated data and plaintext SHA-256 verification. Provider tokens are encrypted in PostgreSQL. Transcripts, reading answers, reports and audit metadata are database records; application-level object encryption does not encrypt the entire database. Restrict database and host access and include database storage in the environment's encryption and backup controls.

Audit entries are hash-linked per aggregate with serialized sequence allocation. This is tamper-evident application history, not an externally notarized or administrator-proof WORM archive. Submission manifests also verify the exact pre-submission interaction records. Review annotations are later derived artifacts and do not rewrite original evidence.

Optional personal coaching, assessor training and model improvement default off. Coaching is available after a reviewed result and is stored separately. Dataset curation requires released results and the corresponding consent. Partitioning is stable per candidate, preventing a retake from entering both training and test splits. Exports contain references and reviewed labels, not raw recordings or an automatic fine-tuning job.

Revocation removes dataset membership and export rechecks permissions. Deletion removes stored evidence, interaction text, reading answers and dataset membership. Candidate-requested deletion also clears report and review text; minimal identity/provenance metadata remains. Previously downloaded exports cannot be recalled by the app.

Invitation access and presentation¶

/login shows a simple Assessments by Deloitte brand header above the Name, Email and Code form and Log in with Deloitte SSO. /invitations is administrator-only and manages candidates, assigned versions, expiry, code replacement and revocation. Main tabs retain their titles without editorial subtitles; the footer, header Research/assessment labels, Behavioral metric note and Validation research notice are removed. Consent and assessment instructions remain in the candidate journey.

server/invitations.sql runs after the English and Behavioral schemas. assessment_invitations stores assigned version IDs, a candidate user ID and consumed attempt IDs. An issued version remains available to its invitation after retirement. Only published versions can be assigned when issuing a new invitation.

POST /api/auth/code requires the exact application Origin and is limited to ten requests per IP per minute. Name comparison normalizes whitespace and case; email comparison trims and lowercases; code comparison ignores whitespace, hyphens and case. Random 96-bit codes are stored as SHA-256 hashes, with plaintext returned only on creation/replacement. Code responses are not cached; codes are never included in audit events or command receipts. The app does not send invitation emails.

Each invitation provisions a separate invitation:<uuid> subject and never links by email to a corporate account. Its session is bound through sessions.invitation_id, expires after at most eight hours or the invitation deadline, and receives no reviewer/admin privilege even when the demo-wide reviewer flag is enabled. Hydration checks current invitation expiry/revocation on every request, including voice authorization. Replacement invalidates all existing sessions and the old code; revocation prevents replacement and future access.

A locked invitation row makes concurrent starts converge on one attempt per assigned type. Consumed attempt IDs survive completion, expiry and deletion, preventing code reuse from granting another attempt. Replacement extends access to the same saved records without changing answers, timers, consent, evidence or reviews. A new invitation authorizes another attempt. Home cards show Continue assessment or View submission for existing code-user attempts. Direct enrollment also enforces assignment. Existing SSO repeat behavior remains available.

Administration APIs are GET/POST /api/invitations, POST /api/invitations/:id/replace-code and POST /api/invitations/:id/revoke. Writes require fresh SSO administrator entitlement, Origin and CSRF. The list returns the latest 500 records without code hashes. See the invitation user guide.

If rolling back to a build from before invitation support, first stop this app and delete only sessions with invitation_id IS NOT NULL. Older builds do not understand the candidate-only session branch. Keep invitations, attempt bindings, evidence and the additive schema. Never restore an old database dump over newer submissions as part of an image rollback.

SSO and authorization¶

Resource	Value
SSO authority	`https://sso.trueserve.deloitte.com`
Stable application slug and client ID	`proficiency`
Existing OAuth provider	125; retain it to preserve subjects
Issuer	`https://sso.trueserve.deloitte.com/application/o/proficiency/`
Primary callback	`https://assessments.dev.trueserve.deloitte.com/api/auth/callback`
Transition callback	Existing `https://langtest.dev.trueserve.deloitte.com/api/auth/callback`
Reviewer access	Every admitted SSO user with `ASSESSMENTS_ALL_USERS_REVIEWERS=true` (default)
Administrator group	Existing identifier `Proficiency Reviewers`
Bootstrap administrator	`erigolden@deloitte.com`; existing members are preserved

The application policy follows the existing Deloitte groups and corporate @deloitte.com accounts. langtest_access=true is required to admit a session. The updated mapping grants langtest_reviewer to every admitted user and separately sets assessments_admin from the administrator group. An explicit assessments_admin=false prevents a broad reviewer claim from granting administration. During migration only, an absent administrator claim falls back to the former restricted reviewer claim. Setting ASSESSMENTS_ALL_USERS_REVIEWERS=false restores claim-based review access, so it must be paired with a restricted Authentik reviewer mapping to restrict the demo.

The existing Deloitte SAML source maps username, email and display name. The enrollment prompt is skipped when mapped username is present; the user-write stage creates the account. This setup was inspected read-only on 2026-10-14 and is not replaced by the app bootstrap. The app normalizes email whitespace/case and display-name whitespace, falls back to given/family names, rejects incomplete profiles, and upserts its local user by verified OIDC subject. It does not create identities from user-entered email.

Login uses authorization code, PKCE, nonce, single-use state, strict issuer/audience validation and matching ID-token/userinfo subjects. The secure host-only session cookie has an eight-hour limit. Server requests revalidate entitlement through userinfo after 60 seconds; sensitive review/admin writes force a fresh check. Access or administrator removal takes effect on that refresh without a new login.

The OAuth provider explicitly enables authorization_code and refresh_token grants. An empty grant list can reject authorization before displaying sign-in. Provisioning retains the existing provider/client ID and shared corporate flows.

Every protected route checks the server session. Candidate write routes enforce ownership even for reviewers. Writes require the expected Origin and a session-bound CSRF token; English domain commands also require a UUID operation ID. The default owned English attempt response always returns aiResult: null; reviewers request ?view=review and must first record AI reveal. Home summaries also omit AI scores. Authoring, model controls, validation, datasets and administrative audit views require administrator permission. Ordinary reviewers receive frozen question banks only after submission.

Both consent notices disclose demo-wide access to submitted recordings, transcripts and AI assessments. The current English notice is assessments-demo-recording-2026-10-14; previous attempts retain their original consent version.

API and voice contracts¶

Run npm run openapi inside the app to regenerate its HTTP inventory and the rendered API reference. Shared blueprint and AI result schemas are included. Per-route request validation remains explicit Zod code in server/routes.ts; the inventory does not claim a fully typed client SDK.

The /voice?attempt=<uuid> endpoint uses the same session cookie and strict Origin. Browser controls are stop, ping and playback.report; candidate media uses the binary format described above. Server events include connecting, ready, microphone.open, audio.segment, recording.saved, saving, saved and error. Binary audio follows its audio.segment metadata in socket order.

OpenAI uses session.start, continuous session.input_audio.append, session.output_audio.delta, client delegation and session.close. Finalization waits up to 15 seconds for session.closed. Realtime API commit/response-create voice events are not interchangeable with Live.

Deployment and recovery¶

The app README contains build and test commands. The dedicated compose file runs tsf-proficiency on the existing Vanguard host with restart enabled, a read-only root filesystem, dropped capabilities, resource limits and rotated logs. It uses a dedicated proficiency database/role on the existing local PostgreSQL instance. This is a single-host pilot, without high availability.

Setting	Location
Private runtime configuration	`apps/proficiency/.env.local`, mode 0600
OpenAI key source	Existing `secret/OpenAIAPIKey.txt`, mounted read-only
Evidence objects	`.local-data/proficiency/evidence`
Host checkout	`/home/admin/trueserve-framework`
nginx vhost	`/etc/nginx/conf.d/proficiency.conf`
TLS material	New primary: `/etc/nginx/tls/assessments.crt` and `.key`; retain `proficiency.crt` / `.key` for the old-host redirect
DNS zone	`dev.trueserve.deloitte.com`
Public gateway	Existing `15.197.182.164`

Unified-host rollout¶

Completed on 2026-10-14 with image trueserve-proficiency:assessments-demo-20261014 (sha256:3c16d24e98a5223ec40167668c966bfca038cd50222a5ec83052102246559622). Authentik API previews and policy tests verified Erik Golden’s administrator access and an ordinary Deloitte user’s reviewer-only access, including exact name/email mappings. The production browser flow reached Deloitte’s Microsoft SAML sign-in. Anonymous assessment/review/admin API calls return 401, the deployed OpenAI credential returns 200, and the retained English submission’s manifest and 130 encrypted objects verify successfully. Interactive user authentication still uses the person’s own corporate session.

The following sequence documents the rollout for future environments; source changes alone do not update running resources. Backups of the database, evidence, runtime settings, nginx and Authentik configuration were saved in the protected .local-data/proficiency/backups/ directory before migration.

Back up the dedicated database and keys. Apply the additive app migrations and deploy the new administrator-aware app before broadening the Authentik reviewer claim. Initially retain the existing APP_URL if the new host is not ready. Existing restricted reviewer claims temporarily map to administrator access; other admitted users get review access from the app default.
Provision DNS and an issued TLS certificate for assessments.dev.trueserve.deloitte.com on the existing gateway. Keep the old hostname certificate valid for redirects.
Review python3 scripts/bootstrap-authentik.py in dry-run mode, then apply it for the target URL. It preserves the provider/client ID, corporate enrollment flows and existing administrator memberships. It prepares both strict callback URLs, changes the application display/launch URL, adds the explicit administrator claim and writes protected runtime settings. --admin-email can ensure additional existing accounts are elevated.
Install the prepared nginx configuration, set the app's APP_URL to the new origin and restart only this dedicated container. Verify both SSO role paths, first-time enrollment, profile extraction, audio upload/playback and the old-host redirect. Host-only cookies require a new app login at the new hostname; SSO can reuse the corporate session. A cutover during active recording must be avoided.

For rollback to the previous image, restore the previous restricted Authentik reviewer mapping before running code that treats reviewer access as administration. Restore the protected runtime/nginx settings and use trueserve-proficiency:pre-assessments-20261014. Keep the additive schema and current data unless performing a separately controlled database restore; restoring an old dump would discard later submissions.

The server performs graceful recording finalization on shutdown. On restart, previously open voice epochs are marked interrupted. Expired worker leases are recovered; provider calls may be repeated after a crash, so this is at-least-once processing. Completed result publication checks the current lease and manifest.

The current login-header build is trueserve-proficiency:login-brand-20261019. It adds the shared brand mark above the login form. Lint, the TypeScript/build checks and both login browser scenarios passed. No schema or assessment logic changed. The previous invitation build is the immediate image rollback; protected rollout details are in .local-data/proficiency/login-brand-rollout.txt.

The October 19 invitation and interface update was deployed as trueserve-proficiency:invitation-access-20261019 (sha256:cc2c5c3077c08e6f7cacba06fa7b09d5a65c856a54c07ed1f492587be388e8be). Verification passed 58 automated tests, nine browser scenarios, lint, TypeScript and the container build; the final field styling also passed the invitation browser scenario. The additive migration preserved every existing assessment, recording and review row, including the unfinished Behavioral attempt. Both completed examples, 152 English evidence objects and three Behavioral recordings passed runtime integrity checks. Protected backups and verification results are identified by .local-data/proficiency/invitation-access-rollout.txt. Full authenticated journeys ran in disposable schemas; the public deployment was checked for health, anonymous protection and the new login assets.

The October 16 repeat-demo build was trueserve-proficiency:repeat-demo-20261016 (sha256:5c8ee03d29e7d5d119b34df4340309f008046618cd2bb1564fd27fd917b30b15), deployed on October 16. Its migration changes Behavioral attempt uniqueness without changing saved assessment data. Health, public assets, anonymous access rejection and original evidence were verified after rollout. The backup for this change is identified by the protected .local-data/proficiency/repeat-demo-rollout.txt file. Keep the partial uniqueness index if rolling the image back; recreating lifetime uniqueness fails after a user has multiple completed attempts.

Unfinished attempts with no active voice connection expire after one day. This releases the active-attempt slot and allows normal retention to run. Full interaction/playback events are available to reviewers in pages of 1,000 records using after and the returned nextCursor.

Before restarting during a pilot, inspect the reviewer queue and wait for active conversations to finish. Restart only this container:

docker logs --tail 100 tsf-proficiency
docker restart --time 40 tsf-proficiency
curl --fail http://127.0.0.1:3050/api/health

Back up the dedicated PostgreSQL database, encrypted evidence directory and protected encryption/signing keys as one recovery set. A database dump without object files loses recordings; object files without the encryption key cannot be recovered. Restore to an isolated database/directory and test decryption before replacing a live deployment. Never rotate the current encryption key by simply replacing its environment value.

The Assessments exportable ACM certificate expires 2027-04-29 (ARN arn:aws:acm:us-east-1:465572510980:certificate/2154deb8-be38-4ae7-81b7-1cfd9834295f). The retained langtest certificate expires 2027-03-31 and is still required for its HTTPS redirect. ACM renewal alone does not install renewed material into nginx. Re-export and install after renewal using the same certificate ARN; the script validates the hostname, removes temporary key material, checks nginx and rolls back on failure:

cd apps/proficiency
# Use an operator Python environment with boto3 installed and AWS credentials configured.
python3 scripts/install-certificate.py --apply \
  --hostname langtest.dev.trueserve.deloitte.com --certificate-name proficiency \
  --certificate-arn arn:aws:acm:us-east-1:465572510980:certificate/fd7a0c95-5bdb-4604-a82e-2a9faf3202c4

For the primary hostname, the script defaults to --hostname assessments.dev.trueserve.deloitte.com --certificate-name assessments; supply the Assessments certificate ARN above. The script uses boto3, writes private key material only to protected local files, validates nginx and removes temporary exports. Existing certificate files are restored if validation or reload fails.

Schedule this command in an operator environment with AWS export permission and Docker host access. The initial deployment does not install a credential-bearing unattended renewal job. Keep a certificate-expiry check in the operating schedule.

CEFR reference and validation work¶

Use the Council of Europe's 2020 Companion volume for descriptors, skill profiles and phonological control. Use Relating examinations to the CEFR for the alignment process, and speaking performance examples for examiner familiarization and benchmarking. Check the licensing conditions before copying descriptor sets or distributing recordings.

The seed rubric and passages are original research content informed by these dimensions. A faithful CEFR claim still requires task specification, trained human raters, benchmark performances, standard setting, and empirical validation with the intended population. Prioritize:

Independent expert review of task coverage and difficulty, especially enough evidence to distinguish C1 from C2.
Double-marked speaking samples with trained raters, disagreement adjudication and a documented linking procedure.
Reading item analysis and standard setting before publishing numerical CEFR cut scores.
Held-out candidate cohorts, test/retest and alternate-form studies, boundary error rates, and subgroup/accent/device analyses.
Accessibility and accommodation trials, including cases where microphone processing changes the available phonological evidence.
Recorded administration audits and adversarial tests for coaching, task drift, prompt injection and insufficient evidence.

OPI/OPIc, IELTS Speaking and TOEFL integrated tasks offer useful format comparisons. Their formats and score mappings do not validate this new assessment. Do not infer an ACTFL or IELTS score from a shared CEFR label.