BPaaS candidate screener¶
Status: Current | Last reviewed: 2026-10-20
The unified Assessments app offers Behavioral and English Proficiency on the same home screen. Behavioral uses /screener, review uses /hiring, and administration uses /hiring/admin. Both journeys share the dedicated deployment, shared session layer, encryption key and hash-linked audit history. The unified deployment went live on 2026-10-14 at assessments.dev.trueserve.deloitte.com; the old langtest address redirects there. Their assessment lifecycles, tables and scoring remain separate.
This is a recruitment demo, not a validated selection instrument. The initial EDJ role configuration uses general customer-service practices because proprietary EDJ policies and scoring validation have not been supplied. Replace the placeholder recruitment contact and review the role rubric before collecting real candidate evidence. The demo consent notice explicitly discloses that every admitted Deloitte user can review submitted evidence.
The architecture guide and interactive diagram cover the shared runtime and both pipelines. Read the production readiness plan for implementation packages, access/storage gaps and independent validation requirements.
Candidate lifecycle¶
Candidates use Name/Email/Code login or Deloitte SSO. Admins manage shared invitations at /invitations, assigning Behavioral, English or both. Invitation users have one attempt per assigned type and no reviewer role. Authentik maps corporate profiles to verified SSO subjects; SSO users retain both assessment types and the repeat-demo controls. Get started creates an attempt from the published configuration and server-owned profile. For SSO users after submission, Start new assessment sends newAttempt: true to create a separate attempt with fresh consent and readiness. Earlier recordings, signed manifests, AI results and human reviews remain unchanged.
An advisory lock and partial unique (user_id, config_id) index allow only one unfinished (ready or active) retained attempt per configuration. Concurrent enrollment resumes it; Continue assessment never resets its timers. The migration creates hiring_user_configuration_active and drops the former lifetime-unique index. An expired unfinished row is marked expired before replacement; normal retention still removes its content. Enrollment requires X-Operation-Id and uses command receipts to replay delayed requests safely. Without newAttempt, enrollment returns the latest retained attempt, preferring an unfinished one.
The home page and history expose AI review ready plus View AI review for completed results. These links remain available while a newer attempt is unfinished and lead to the existing explicit reviewer reveal. The candidate projection includes availability only, never the AI score. See the demo walkthrough.
Authenticated APIs use the shared HTTP-only session, expected Origin and session-bound CSRF token. SSO sessions refresh entitlement periodically; invitation sessions check expiry/revocation on every request. Candidate endpoints accept an optional attemptId and enforce user-ID ownership. New unified invitation login uses /api/auth/code; the old Behavioral-only access endpoints remain removed and their records are historical. A locked shared invitation row pins the assigned configuration and consumed attempt ID, so newAttempt cannot bypass a code grant. Code replacement returns the same attempt, including after submission; a new invitation is required to retake it. See shared access implementation.
Consent records the bpaas-sso-demo-2026-10-14 notice version and timestamp. Model-improvement permission is separate and initially off. The ten-second microphone test stays in the browser: candidates must play it and acknowledge audibility before starting. The server enforces the check's minimum elapsed time; it does not claim to verify the candidate's hearing or physical microphone.
Five questions cover an upset customer, ownership, proactive action, complexity and learning from negative feedback. Future prompts remain server-side. When the next prompt is opened, a server deadline gives 30 seconds to begin recording. Starting recording sets a five-minute deadline once. Candidates can submit early; the browser stops at the deadline. A ten-second upload grace period accommodates the final in-flight chunk without increasing the permitted sample duration. The worker closes expired visible questions even when the browser is gone. Unseen questions wait for the candidate to reconnect before their timer starts.
Recording uses the existing AudioWorklet and mono PCM16 at 24 kHz. Two-second chunks upload in sequence with bounded retries. Identical sequence retries are idempotent; replacement bytes and sequence gaps are rejected. The server caps total samples against elapsed recording time and the five-minute limit. Submitted answers are immutable. Missing starts become No response, advance and do not become zero competence.
Recovery¶
Reopen My assessments and choose the saved attempt. If the session expired, use the same invitation details or Deloitte account. The attempt is tied to the immutable application user ID, not the current email string or a browser cookie. An administrator can replace a lost invitation code without resetting the attempt. Recovery cannot replace submitted answers or reset preparation/recording deadlines.
Within an active recording window, Resume recording rotates the writer token and continues after the last saved sequence. Old tabs lose write access. Saved audio is preserved and interruptions are recorded for review. Browser-crash audio that was never uploaded cannot be recovered. If the recording window has expired, the saved prefix is finalized and the interruption remains visible. This is not a mechanism for retaking an answer.
Evidence, models and scoring¶
Tables are prefixed hiring_ in the dedicated Proficiency database. For this bounded demo, audio chunks and provider artifacts use AES-256-GCM encryption inside PostgreSQL, with per-chunk authenticated context and SHA-256 hashes. A 25-minute maximum recording is about 72 MB of PCM before encryption and database overhead. Move the adapter to private object storage before a scaled rollout; the existing language-evidence tables are not reused because their foreign keys represent a different lifecycle.
The attempt snapshots its question bank, rubric, models, retention and baseline. Final submission creates an HMAC-signed manifest of the configuration hash, consent, answer states/timestamps and ordered chunk hashes. Scoring and reviewer evidence access verify the manifest; audio playback also verifies decrypted bytes. Hash-linked audit events cover consent, recovery, answer locking, media access, AI reveal, configuration publication and human review.
Scoring uses durable jobs with row-locked claims, five-minute renewable leases and up to three attempts after handled failures. Each successful provider response records its usage before parsing so invalid outputs and subsequent retries remain visible. Provider artifacts include raw output, prompt version and request hash, encrypted at rest. Network/authentication failures without provider usage are not included in token totals.
- The configured audio model generates a conservative transcript and intelligibility/recording observations through Chat Completions audio input.
- The configured text model uses Responses structured outputs to assess transcript content against the frozen role rubric. Speech is untrusted evidence, never instructions. The scorer receives neither candidate identity nor voice-derived suitability measures.
- The server checks every supporting quotation against the transcript. Unverifiable evidence withholds its score. Prompt-only repetition is additionally detected deterministically.
- The server aggregates the verified criteria. There is no model-generated accept/reject instruction or automatic disposition.
Documented rubric¶
Default weights are relevance 20%, STAR evidence 15%, described customer empathy 20%, problem solving 30%, and role knowledge 15%. All are configurable with a required total of 100.
Each criterion uses shared, editable 0–4 anchors: direct counterproductive evidence, limited approach, workable basic approach, sound supported approach, and detailed supported judgement with verification/learning. Null means insufficient evidence; 0 requires quoted evidence of counterproductive behavior.
Only meaningful responses with verified evidence contribute to each competency mean. The overall score is the weighted mean of the competency percentages. At least three meaningful responses by default (configurable to four or five), and some evidence for every competency, are required for an overall score. Missing questions are reported as coverage gaps, never silently scored zero or hidden by the overall number. Per-question scores are withheld if a criterion has no evidence.
Intelligibility, estimated words per minute and quiet audio seconds appear separately. Pace includes silence; quiet intervals can reflect pauses or microphone conditions. Neither enters competence scoring. The system must not infer emotions, personality, cultural fit, demographic traits or honesty from voice.
Human review and administration¶
Every admitted Deloitte user has reviewer access by default. Candidate completion pages say that feedback will follow later; candidate API projections never contain AI scores. For SSO reviewers, Open reviewer workspace provides an explicit route to AI evidence, including one's own assessment. Recordings are available after submission and AI requires a separate reveal action. Review-session history prevents a reviewer who has revealed AI from later claiming an independent rating. Human self-review is rejected by immutable user ID (email fallback only for legacy invitations), even if the profile email changes. Notes and dispositions are append-only; rescoring preserves review history.
Configuration and model administration require the separate assessments_admin claim. The bootstrap preserves the existing Proficiency Reviewers group as the administrator allowlist and ensures erigolden@deloitte.com is included. Broad reviewer permission never grants administration. Configuration changes create immutable drafts; a different administrator must publish a draft. Attempts retain their pinned configuration, including model selection. The initial seed is a system-published demo configuration. Ordinary reviewers cannot retrieve unsubmitted question banks through review endpoints.
The console supports deployed fine-tuned model IDs, consent-based training eligibility manifests and comparison against independent human scores. In line with Proficiency reuse rules, manifest export does not export audio/transcripts, upload a dataset or start a training job. Actual fine-tuning requires an approved external training workflow, current consent checks, de-identification, held-out evaluation and a reviewed configuration before use.
Retention is configured between one and 90 days from enrollment (default 30). The worker deletes recordings, provider artifact contents, transcripts/results and reviewer notes when the attempt expires, and removes copied candidate name/email from the attempt and any legacy invitation. User/attempt identifiers and non-content model/usage metadata remain for accounting and audit. No recordings or transcripts are logged in audit events. Authorized reviewers can withdraw optional reuse consent; eligibility is recalculated at export.
Demo and measurements¶
Run from apps/proficiency:
npm run migrate
npm run typecheck
npm run lint
npm test
npm run build
npm run test:e2e
npm run test:bpaas-demo
Migration is additive and seeds the initial configuration. Do not run a live migration or restart the official app just to diagnose an incident. The test fixture and real-provider demo create disposable database schemas; they do not populate the live queue.
test:bpaas-demo uses the configured OpenAI key to synthesize three substantive responses and one repeated prompt, enrolls a synthetic SSO user through candidate APIs, expires the fifth question without a recording, runs real transcription/scoring and records a human follow-up disposition. Only disposable test clocks are accelerated; Authentik is mocked in this integration fixture. It asserts three meaningful responses, a repeated-prompt flag, a missing answer and a non-null overall score, then writes artifacts/bpaas-demo/result.json. It fails rather than fabricating results when the provider is unavailable. No confidential candidate data is sent.
For a live walkthrough, sign in and choose Behavioral from My assessments, complete /screener, then open the reviewer workspace to reveal AI. Use another signed-in user's submission to demonstrate saving human notes/disposition. Include a prompt-only answer and allow a preparation timer to expire.
The dashboard reports:
| Measure | Definition |
|---|---|
| Candidate completion | Submitted attempts / attempts that started |
| Time to results | Mean submission-to-first-result time, including queue time |
| Reviewer time | Mean total active review-session seconds per submitted candidate, excluding the candidate's own viewing time |
| Manual baseline | Configured assumption in minutes, default 20; validate independently |
Review time uses 15-second heartbeats while the tab is visible; individual intervals are capped at 20 seconds. It is an engagement proxy, not exact attention measurement. Synthetic attempts and deleted/expired attempts are excluded from live metrics. Model performance shows the first independent rated review per candidate with an AI score and the mean absolute score difference; this small-sample comparison is not proof of validity or fairness. Missing observations display “—”.
Operational boundaries¶
- Candidate API prefix:
/api/bpaas/candidate/; reviewer/admin prefix:/api/bpaas/reviewer/. - Additive schema:
server/bpaas/schema.sql; runtime:server/bpaas/; shared contracts:shared/bpaas.ts. - The existing secret-file configuration supplies provider credentials. A provider 401 requires repairing that credential, not changing candidate scores.
- API inventory regeneration:
npm run openapi. See Proficiency API and the user guide. - Official Dev deployment is separate from local implementation and test verification. Do not alter core voice services or deploy to the official environment without the appropriate authorization.