Phone emulator

Status: Current | Last reviewed: 2026-10-20

shared/phone-emulator is the shared iPhone UI for the Preview panel in Agent Studio, the standalone /emulator page, the embeddable web component and the separate Ari Assistant app. Read this before changing anything visual in it: the package renders inside a shadow root and its artwork is served by the host rather than bundled. Studio and the existing emulator hosts consume the packed tarball.

Ari Assistant builds its custom app inside the canonical PhoneFrame directly from shared source. Its build aliases React to the Assistant installation and injects its versioned app stylesheet into the frame's shadow root. Changes to the shared frame require rebuilding Ari as well as repacking the existing hosts. Ari does not import the calling runtime.

The reference design it is ported from is a standalone Next.js app (ios-phone-simulator), which is where the traced SVG glyphs, the wallpapers, and the 21 app-icon PNGs come from.

Telephony Preview passes initialApp="phone" and a host-owned dialerContent to open directly on its connection/number picker and dial pad. initialApp defaults to Home and is read only at mount; the Preview host keys the phone when switching modes while idle. Calls and the Home gesture retain their normal navigation. No tenant selector or caller-variable form is added to the phone.

PhoneKeypad supplies the outbound Preview and shared in-call DTMF keys: 80px circles, 36px SF Pro/system digits, and rows distributed through the available phone height. It owns local DTMF playback and calls onDigit once; the host alone decides whether to edit a destination or send a live DTMF event. Both keypad surfaces use inline styles because the shadow stylesheet is fixed.

HomeScreen.extraApps accepts optional host-owned app shortcuts (ID, label, icon URL and selection callback). Director uses it to launch a personal assistant without altering the native apps. PhoneKeypad.activeDigit is an optional authored pressed key with matching aria-pressed; it does not send DTMF by itself. The package exports dtmf-frequencies.json for server-side Scene audio without importing React or browser-only modules. See personal assistant Scenes.

Native phone typography

src/utils/phone-typography.ts defines display and text stacks following call-screen-fonts.md: -apple-system, BlinkMacSystemFont, then system-ui, with explicit SF names afterward. This avoids selecting an installed single-weight SF face ahead of the system family. Smoothing is scoped to the phone host. The status clock uses 17px/22px at weight 600 (15px with the expanded call island); the call timer uses 20px/600, caller name 30px/600, and control labels 16px/400. Long caller names wrap without ellipsis.

PhoneFrame.fontFamily optionally sets both --pe-font-display and --pe-font-text for all native phone surfaces, including Messages and lock-screen clocks. The host must load that font. Director supplies bundled Scene Inter for consistent phone text on Apple, Windows and Linux, and preloads its regular and semibold faces before capture. Live hosts without the override retain the system-first stack. Brand fonts remain independent of native phone chrome.

Safari browser chrome

BrowserApp follows the Safari styling in allianz-phone-simulator 1.zip: a 44px frosted address pill below the status bar and a floating 48px, five-icon toolbar above the Home gesture. The content viewport reserves 116px at the top and 84px at the bottom, so neither bar covers card actions while scrolling. Its scroll viewport uses block flow so the catalog iframe retains its measured height instead of shrinking as a flex child and creating a second scroll region. The shared call island expands to a plain black pill with a green handset and tabular timer. During expansion, the status bar keeps the clock and battery visible and hides the network glyphs to prevent overlap.

Back returns to the prior phone app; the share-shaped Open in browser control uses the existing external-link callback. Page options, reload, forward, bookmarks, and tabs remain disabled simulator chrome. Live cards still update in place. The browser keeps the native React surface and existing authentication handoff; it does not navigate arbitrary URLs inside the phone. These styles are shared by Agent Studio Preview, the standalone emulator, and Director.

Edits do not reach the app until you pack

apps/ai-manager depends on the package as a file tarball, not a workspace link. Editing src/ and reloading the browser shows you nothing.

cd shared/phone-emulator
npm run release   # build (tsup) + npm pack + npm install the tgz into apps/ai-manager

Then npm run typecheck in apps/ai-manager — a bare tsc --noEmit there fails on generated .next types (see repo hygiene).

Warning: build-shadow.py in the package root is a dead one-shot scaffold. It writes embedded-css.ts and shadow-wrapper.tsx, neither of which exists any more. Do not run it.

The shadow root, and why the CSS string must be a constant

ShadowContainer attaches an open shadow root, injects a css string into it, and portals its children into a display: contents mount. Host CSS cannot reach inside, which is the point — Agent Studio's Tailwind build and the emulator's are unrelated. React context still crosses the portal, so a provider rendered outside PhoneFrame reaches components inside it.

It attaches once and early-returns when a shadow root already exists. So the css prop must be a stable module-level constant; building it inline per render silently pins whatever string the first render produced. phone-frame.tsx holds the one combination:

const SHADOW_CSS = `${PHONE_EMULATOR_CSS}\n${PHONE_EMULATOR_MOTION_CSS}`

phone-emulator-css.ts is a generated ~102 KB Tailwind dump — never hand-edit it. New keyframes and easing curves go in the hand-authored phone-emulator-motion.ts, which is concatenated onto it.

Because host CSS cannot reach in and the Tailwind dump is fixed, nearly everything in the package is styled with inline styles.

Motion: springs translated to bezier curves

The reference design animates with framer-motion; the package has no such dependency and does not add one. Its four springs are reimplemented as CSS transitions with matched durations:

Reference spring	Package export	Duration	Curve
iosGentle — spring(260, 30)	EASE_GENTLE / DURATION_GENTLE	520 ms	cubic-bezier(0.32, 0.72, 0, 1)
iosSnappy — spring(340, 34)	EASE_SNAPPY / DURATION_SNAPPY	440 ms	cubic-bezier(0.32, 0.72, 0, 1)
iosBanner — spring(420, 28)	EASE_BANNER / DURATION_BANNER	480 ms	cubic-bezier(0.34, 1.32, 0.42, 1)
keyboard	EASE_KEYBOARD / DURATION_KEYBOARD	250 ms	cubic-bezier(0.38, 0.7, 0.125, 1)

iosBanner is underdamped in the original (ζ ≈ 0.68), which is why its curve overshoots and the other two do not. The sheet also declares @keyframes pe-bubble-in, pe-screen-in, pe-app-grid-in and pe-banner-drop with matching utility classes, plus a prefers-reduced-motion block that collapses everything to 150 ms with no delay.

Message bubbles stagger in at BUBBLE_STAGGER_MS (110 ms) up to BUBBLE_STAGGER_CAP (8 bubbles). messages-app.tsx snapshots the ids present when a thread opens during render — React's documented "adjust state when props change" escape hatch — so the first painted frame already carries each delay. Assigning delays in an effect would restart animations already in flight, and a message arriving mid-conversation animates immediately rather than queueing behind history.

Artwork is served by the host, not bundled

The real iOS-style app icons, wallpapers, and call-button glyphs total ~1.2 MB. Bundling them would put that into every consumer's JS payload, so the host serves the files and the package resolves URLs against display.assetBaseUrl (default /phone-emulator).

Where	What
apps/ai-manager/public/phone-emulator/	the 26 files: 21 app-icon-*.png, sms-notification-app-icon.png, wallpaper-home.jpg, wallpaper-lock.jpg, call-accept.svg, hangup.svg
src/assets.ts	PHONE_ASSETS, APP_ICON_FILES, resolveAssetUrl()
src/components/phone-assets-context.tsx	PhoneAssetsProvider / usePhoneAssets() → asset(file), appIcon(name)

PhoneEmulator wraps its whole tree in the provider, so every consumer gets it; a component rendered outside one falls back to the default base URL. The embed web component iframes ai-manager's /emulator route, so the absolute /phone-emulator/... paths resolve there too.

Home-screen icons are <img> elements with no border and no inner highlight. Earlier versions faked the icons with lucide glyphs on CSS gradients, and the border existed only to give those flat glyphs depth — with real artwork it reads as a frame around each icon. Same rule for the call buttons: call-accept.svg and hangup.svg are complete 51×51 buttons carrying their own coloured discs, so the <button> underneath them must be fully transparent.

lock-time-fill.svg from the reference is deliberately not copied. It is a static "9:41" mask backing a frosted-glass clock, which cannot show real time; the lock screen renders live text at the same box, weight, and shadow instead.

That box — 205.06 × 87.4 at top: 115 — was traced around those four glyphs, so a live clock has to fit the same width. The lock clock therefore drops the day period (Intl.DateTimeFormat().formatToParts(), filtered, so a 24-hour locale is untouched) and sets white-space: nowrap. iOS omits AM/PM on the lock screen too, and the status bar above still shows it, so the two are meant to differ. Without both, 2:30 PM at 84px wraps to a second line and — because the box centres its content — overflows upward through the date widget above it.

The lock screen is presentational and opt-in

LockScreen gates nothing. It is a wallpaper, a live date and clock, and a grabber, and tapping it unlocks. Turn it on per host:

{ "display": { "lockScreen": { "enabled": true, "relockOnCallEnd": true } } }

enabled defaults to false, so Agent Studio's Preview panel and the embed keep the idle→call flow they have always had. Only the standalone /emulator page opts in, and only when demo mode is on (its Demo Settings → Start on the lock screen checkbox).

With it on: the phone opens locked, tapping reveals the home screen, and placing or answering a call slides the lock away. An incoming call rings over the lock screen first, the way iOS does — the banner sits at z-index: 45 — and the lock lifts when the call reaches calling or connected. When the call ends, the same layer slides back down over the home screen (relockOnCallEnd, default true).

The layer lives at z-index: 20 inside phone-frame.tsx — above the app page, below the status bar (30), Dynamic Island (40), banners (45/46) and home indicator (50) — so all the shared chrome shows through it, matching the reference composition. It stays mounted while unlocked so the re-lock animates the same element rather than remounting one.

PhoneEmulator also suppresses the home-screen call banner while locked: its "tap anywhere to call" affordance would compete with tapping the lock screen to unlock.

Absolute positioning and the traced coordinates

The reference's header coordinates (top-[61px], top-[78px], and so on) transfer verbatim because absolutely-positioned children lay out against their containing block's padding box — paddingTop on an app container does not offset them. This is why the Messages thread view sets no top padding on its container and instead pads its scroll region, with the floating back pill and centred name group positioned absolutely over it.

A light app paints behind the status bar, and the clip box has to allow it

PhoneFrame pads its children down by 62px to clear the status bar. A light app — Messages, Mail — cancels that with marginTop: -62 plus minHeight: calc(100% + 62px), so its own white background reaches the top of the screen and the dark-tone clock reads against it. That is how iOS looks, and it is why statusBarTone switches to dark for those apps.

AppShell wraps every non-home app in an overflow-hidden flex column, which clipped exactly that 62px band. The wrapper now carries the same marginTop: -62 / paddingTop: 62 pair: the clip box starts at the top of the screen while every child still lays out where it did before.

Warning: the symptom does not look like clipping. The band goes black, and a dark-tone status bar is black-on-black — so the screen reads as "the status bar is missing on Messages", not "something is clipped". getBoundingClientRect() is no help either: it reports the unclipped border box, so the panel measures as starting at the top of the screen while painting 62px lower. Compare a pixel against the DOM when they disagree.

Notifications reset per session, and reads persist server-side

useNotifications (use-notifications.ts) subscribes to the demo-notification SSE stream (/api/phone/notifications/stream). Two behaviours exist because of a real report — messages from earlier test calls piling up, and a banner sliding in minutes after a test ended:

• A new session is a clean slate. The SSE effect clears notifications, latestNotification, and the seen-id set whenever sessionId changes — each Preview test call is a new conversation id, so the previous run's messages never carry over.
• A replay must not re-raise the banner. conversation-api replays every unread notification on each SSE connect, and EventSource reconnects on its own — so the hook tracks seen ids and only sets latestNotification for a first-sighted, unread notification. The list still repopulates (dedup by id); only the banner is gated.
• Reads persist. markAsRead PATCHes {markReadEndpoint}/{id}/read?session_id=… when the host config provides markReadEndpoint (all four ai-manager config producers pass /api/phone/notifications; the BFF route proxies to conversation-api's PATCH /api/notifications/{id}/read, which scopes the UPDATE to the session when session_id is supplied). Without persistence, every reconnect replayed the "unread" rows forever. The fetch is fire-and-forget — a failed persist costs one extra replay, never an error.

Preview can opt into real SMS

Agent Studio Preview normally stamps X-Multimodal-Sms-Delivery: demo_notification and routes every capability message to this notification stream. Set NEXT_PUBLIC_FEATURE_REAL_SMS_TESTING=true at ai-manager build time to show SMS in the full-screen Preview header. When enabled, Preview requires a non-blank ani Template Variable, stamps real_sms plus that ANI, and keeps only the audio leg in the browser emulator. Multimodal links, Customer Authentication codes/links, and future features using the shared SMS delivery setting all switch together.

The feature flag and mode both default off. This is deliberate: merely opening Preview must never make a carrier send possible, and real mode must never substitute the synthetic +15550100000 demo handset number when ANI is blank.

Connect Telephony Preview uses a server-owned contact binding to carry the same delivery choice through Voice Connector/ARI. The notification stream uses the returned notificationSessionId; AI Activity and the multimodal surface use the separately correlated TrueServe conversation. A provider contact ID must not activate an AI runtime. See Connect Preview.

See also

• ai-manager file layout
• A2UI rendering
• Shared libraries
• Testing with the emulator (user guide)

Multimodal styles inside the phone

The catalog component preview is the visual reference for Agent Preview. Compare the same published bundle, component data, width, and color scheme. A defect that appears identically in both previews is a catalog issue, not a phone styling regression.

PhoneMultimodalStyleBoundary portals the native React cards into a same-origin blank iframe inside Safari. This isolates font-face lookup as well as CSS; a shadow root alone still sees Studio’s loaded fonts. The iframe contains only the existing native card tree, not a second Surface App session. Actions retain their native callbacks, and a resize observer keeps scrolling in the phone’s Safari viewport. It loads apps/ai-manager/public/phone-catalog-preview.css, generated from Surface App's stylesheet and default theme. Studio's global styles must not enter this boundary: Studio and Surface App use different color tokens and defaults. The boundary mirrors only runtime catalog stylesheet links, including later insertions, replacements, and removals. It preserves the catalog preview's existing selector scope rather than adding phone-only catalog scope classes. The native surface uses the same 16px padding as the catalog preview.

The stylesheet generator compiles Surface App's actual source scan and includes its default theme. The phone follows the browser color scheme, matching a fresh catalog preview's default; a custom theme stored independently on the Surface App origin is not synchronized to Studio. No normal Surface App selectors escape into the phone chrome or Studio.

After changing Surface App styling, theme definitions, or classes used by its source scan, run node apps/ai-manager/scripts/generate-phone-catalog-styles.mjs from the repository root. Commit the generated CSS file. Use --check to detect drift. Generation requires dependencies installed in both apps; deployed Studio images consume the checked-in files and do not require Surface App sources or an extra runtime request to that service.

The regression spec apps/ai-manager/e2e/as-285-multimodal-css.spec.ts compares computed styles against an independently compiled Surface App reference in both color schemes. It also verifies isolation from Studio/phone chrome, late catalog CSS, and preserved action state. For live validation, open an agent Preview call, follow its simulated multimodal message, and compare its card with the catalog component preview. Wait for chart animations and keep pointer/selection state identical when comparing screenshots.

Director Scene playback

Director consumes the native presentational phone components through AppShell; SMS, email and browser chrome have no separate Scene implementation. PhoneFrame and LockScreen accept an optional clockTime epoch, MessagesApp accepts an optional referenceTime for relative date labels, and MailApp accepts focusedId for controlled playback. Omitting these retains live clock/navigation behavior. Director's phone projection and animation seeking are local to its player; live phone hooks and notification delivery are not mounted by a recording.

Authentication and multimodal share one customer page

Agent Studio wraps opened messages in PhoneCustomerExperience, which polls the same page-bound auth state as Surface App. A later verification temporarily hides the mounted native cards; a completed simulated verification reveals them again without another SMS or losing input state. Auth-first simulation reaches the same landing page. Real WebAuthn hands off to the same share URL at top level. Presence is registered only after the message is opened. See Customer Authentication.