Live Stage & Call UI
Relevant source files
The following files were used as context for generating this wiki page:
The Live Stage & Call UI system provides the primary interface for real-time interaction within the OpenLive platform. It handles the transition from a setup "lobby" to an active voice session, integrating media capture (microphone, camera, screen share), on-device AI model management, and a reactive visual "Orb" that represents the AI's state.
This module is designed as a "thick client" where voice processing—including Voice Activity Detection (VAD), Speech-to-Text (STT), and Text-to-Speech (TTS)—occurs on-device, while the UI coordinates these services with a remote agent via WebSockets.
Session Lifecycle Architecture
The live session lifecycle is managed by the LiveDock component, which acts as a container for three distinct states: Pre-call setup, Active In-call, and Minimized "MiniSphere" mode.
The diagram shows the progression of a live session from initial setup to various UI modes and termination.
Sources: apps/web/src/components/live/LiveDock.tsx:18-62, apps/web/src/lib/live/useLiveSession.ts:122-135
Component Responsibilities
| Component | Role | Key Features |
|---|---|---|
| LiveDock | Orchestrator | Controls visibility of setup vs. active call; manages AnimatePresence. |
| PreCall | Lobby UI | Device selection, Mic meter, Study goal input, Model download progress. |
| InCall | Primary UI | Full-screen Orb, Control bar, Sidebar transcript, Media tiles (Camera/Screen). |
| MiniSphere | Minimized UI | Tiny floating Orb, Focus check-in prompts, App region dragging. |
| TranscriptPanel | History | Word-by-word reveal of assistant text synced to audio playback. |
Sources: apps/web/src/components/live/LiveDock.tsx:20-35, apps/web/src/components/live/InCall.tsx:44-120, apps/web/src/components/live/LiveStage.tsx:50-100
Pre-Call Configuration (Lobby)
The PreCall component serves as a gatekeeper to ensure the environment is ready for a low-latency session. It handles three critical prerequisites:
- Media Permissions: Requests and monitors microphone access using a live
MicMeter. - On-Device Models: Facilitates the download of speech (STT), voice (TTS), and turn-taking models.
- Context Setting: Captures the user's
studyGoalandinterruptLevel(Quiet, Balanced, or Active) to prime the AI persona.
Model Download Management
The system tracks the progress of three distinct models required for local execution.
const MODEL_ROLE: Record<string, string> = {
stt: "hears you",
tts: "speaks back",
turn: "knows when you're done"
};
Sources: apps/web/src/components/live/LiveStage.tsx:16-17, apps/web/src/components/live/LiveStage.tsx:57-95
In-Call Interface & Media Controls
Once active, the UI shifts to the InCall component. This interface is centered around the Orb, a canvas-based visualization that reacts to audio frequency bands.
Media Capture Integration
The system supports simultaneous camera and screen sharing. When active, these streams are displayed as floating tiles:
- CameraPiP: A Picture-in-Picture style camera preview.
- ScreenTile: A large tile for the shared screen, which is also used for the "proactive observe" loop where the AI analyzes the screen content to offer help.
Sequence of media control interactions and their downstream effects on the engine and server connection.
Sources: apps/web/src/lib/live/useLiveSession.ts:360-420, apps/web/src/components/live/InCall.tsx:102-115
The Control Bar
A floating bar at the bottom of the screen provides quick access to core functions:
- Mic/Camera Menus: Allows switching devices mid-call via
ControlWithMenu. - Minimize: Collapses the interface into the
MiniSphere. - End Call: Triggers a teardown that stops all media tracks and closes the WebSocket.
Sources: apps/web/src/components/live/InCall.tsx:115-155, apps/web/src/components/live/LiveDock.tsx:40-45
Transcript and Real-time Feedback
The TranscriptPanel provides a resizable sidebar containing the conversation history. Unlike standard chat interfaces, the assistant's text is revealed word-by-word in lockstep with the voice audio to ensure visual-audio synchronization.
Data Flow for Transcript Reveal
VoiceEnginetriggersonAgentTextwhen an audio chunk starts playing.useLiveSessioncalculates the display duration based on the audio length.- A
requestAnimationFrameloop inuseLiveSessionupdates thechatStoreword-by-word. TranscriptPanelre-renders the partial text.
Sources: apps/web/src/lib/live/useLiveSession.ts:182-210, apps/web/src/components/live/TranscriptPanel.tsx:18-45
Minimized Modes
MiniSphere
A tiny, always-on-top floating sphere (64px) used when the user wants the AI to observe their work without taking up screen real estate. It uses Electron bridge calls (window.openlive.mini()) to resize the native window.
Sources: apps/web/src/components/live/MiniSphere.tsx:18-50
MiniBar
An alternative minimized state that includes a small caption area and compact media controls. It dynamically grows its height to accommodate camera or screen share previews.
Sources: apps/web/src/components/live/MiniBar.tsx:45-85
Technical Summary
The Live Stage & Call UI is a state-driven system that bridges high-performance media processing with a modern React interface. By utilizing useLiveStore for global state and useLiveSession for imperative media/socket control, the system maintains synchronization between on-device AI operations and user-facing UI components like the reactive Orb and synchronized transcript.
Sources: apps/web/src/lib/live/useLiveSession.ts:35-110, apps/web/src/components/live/LiveDock.tsx:10-40