Real-time Updates (WebSockets)
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Real-time Updates system in ClassMate leverages WebSocket technology to facilitate live, bi-directional communication between client applications and the AI backend. This system is primarily utilized for streaming audio data during lecture recordings and receiving instantaneous transcription results, speaker identification, and session status updates. By bypassing the overhead of traditional HTTP polling, the system ensures that users receive feedback within seconds of speaking.
Sources: PROJECT_STRUCTURE.md:120-123, web_app/src/main.js:1-15
Architecture and Data Flow
The WebSocket architecture connects the frontend (Web App/PWA) directly to the FastAPI backend. During a recording session, the client captures audio chunks and transmits them as Base64 encoded strings over the WebSocket connection. The backend processes these chunks through the AI pipeline and emits transcription results back to the client.
Client-Side Connection Lifecycle
The client initiates a connection to a specific endpoint structured with the session and user identifiers. Once established, it handles various message types including transcription results and full session transcripts.
The diagram above illustrates the continuous loop of audio chunk transmission and result retrieval.
Sources: web_app/src/main.js:109-145, web_app/src/main.js:180-210
Implementation Details
Connection Endpoint
The WebSocket connection is established using a specific URL pattern that incorporates session and user context for routing and data persistence.
- Endpoint Format:
ws://localhost:8000/api/v1/ws/transcribe/${sessionId}/${userId} - Protocol: Standard WebSocket (WS)
Sources: web_app/src/main.js:129-130
Message Schema
Communication over the WebSocket uses a JSON-based messaging protocol. The following table summarizes the primary message types identified in the system:
| Message Type | Direction | Payload Description |
|---|---|---|
connection_established | Backend -> Client | Confirmation of successful socket handshake. |
audio_chunk | Client -> Backend | Base64 encoded audio data with chunk_index. |
transcription_result | Backend -> Client | Live segment including text, timestamp, speaker, and confidence. |
get_transcript | Client -> Backend | Request for the final processed session text. |
session_transcript | Backend -> Client | Full summary including transcript, speakers, and speaker_statistics. |
error | Backend -> Client | Error details if processing or connection fails. |
Sources: web_app/src/main.js:147-165, web_app/src/main.js:239-250
Audio Chunking Logic
To maintain "real-time" responsiveness, the system does not wait for the entire recording to finish. It utilizes the MediaRecorder API to generate audio slices at 1000ms intervals.
// web_app/src/main.js:103-106
this.mediaRecorder.start(1000); // Start recording in 1-second chunks
Each slice is converted to Base64 via a FileReader before being sent through the socket.
Sources: web_app/src/main.js:103-106, web_app/src/main.js:256-261
UI Integration and Live Feedback
The frontend dynamically updates the user interface as WebSocket messages arrive. This includes:
- Live Transcript Append: Appending new text blocks to the
live-transcriptcontainer. - Speaker Identification: Tagging segments with speaker names and displaying confidence scores.
- Speaker Statistics: Displaying word counts, speaking time, and segment counts upon session completion.
The flow shows how incoming data types trigger specific UI updates.
Sources: web_app/src/main.js:167-210, web_app/src/main.js:335-345
Known Limitations and Identified Gaps
The current implementation of real-time updates faces several architectural challenges as identified in project audits:
- Connection Persistence: Background recording support is currently lacking; if the browser tab is hidden or killed, the WebSocket connection may terminate, leading to data loss.
- Error Recovery: While the system handles basic disconnects, there is no robust "retry-with-backoff" mechanism or chunk deduplication if a chunk is re-sent after a brief network interruption.
- Scaling: The backend currently uses an in-memory dictionary for rate limiting, which is not shared across multi-worker deployments.
Sources: GAPS_IMPLEMENTATION_SUMMARY.md:10-15, GAPS_IMPLEMENTATION_SUMMARY.md:120-130, COMPETITIVE_ANALYSIS.md:50-60
Conclusion
The WebSocket-based Real-time Updates system is a core component of the ClassMate "Recording Pipeline," transforming a standard recorder into an interactive AI assistant. By providing sub-second transcription and speaker diarization feedback, it enables a collaborative environment where lecture content is made visible as it happens. Significant future work is planned to improve the robustness of these connections against network instability and background execution constraints.