Server-Sent Events (SSE) Streaming

Relevant source files

The following files were used as context for generating this wiki page:

• server.ts
• src/components/features/ExecutionWorkspace.tsx
• src/components/features/InputEnvironmentPanel.tsx
• ORIGINAL_REQUEST.md
• ABOUT.md

The Server-Sent Events (SSE) streaming system in Olive Studio provides a real-time, unidirectional communication channel from the Node.js/Express backend to the React frontend. Its primary purpose is to stream live execution logs, environment setup progress, and status updates during Microsoft Olive optimization runs. This ensures that users receive immediate feedback without the overhead of polling or the complexity of bidirectional WebSockets.

Sources: ABOUT.md, ORIGINAL_REQUEST.md:27-32, server.ts:1062-1100

Architecture and Data Flow

The SSE architecture is built around a job registry on the server and an EventSource listener on the client. When an optimization job is initiated, the server spawns a Python process and captures its stdout and stderr streams. These streams are then pushed to all active SSE subscribers associated with that specific job ID.

Backend Implementation

The server maintains an in-memory jobRegistry to track active and past jobs. Each job object contains a list of historical logs and an array of subscriber callback functions.

Connection Lifecycle

1. Job Initiation: The client sends a POST request to /api/olive/run. The server returns a unique jobId.
2. SSE Connection: The client opens a GET request to /api/olive/stream/:jobId.
3. Header Configuration: The server sets headers for text/event-stream, no-cache, and keep-alive.
4. Log Replay: The server immediately sends all existing logs for that job to the new subscriber.
5. Live Streaming: As the Python process emits data, the server wraps it in a JSON string and writes it to the stream.
6. Termination: When the process exits, the server sends a named done event to close the client-side connection.

Sources: server.ts:1062-1085, src/components/features/ExecutionWorkspace.tsx:505-555

Key Components and Endpoints

API Endpoints

The following endpoints facilitate the SSE streaming lifecycle:

Endpoint	Method	Description
/api/olive/run	POST	Initiates the job, writes the recipe, and triggers the SSE stream start.
/api/olive/stream/:jobId	GET	Establishes the EventSource connection for real-time log streaming.
/api/olive/status/:jobId	GET	Non-streaming endpoint to check current job state and exit codes.
/api/olive/cancel/:jobId	DELETE	Kills the associated process and pushes a cancellation log to the stream.

Sources: server.ts:909, server.ts:1062, server.ts:1103, server.ts:1114

Data Structures

The server utilizes a specific interface to manage job state and subscribers:

interface OliveJob {
  id: string;
  status: "setting_up" | "running" | "completed" | "failed";
  exitCode: number | null;
  logs: string[];
  subscribers: Array<(line: string) => void>; // SSE callbacks
  process: ReturnType<typeof spawn> | null;
}

Sources: server.ts:166-173

Client-Side Integration

The ExecutionWorkspace.tsx component manages the client-side lifecycle of an SSE connection using a useRef to hold the EventSource instance.

Event Handling Logic

The client listens for two types of messages from the server:

• Standard Message: Contains a JSON payload with a line property containing log text.
• Named "done" Event: Signals the completion of the optimization process, providing the final exit code.

Setup and Dependency Streaming

A critical feature of the SSE implementation is streaming the environment setup progress. If the .venv or required Python packages (like olive-ai, onnxruntime-gpu, or tensorrt) are missing, the server performs these installations before running the optimization. These "pre-run" logs are tagged with [setup] or [deps] and streamed through the same SSE channel so the user is aware of the installation progress.

Sources: server.ts:211-230, server.ts:945-960

Conclusion

The SSE Streaming system is the backbone of the "Execute Live" experience in Olive Studio. By leveraging a centralized job registry and subscriber pattern, it provides a robust mechanism for monitoring complex, long-running model optimization tasks and their prerequisite environment configurations.

Sources: ABOUT.md, server.ts:175-180