Express API & Server Configuration
Relevant source files
The following files were used as context for generating this wiki page:
The Express API and Server Configuration in Olive Studio acts as the central orchestration layer between the React-based frontend and the Microsoft Olive CLI optimization engine. It manages high-level tasks including Python virtual environment management, hardware-aware dependency installation, optimization job scheduling, and streaming real-time logs via Server-Sent Events (SSE).
The server is designed to facilitate local-first model optimization, providing endpoints for model source configuration, hardware probing, and AI-assisted recipe validation. It ensures that the required execution environments (such as CUDA, TensorRT, or OpenVINO) are correctly initialized before spawning Olive processes.
Sources: ABOUT.md:1-12, server.ts:44-55, README.md:1-10
Architecture and Core Components
Olive Studio uses a Node/Express server that integrates Vite for development and serves a production bundle in production mode. The architecture revolves around a job registry and a series of specialized helpers for environment and hardware management.
Server Lifecycle and Environment
The server initializes by loading environment variables from .env and .env.local. It listens on port 3000 by default and configures Express to handle JSON payloads up to 10MB.
Sources: server.ts:33-55, server.ts:1145-1175, package.json:20-25
Hardware Probe and Dependency Management
A critical function of the server is identifying local hardware capabilities to suggest compatible optimization recipes. It probes for NVIDIA GPUs (via nvidia-smi), AMD ROCm (via rocm-smi), and Intel OpenVINO runtimes.
| Component | Functionality | Key Implementation |
|---|---|---|
| Hardware Probe | Detects GPUs, VRAM, and available ONNX Runtime providers. | probeSystemHardware() |
| Venv Manager | Ensures .venv exists and olive-ai is installed. | ensureVenv() |
| Dependency Resolver | Infers and installs packages like torch, tensorrt, or peft based on recipe needs. | inferRequiredPackages() |
| GPU Launcher | Preloads CUDA DLLs to resolve Windows EP-registration issues. | scripts/olive_gpu_launcher.py |
Sources: server.ts:600-750, README.md:43-55, CONTRIBUTING.md:45-50
Olive Execution Engine
The server manages the execution of Microsoft Olive workflows through a dedicated registry and streaming mechanism.
Job Lifecycle
When a user triggers an optimization, the server validates the JSON recipe, writes it to a temporary file, and spawns a Python process within the virtual environment.
API Endpoints for Optimization
POST /api/olive/run: Receives recipe JSON, triggers setup, and returns ajobId.GET /api/olive/stream/:jobId: Provides an SSE stream of process logs.GET /api/olive/status/:jobId: Returns the current state (setting_up, running, completed, failed).DELETE /api/olive/cancel/:jobId: SendsSIGTERMto the active Olive process.
Sources: server.ts:790-1010
AI Provider Integration
Olive Studio supports multiple AI providers for recipe validation, workspace analysis, and assistant chat. These integrations are configured as stateless proxies in server.ts.
Supported Providers
The server handles request mapping for several LLM backends, normalizing their varying API structures (e.g., Gemini's system_instruction vs. OpenAI's role: system).
| Provider | Method | Default Model |
|---|---|---|
| Google Gemini | callGemini | gemini-2.5-flash |
| OpenAI / Mistral | callOpenAICompat | gpt-4o-mini / mistral-large-latest |
| Anthropic | callAnthropic | claude-haiku-4-5-20251001 |
| OpenAI-Compatible | callOpenAICompat | Custom (via Base URL) |
Sources: server.ts:58-185, src/components/features/GeminiSidebar.tsx:15-50
AI Endpoints
POST /api/ai/validate: Analyzes a recipe for execution failures or suboptimal settings.POST /api/ai/analyze-state: Evaluates the UI state and provides an "Optimization Score" and "Autofixes".POST /api/ai/chat: Handles conversational queries using the current workspace as context.
Sources: server.ts:1062-1140
External Services and Proxies
GitHub Recipe Proxy
To avoid CORS issues when fetching community recipes from the microsoft/olive-recipes repository, the server provides a proxy endpoint.
GET /api/github/raw: Fetches raw JSON files fromraw.githubusercontent.comusing repository slug, branch, and path parameters.
Sources: server.ts:316-368
HuggingFace Token Management
Tokens for accessing gated models are managed via in-memory storage during the session.
GET /api/env/hf-token-status: Checks if a token exists in server memory or system environment variables.POST /api/env/hf-token: Stores a user-provided token in a runtime variable (runtimeHfToken).
Sources: server.ts:1016-1036
Summary
The Express API and Server Configuration provide a robust bridge between the user's intent in the GUI and the low-level requirements of the Microsoft Olive optimization toolkit. By managing Python environments, hardware-specific drivers, and real-time log streaming, it abstracts the complexity of model optimization into a manageable, local-first service architecture.
Sources: README.md:120-135, ABOUT.md:25-30