System Architecture
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Olive Studio is a local-first desktop web application designed to provide a graphical interface for Microsoft Olive, a model-optimization toolkit. The architecture is structured to facilitate a three-step optimization pipeline: model source selection, hardware target configuration, and recipe execution. Sources: ABOUT.md:7-9, README.md:5-7
The system operates as a hybrid application combining a React-based frontend with a Node.js/Express backend. It emphasizes local execution, automatically managing Python virtual environments and hardware-specific dependencies (such as CUDA, TensorRT, and OpenVINO) to run optimization passes like quantization, pruning, and LoRA adaptation without requiring cloud-based processing. Sources: ABOUT.md:12-23, README.md:9-10
Core Component Stack
The application is divided into three primary layers that handle user interaction, server-side orchestration, and the underlying optimization engine.
| Layer | Technology | Primary Responsibility |
|---|---|---|
| Frontend (UI) | React 19, Tailwind CSS, Radix UI, Lucide Icons | Recipe building, hardware probing visualization, and live log display. |
| Backend (Server) | Node.js, Express, Vite (dev), SSE streaming | Job registry, Python venv management, and process spawning. |
| Optimization | Python 3.9+, olive-ai, ONNX Runtime | Executing optimization passes on local CPU/GPU/NPU hardware. |
Sources: ABOUT.md:27-32, package.json:59-75, README.md:104-106
High-Level Component Relationship
The following diagram illustrates the interaction between the React frontend, the Express server, and the local Python environment.
Sources: server.ts:145-152, server.ts:213-220, README.md:139-147
Backend Execution Engine
The backend serves as the orchestrator for the Olive CLI. It manages the lifecycle of an optimization job, from environment preparation to real-time log streaming. Sources: server.ts:145-156
Job Lifecycle Management
When a user initiates an execution, the server follows a strict sequence to ensure the local environment is capable of running the requested recipe.
- Venv Validation: Checks for the existence of
.venv. If missing, it creates one and installsolive-ai. Sources: server.ts:223-238 - Dependency Inference: Inspects the recipe JSON to identify required Python packages (e.g.,
transformers,onnxruntime-gpu,autoawq). Sources: server.ts:333-345 - Hardware Prep: For GPU targets, it validates CUDA tags and ensures runtime libraries for TensorRT or DirectML are loadable. Sources: server.ts:484-490, server.ts:602-610
- Preflight Check: Runs
olive run --list_required_packagesto validate the RunConfig schema before the actual optimization starts. Sources: server.ts:430-440 - Execution: Spawns the Olive process and captures output. Sources: server.ts:881-890
Job Registry and Streaming
The server maintains an in-memory jobRegistry to track active processes and their logs. Real-time feedback is provided via Server-Sent Events (SSE).
Sources: server.ts:145-160, server.ts:822-828, server.ts:938-945
Recipe Management & Validation
The system relies on a structured JSON format to define optimization pipelines. This structure is strictly validated to prevent execution-time failures.
Structural Validation
The src/lib/oliveRecipeSchema.ts file defines the requirements for a valid recipe, which must include:
input_model: Defines the source (HuggingFace, Local, etc.) and configuration.systems: Specifies theLocalSystemand available accelerators (devices and execution providers).passes: A map of optimization steps (e.g.,OnnxConversion,OnnxQuantization).engine: Configuration for the search strategy, host, and output directories.
Sources: src/lib/oliveRecipeSchema.ts:31-75
Hardware Probe Integration
Before execution, the system probes the local hardware to ensure compatibility between the selected execution provider and the physical hardware.
| Probed Element | Method | Purpose |
|---|---|---|
| NVIDIA GPU | nvidia-smi | Detect VRAM, Driver version, and CUDA capability. |
| AMD GPU | rocm-smi | Detect ROCm product compatibility. |
| OpenVINO | import openvino | Check if Intel optimization runtime is available in Python. |
| ORT Providers | ort.get_available_providers() | Verify which execution providers are currently registered. |
Sources: server.ts:673-680, server.ts:705-710, server.ts:725-730
Data Flow: Model Reconstruction
For large models, the system architecture supports local binary reconstruction of chunked model weights (e.g., .bin.001, .bin.002).
Summary
The Olive Studio architecture is a specialized frontend-backend bridge designed to simplify complex ML optimization workflows. By abstracting Python environment management, hardware-specific library dependencies, and the Olive CLI's JSON configurations into a visual interface, it provides a stable environment for local model optimization. The use of SSE for log streaming and a robust validation layer ensures that developers receive immediate, actionable feedback throughout the optimization lifecycle. Sources: ABOUT.md:11-16, README.md:131-137