Home
Relevant source files
The following files were used as context for generating this wiki page:
Optimum Workbench (ONNX-Station) is a technical interface designed to configure, simulate, and benchmark the export of machine learning models to the ONNX (Open Neural Network Exchange) format using the Hugging Face optimum-cli. It provides a local-first environment for developers to experiment with various optimization strategies, hardware targets (CPU vs. CUDA/WebGPU), and precision overrides (fp32, fp16, bf16) before deploying models.
The system integrates high-fidelity simulation of operator distributions and VRAM usage with live benchmarking capabilities using onnxruntime-web. Users can ingest models directly from the Hugging Face Hub or via local file paths, generate CLI request strings for actual exports, and visualize the resulting graph topology and performance metrics.
Sources: README.md, src/WorkbenchContext.tsx:114-150, src/components/CenterCanvas.tsx:16-25
System Architecture
The application is built using React and organized around a central state management provider, the WorkbenchProvider. This provider manages the ExportConfig state, log orchestration, and the execution flows for both model export simulation and performance benchmarking.
Core Components
The user interface is divided into three primary functional areas:
- Sidebar: Handles model sourcing (Hub vs. Local), task definition (Audio, Text, Vision), and preset profile management.
- CenterCanvas: The primary workspace containing CLI command previews, hardware target configuration, and the structural graph visualizer.
- Console: A terminal-like interface providing real-time orchestration logs and system resource monitoring (JS Heap memory).
Sources: src/App.tsx:15-26, src/components/Sidebar.tsx:32-35, src/components/CenterCanvas.tsx:9-13, src/components/Console.tsx:30-35
Data Flow Diagram
The following diagram illustrates the flow from configuration input to the generation of benchmarks and export artifacts.
The application coordinates configuration updates through updateConfig, which triggers re-simulation of model metadata via generateCompiledModelInfo.
Sources: src/WorkbenchContext.tsx:102-112, src/utils/modelInfo.ts:6-9
Model Export Configuration
The platform maps UI inputs directly to optimum-cli arguments. Configurations are categorized into source strategies, task definitions, and compilation directives.
Hardware and Precision Targets
The system supports specific execution providers and data types, with built-in validation for compatibility (e.g., bf16 requiring a cuda device).
| Parameter | CLI Flag | Description |
|---|---|---|
| Target Opset | --opset | ONNX operator version (14, 15, or 17). |
| Device | --device | Target hardware: cpu or cuda (WebGPU). |
| Precision | --dtype | Data type: fp32, fp16, or bf16. |
| Optimization | --optimize | Levels O1-O4, ranging from basic fusion to ORTQuantizer. |
Sources: src/WorkbenchContext.tsx:116-125, src/components/CenterCanvas.tsx:107-155, src/WorkbenchContext.tsx:210-213
System Presets
To facilitate common workflows, the system provides SYSTEM_PRESETS for standardized targets:
- WebGPU fp16 Target: Opset 17, CUDA, fp16, O3 optimization.
- CPU fp32 Portable: Opset 14, CPU, fp32, O1 optimization.
- CUDA bf16 Heavy: Opset 17, CUDA, bf16, O4 optimization.
- Low-Memory Raw Debug: Opset 14, CPU, O0 (none), no post-processing.
Sources: src/WorkbenchContext.tsx:5-41
Graph Visualization & Simulation
Because live exports can be resource-intensive, the system utilizes a high-fidelity simulation engine to estimate graph topology and VRAM requirements before actual compilation.
Simulation Logic
The generateCompiledModelInfo utility computes operator distributions based on the model path and modality:
- Layer Estimation: Identifies model depth (e.g., Whisper Large = 24 layers, LLaMA = 32 layers).
- DType Scaling: Adjusts memory estimates based on precision (e.g., fp16 = 0.50 scale).
- Optimization Impact: Models the reduction in operator counts due to fusions (O1-O3) or the addition of quantization nodes (O4).
Sources: src/utils/modelInfo.ts:11-66
Operator Distribution
The OnnxGraphVisualizer component renders this data using D3.js, showing the frequency of operation types (MatMul, Conv, etc.) and estimated VRAM bottlenecks.
Benchmarking Engine
The benchmark suite uses onnxruntime-web to execute inferences on either synthetic data or ingested local models to measure real-world performance.
Execution Flow
The benchmarking process follows a strict sequence to ensure accuracy:
- Provider Resolution: Attempts to initialize
webgpufor CUDA targets, falling back towasmif unsupported. - Input Generation:
generateSyntheticInputscreates tensors (int64 for text IDs, float32 for pixel/audio values) matching the expected model modality. - Warmup: Executes 3 iterations to prime the execution engine.
- Timed Runs: Performs 20 iterations to calculate Mean, p50, and p95 latencies.
Sources: src/lib/benchmark.ts:109-158, src/lib/benchmark.ts:186-210
Performance Metrics
The system returns a BenchmarkResult containing:
- Latency: Mean, P50, and P95 measurements in milliseconds.
- Throughput: Calculated based on modality (e.g.,
tokens/secfor text,images/secfor vision). - Memory Usage: Estimated MB based on model size and runtime intermediates.
Sources: src/lib/benchmark.ts:21-32, src/lib/benchmark.ts:212-230
Local Model Ingestion
A critical feature for privacy and speed is the "Local-First" ingestion strategy. Users can drag and drop .onnx, .bin, or .pb files directly into the browser.
- Memory Mapping: Files are stored as local Blob URLs (
URL.createObjectURL(file)) and kept in-browser memory. - Sandbox Execution: Benchmarks run against these local blobs without sending weights to a remote server.
- Clean Up:
URL.revokeObjectURLis called when a model is removed to prevent memory leaks.
Sources: src/components/Sidebar.tsx:126-175, src/WorkbenchContext.tsx:128-144
Summary
Optimum Workbench serves as a bridge between high-level model definitions and deployment-ready ONNX artifacts. By providing immediate visual feedback on graph fusions and VRAM allocation alongside a browser-based benchmarking suite, it allows architects to validate their optimization strategies before committing to long-running server-side export processes.