Building and Deploying
Relevant source files
The following files were used as context for generating this wiki page:
The "Building and Deploying" process for ONNX-Station encompasses the local development environment setup, the compilation of machine learning models into the ONNX format via the optimum-cli, and the deployment of a React-based workbench for benchmarking and optimization. The system is designed to facilitate the transition of AI models from frameworks like PyTorch or TensorFlow into optimized ONNX runtimes for web or edge deployment.
Sources: README.md:1-18, package.json:1-42
Local Development Environment
The project is a React application built with Vite and TypeScript. It requires Node.js as a prerequisite and utilizes several key dependencies for UI (Tailwind CSS, Lucide React), visualization (D3), and AI execution (onnxruntime-web).
Installation and Execution
To run the application locally, the environment must be initialized through the following steps:
- Dependency Installation: Executing
npm installto pull required packages defined inpackage.json. - Configuration: Setting the
GEMINI_API_KEYin a.env.localfile. - Local Server: Launching the development server using
npm run dev.
Sources: README.md:9-18, package.json:6-12
Build Pipeline and Configuration
The project uses Vite for its build pipeline, configured to handle React and Tailwind CSS. The configuration includes specific handling for Hot Module Replacement (HMR) based on environment variables.
The following diagram illustrates the build pipeline from source code to the final distribution bundle.
Sources: vite.config.ts:6-25, package.json:7-8
Model Compilation and Export
A core feature of the workbench is the "Export Strategy," which generates optimum-cli commands to compile models into ONNX format. The application manages the complex configuration required for optimum-cli export onnx.
Export Parameters
The compilation process is governed by the ExportConfig interface, which tracks source frameworks, hardware targets, and optimization levels.
| Parameter | Type | Purpose |
|---|---|---|
modelSourceType | 'hub' | 'local' | Determines if the model is fetched from Hugging Face or a local path. |
opset | number | Sets the ONNX operator version (e.g., 14, 15, 17). |
device | 'cpu' | 'cuda' | Maps to the target runtime execution provider. |
dtype | 'fp32' | 'fp16' | 'bf16' | Controls precision overrides for memory optimization. |
optimize | O1 to O4 | Directs the level of graph transformation and kernel fusion. |
Sources: src/types.ts:31-48, src/WorkbenchContext.tsx:210-230
Compilation Flow
When a user initiates startExport, the WorkbenchContext simulates the multi-step compilation process used by the Optimum CLI.
The sequence diagram shows the internal state transitions during a model export simulation.
Sources: src/WorkbenchContext.tsx:244-315, src/components/CenterCanvas.tsx:49-65
Benchmarking and Runtime Deployment
Once compiled, the application provides a mechanism to benchmark the ONNX model using onnxruntime-web. This allows developers to validate performance metrics like latency and throughput before final deployment.
Execution Providers
The benchmarking utility (runBenchmark) attempts to initialize the InferenceSession using the specified hardware target.
- WebGPU: Attempted when the device is set to
cuda. - WASM: Used as a fallback or when the device is set to
cpu.
Sources: src/lib/benchmark.ts:109-125
Benchmark Logic
The system generates synthetic inputs based on the model's modality (text, vision, audio) and structural dimensions (e.g., sequence_length, batch_size) to execute timed iterations.
// Example of runtime provider resolution
let executionProvider = 'wasm';
if (params.device === 'cuda') {
if ((navigator as any).gpu) {
executionProvider = 'webgpu';
}
}
Sources: src/lib/benchmark.ts:110-118, src/lib/benchmark.ts:25-83
Deployment Artifacts
Final artifacts for deployment are managed via the downloadConfig function, which packages the current ExportConfig into a export_config.json file. This file acts as the deployment manifest for reproducing the optimized model export in a production environment.
Sources: src/WorkbenchContext.tsx:432-443, src/components/CenterCanvas.tsx:28-32
The "Building and Deploying" module ensures that developers can move from high-level model definitions to production-ready ONNX artifacts through a structured, validated, and benchmarked pipeline.