WebGPU Benchmarking
Relevant source files
The following files were used as context for generating this wiki page:
WebGPU Benchmarking within Olive Studio provides a localized environment for validating and measuring the performance of optimized ONNX models directly in the user's browser. It serves as a critical bridge between server-side model optimization (conversion, quantization, etc.) and real-world edge deployment. By leveraging onnxruntime-web, users can probe their local hardware, estimate VRAM usage, and execute inference cycles to capture detailed latency and throughput metrics without leaving the application.
The benchmarking system is integrated into the ExecutionWorkspace, allowing developers to test models exported from Olive recipes. This ensures that optimization passes like INT4/INT8 quantization or graph transforms yield the expected performance gains on target web hardware, specifically identifying capabilities for WebGpuExecutionProvider and WasmExecutionProvider.
Sources: ABOUT.md:10-20, src/components/features/ExecutionWorkspace.tsx:640-660, src/components/features/WebGpuBenchmarkPanel.tsx:160-190
Architecture and Core Logic
The WebGPU benchmarking feature is built upon a reactive architecture using React and onnxruntime-web. The system initializes by querying the browser's navigator.gpu to identify available hardware adapters and supported features.
Initialization Flow
When the benchmark panel is mounted, it performs a three-step initialization:
- Hardware Probing: Requests the GPU adapter and extracts
GPUAdapterInfo(vendor, architecture) and device limits (e.g.,maxBufferSize). - Pipeline Validation: Optionally uses
TypeGPUto ensure the compute pipeline is ready for post-processing tasks. - Runtime Loading: Dynamically imports
onnxruntime-webto prepare the inference session.
Benchmark Execution Lifecycle
The execution logic follows a structured lifecycle to ensure accurate measurement, involving session creation with specific execution providers (EPs) and a mandatory warm-up phase to mitigate JIT compilation overhead.
Components and Data Structures
Key Data Structures
The benchmarking process relies on several interfaces to track hardware capabilities and performance results.
| Structure | Purpose | Key Fields |
|---|---|---|
AdapterInfo | Stores GPU hardware metadata. | name, vendor, features, maxBufferSize |
BenchmarkResult | Contains performance telemetry. | avgMs, p50Ms, p99Ms, throughputPerSec, latencies |
BenchmarkPreset | Defines run configurations. | iterations, warmup, batchSize |
Sources: src/components/features/WebGpuBenchmarkPanel.tsx:34-75
Configurable Presets
The system provides three standard presets to balance speed and accuracy:
- Quick: 10 iterations, 2 warmup runs.
- Standard: 50 iterations, 5 warmup runs.
- Thorough: 200 iterations, 10 warmup runs.
Sources: src/components/features/WebGpuBenchmarkPanel.tsx:77-81
Performance Metrics and Visualization
The benchmarking module calculates high-granularity timing data to provide a comprehensive performance profile.
Statistical Calculations
Latency metrics are captured using performance.now() for sub-millisecond precision. The system specifically calculates:
- Average Latency: Total execution time divided by iterations.
- Percentiles (p50, p99): Calculated by sorting the
latenciesarray to identify median and tail-end performance. - Throughput: Expressed as iterations per second (
1000 / avgMs).
Sources: src/components/features/WebGpuBenchmarkPanel.tsx:119-130, src/components/features/WebGpuBenchmarkPanel.tsx:325-336
VRAM and Memory Estimation
Before and during execution, the system estimates memory footprints:
- Model Size: Derived from the
arrayBufferlength of the selected file. - Estimated GPU Memory: Calculated using the formula
(modelSizeBytes * 1.5 + tensorSizeBytes * 2) / (1024 * 1024)to account for model weights and intermediate activations. - Hardware Probe: In the wider project context, a hardware probe detects available VRAM to flag potential out-of-memory issues before the benchmark starts.
Sources: src/components/features/WebGpuBenchmarkPanel.tsx:288-291, src/components/features/VramEstimateBanner.tsx:50-70
Integration with Olive Workflows
WebGPU benchmarking is integrated into the ExecutionWorkspace as a primary view mode alongside the Recipe Graph and JSON Editor.
OWR Export Helper
A critical function of the benchmarking context is generating configuration for ONNX Runtime Web/Mobile (OWR). This allows users to bundle the optimized model with:
ort_config.json: Session options includingexecution_providers(WebGPU/WASM) andintra_op_num_threads.onnx_model_manifest.json: Metadata about the architecture and quantization applied.web_init.js: Boilerplate initialization code for loading the model and running inference in a web application.
Sources: src/components/features/ExecutionWorkspace.tsx:106-180, src/components/features/ExecutionWorkspace.tsx:714-730
Summary
The WebGPU Benchmarking system provides developers with an essential toolkit for validating model optimizations within the browser environment. By combining hardware-aware probing, precision telemetry, and export-ready configuration generation, it ensures that models optimized via Microsoft Olive are performance-verified for edge deployment on modern web hardware.
Sources: ABOUT.md:5-15, src/components/features/WebGpuBenchmarkPanel.tsx:610-625