Quantization Presets
Relevant source files
The following files were used as context for generating this wiki page:
Quantization Presets in Olive Studio are pre-configured sets of parameters designed to simplify the application of model optimization techniques within the Microsoft Olive ecosystem. These presets map user intent—such as prioritizing latency over accuracy or targeting specific hardware—to valid Olive configuration schemas. They cover a range of algorithms including Post-Training Quantization (PTQ), Activation-aware Weight Quantization (AWQ), and Quantization-Aware Training (QAT).
The system allows users to select from built-in standard configurations, request AI-driven recommendations based on their current hardware and model context, or save custom parameter combinations as reusable presets. This abstraction layer ensures that complex pass dependencies and hardware-specific constraints (e.g., NVIDIA vs. Intel vs. Qualcomm) are handled correctly before a recipe is executed.
Sources: ABOUT.md, README.md, src/components/features/recipe-graph/inspectors/QuantizationInspector.tsx:103-172
Core Architecture and Data Flow
The quantization preset system operates as a bridge between the UI state and the final Olive recipe JSON. When a user selects a preset, the application updates the UIState["passes"] object, which is then used by the oliveRecipeBuilder to generate the finalized configuration for the olive-ai engine.
Preset Selection Logic
Selection is handled through the QuantizationInspector component, which manages three types of presets:
- Built-in Presets: Static configurations defined for common optimization paths (e.g., "Default INT4", "AWQ Balanced").
- Custom Presets: User-defined configurations persisted via local storage helpers (
loadCustomPresets,saveCustomPreset). - AI Recommendations: Dynamic configurations fetched from a specialized endpoint that analyzes the current model and hardware state.
This diagram illustrates the flow from preset selection to recipe generation:
The diagram shows how different preset sources converge to update the application state, which is eventually processed by the recipe builder.
Sources: src/components/features/recipe-graph/inspectors/QuantizationInspector.tsx:174-250, src/lib/tests/oliveRecipeBuilder.test.ts:183-250
Built-in Quantization Presets
The system provides several standardized presets tailored for specific algorithms and performance goals. These are defined in the QUANT_PRESETS constant.
| Preset Label | Description | Key Configuration Fields |
|---|---|---|
| Default INT4 | Broadest compatibility PTQ | quantMethod: "ptq", quantPrecision: "int4" |
| AWQ Balanced | Symmetric 128-group AWQ | quantMethod: "awq", awqGroupSize: 128, awqSym: true |
| GPTQ High Quality | Best accuracy GPTQ | quantMethod: "gptq", gptqBlockSize: 128, gptqDescAct: true |
| QAT INT4 — Best Accuracy | Entropy calibration QAT | quantMethod: "qat", qatCalibrateMethod: "entropy", qatCalibrateSteps: 20 |
Sources: src/components/features/recipe-graph/inspectors/QuantizationInspector.tsx:103-172
AI-Driven Recommendations
Users can utilize the "Ask AI..." feature to automatically configure quantization parameters. The system sends the current UIState—including model source and hardware target—to the /api/ai/recommend-quant endpoint.
AI Recommendation Sequence
The sequence diagram details the asynchronous communication between the client and the AI backend to retrieve optimized quantization settings.
Sources: src/components/features/recipe-graph/inspectors/QuantizationInspector.tsx:174-206
Pass-Specific Configuration Mapping
When a preset is applied, it maps to specific Olive pass types and parameters. The oliveRecipeBuilder translates these into the classes used by the Microsoft Olive engine.
Quantization Method Mapping
- PTQ (Post-Training): Maps to
OnnxQuantization(ONNX) orIncQuantization(Intel). - AWQ: Maps to
AutoAWQQuantizer. - GPTQ: Maps to
GptqQuantizer. - HQQ: Maps to
OnnxHqqQuantization. - RTN: Maps to
OnnxBlockWiseRtnQuantization.
Configuration Logic Example
// From src/lib/__tests__/oliveRecipeBuilder.test.ts:241-249
it("creates AutoAWQQuantizer for AWQ method", () => {
const state = baseState({
ihvProvider: "CUDAExecutionProvider",
passes: { ...DEFAULT_PASSES, quantization: true, quantMethod: "awq", quantPrecision: "int4" },
});
const recipe = buildOliveRecipe(state);
const q = recipe.passes.quantization;
expect(q.type).toBe("AutoAWQQuantizer");
expect(q.config.bits).toBe(4);
});
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:39-85, src/lib/tests/oliveRecipeBuilder.test.ts:241-290
Hardware and Model Constraints
Quantization presets are filtered based on the selected Execution Provider (EP). For instance, specific algorithms like AWQ or GPTQ typically require GPU-based providers (CUDAExecutionProvider, TensorrtExecutionProvider), while INT8 PTQ might be suitable for OpenVINOExecutionProvider (Intel) or QNNExecutionProvider (Qualcomm).
Hardware Compatibility Schema
The ER diagram represents the relationship between hardware capabilities and the optimization methods made available through the preset system.
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:231-255, src/components/features/recipe-graph/inspectors/QuantizationInspector.tsx:210-220
Summary
Quantization Presets serve as a critical usability feature in Olive Studio, enabling developers to navigate the complex landscape of model optimization. By providing a multi-tiered approach—standard defaults, AI-assisted configuration, and custom persistence—the system ensures that users can effectively deploy optimized models across diverse hardware targets like NVIDIA RTX GPUs, Intel CPUs, and Qualcomm NPUs without manual JSON manipulation.