Optimization Passes
Relevant source files
The following files were used as context for generating this wiki page:
Optimization passes are the core functional units within Microsoft Olive that transform a base machine learning model into an optimized version for specific hardware targets. These passes perform a variety of operations including format conversion, precision reduction (quantization), structural simplification (pruning), and fine-tuning (LoRA). Within Olive Studio, these passes are organized into a directed pipeline where the output of one pass typically serves as the input for the next.
The optimization engine supports complex workflows by chaining multiple passes together, such as converting a PyTorch model to ONNX, applying transformer-specific graph optimizations, and finally quantizing the weights to 4-bit or 8-bit precision.
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:27-31, README.md:12-25, ABOUT.md:7-13
Pass Architecture and Categories
Olive Studio categorizes optimization passes based on their primary function. This categorization ensures that users can follow a logical flow from model ingestion to final hardware-specific deployment.
Functional Categories
| Category | Key Passes | Purpose |
|---|---|---|
| Conversion | OnnxConversion, OpenVINOConversion | Transforms models from source frameworks (PyTorch, TF) to inference formats. |
| Quantization | OnnxQuantization, NVModelOptQuantization, IncQuantization | Reduces numerical precision (e.g., FP32 to INT8/INT4) to decrease size and latency. |
| Graph Optimization | OrtTransformersOptimization, OnnxModelOptimizer | Fuses layers, optimizes attention mechanisms, and simplifies the computation graph. |
| Pruning/Sparsity | IncPruning, SparsityFineTuning | Removes redundant parameters or weights based on magnitude or structured patterns. |
| Fine-Tuning/PEFT | LoRA, QLoRA, MultiLoRA | Efficiently adapts models to specific tasks or applies quantized fine-tuning. |
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:33-145, README.md:32-41
Data Flow and Pipeline Logic
The optimization pipeline follows a strict ordering requirement. For instance, quantization or graph optimization passes generally require the model to have been previously converted to a supported inference format like ONNX.
The diagram shows the standard progression from source model to optimized output, overseen by a validation engine that enforces compatibility rules.
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:279-282, src/lib/tests/recipePipeline.test.ts:54-58
Quantization Pass Details
Quantization is the most diverse pass category in Olive, with multiple vendor-specific implementations.
Quantization Methods and Hardware Compatibility
Quantization selection is heavily dependent on the target Execution Provider (EP). Certain advanced algorithms like AWQ (Activation-aware Weight Quantization) or GPTQ are restricted to GPU targets.
- PTQ (Post-Training Quantization): Universal support across CPU, GPU, and NPU.
- AWQ/GPTQ: Requires NVIDIA GPU (CUDA/TensorRT) or ROCm.
- QAT (Quantization-Aware Training): Supported on all except QNN (Qualcomm NPU).
- HQQ (Half-Quadratic Quantization): Calibration-free, supported across most providers.
Sources: src/lib/tests/pipelineValidation.test.ts:44-86, OLIVE_COMPREHENSIVE_API_REFERENCE.md:52-88
Configuration Parameters
Common parameters for quantization passes include:
weight_type: "int8", "uint8", "int4", or "nf4".quant_format: "QOperator" or "QDQ".calibration_data_dir: Directory containing representative data for static quantization.per_channel: Boolean indicating if quantization scales are calculated per output channel.
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:39-50, OLIVE_COMPREHENSIVE_API_REFERENCE.md:65-72
Conversion and Graph Optimization
Conversion passes serve as the entry point for most optimization pipelines.
Format Conversion Logic
The system supports several target formats based on the selected hardware.
The flow illustrates how a PyTorch model can be directed into different optimized formats based on pass selection.
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:92-110, OLIVE_COMPREHENSIVE_API_REFERENCE.md:129-132
Transformer-Specific Optimizations
The OrtTransformersOptimization pass is critical for Large Language Models (LLMs). It handles specialized layer fusions:
- Attention Fusion: Consolidates multiple operations into a single optimized Attention kernel.
- LayerNorm Fusion: Fuses normalization layers with preceding/succeeding operations.
- Embedding Fusion: Optimizes lookups for tokens and positions.
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:118-127
Validation and Constraints
Olive Studio implements a validation layer to prevent "impossible" pass combinations before execution.
Conflict Detection Rules
- Provider Mismatch: Using
OpenVINOConversionwithout theOpenVINOExecutionProvideris considered a critical conflict. - Sequential Dependencies: Quantization cannot occur before ONNX conversion in a standard ONNX pipeline.
- Method Exclusion: AWQ cannot be combined with model pruning passes in a single stage.
- Hardware Limits: NPU targets (like Qualcomm QNN) require specific power-of-2 scales and symmetric quantization.
Sources: src/lib/tests/pipelineValidation.test.ts:162-205, OLIVE_COMPREHENSIVE_API_REFERENCE.md:283-294
Pass Activation Logic
The StepInspector and validation engine manage the state of each pass.
Summary of Key Passes
| Pass Class | Framework | Notable Parameters |
|---|---|---|
OnnxConversion | PyTorch | target_opset, dynamic_axes, input_shapes |
OnnxQuantization | ONNX | quant_format, calibrate_method, weight_type |
LoRA | PyTorch | lora_rank, lora_alpha, target_modules |
IncPruning | PyTorch/ONNX | pruning_type, target_sparsity |
OrtPerfTuning | ONNX | tuning_type, enable_profiling |
Sources: OLIVE_COMPREHENSIVE_API_REFERENCE.md:93-105, OLIVE_COMPREHENSIVE_API_REFERENCE.md:158-166, OLIVE_COMPREHENSIVE_API_REFERENCE.md:183-194
Optimization passes within Olive Studio provide a modular and validated approach to model compression. By enforcing hardware-aware constraints and offering specialized graph transformations, the system ensures that generated recipes are technically viable for the target execution environment.