Enterprise Infrastructure
Relevant source files
The following files were used as context for generating this wiki page:
Enterprise Infrastructure in Olive Studio refers to the system of shared caching, remote data sourcing, and distributed processing capabilities that enable large-scale model optimization workflows. It is designed to minimize redundant processing across teams by leveraging cloud-based storage and standardized local environment configurations.
The infrastructure scope covers three primary areas: local environment management via Python virtual environments, integration with Azure ML for remote model assets, and a shared caching mechanism that can switch between local storage and Azure Blob storage depending on the deployment scale.
Sources: ABOUT.md, ORIGINAL_REQUEST.md:21-36
Shared Caching and Distributed Infrastructure
Olive Studio implements a dual-mode caching strategy. By default, the system utilizes a local cache directory to store intermediate optimization passes. For enterprise environments, the application supports "Distributed Caching," which redirects the Olive engine's cache directory to an Azure Blob storage container.
Cache Configuration Logic
The system determines the engine.cache_dir in the generated Olive recipe based on the presence of an Azure connection string and the state of the distributed caching toggle.
The logic ensures that if distributed caching is enabled and a valid Azure connection string is provided, the optimization engine prioritizes the remote shared cache to facilitate collaboration across different compute nodes.
Sources: src/components/features/InputEnvironmentPanel.tsx:1200-1234, ORIGINAL_REQUEST.md:112-117
Infrastructure Components
| Component | Description | Default / Requirement |
|---|---|---|
| Local Cache Directory | Path on the local filesystem for storing Olive session artifacts. | ~/.cache/olive |
| Azure Blob Connection String | Credentials for accessing shared enterprise storage. | Required for Distributed Caching |
| Distributed Caching Toggle | UI switch in EnterpriseInfraPanel that controls the recipe generation logic. | false |
| Azure ML Model Path | Remote URI for sourcing models directly from Azure ML Workspaces. | azureml://... |
Sources: src/components/features/InputEnvironmentPanel.tsx:1200-1234, src/lib/aiWorkspaceContext.ts:104-108
Azure ML Integration
For enterprise workflows that do not rely on local files or the Hugging Face Hub, the infrastructure provides a dedicated "Azure ML" source. This allows users to point to existing model versions within an Azure ML Workspace using the standard azureml:// URI format.
Data Flow for Remote Assets
When an Azure model path is specified, the frontend updates the UIState which is then used by the AiWorkspaceContext and recipePipeline to build the input_model section of the Olive JSON.
Local Runtime Infrastructure
The "Enterprise Infrastructure" also encompasses the local execution environment. To ensure consistency across different developer machines, the backend automatically manages a Python virtual environment (.venv).
Automated Venv Management
The server-side infrastructure performs the following steps during the initial execution trigger:
- Dependency Probe: Checks for the existence of
.venvin the project root. - Environment Initialization: Creates the environment using
python -m venv .venv. - Package Provisioning: Installs
olive-aiand hardware-specific dependencies (e.g.,onnxruntime-gpu,nvidia-cudnn-cu12). - Log Streaming: Streams setup progress (e.g., "Installing olive-ai...") to the frontend via Server-Sent Events (SSE).
Sources: ORIGINAL_REQUEST.md:21-36, README.md:55-65
Batch Processing Infrastructure
The infrastructure supports a batch queue system where multiple optimization jobs can be staged with shared validation rules. This is critical for enterprise "sweep" operations where a single model might be optimized for multiple hardware targets (CPU, GPU, and NPU) simultaneously.
Summary
The Enterprise Infrastructure of Olive Studio provides the necessary bridge between local development and cloud-scale deployment. By integrating Azure Blob storage for distributed caching and Azure ML for remote asset management, it enables teams to share optimization results and scale their workloads beyond a single workstation while maintaining a consistent, automated local execution environment.