AI Copilot Integration
Relevant source files
The following files were used as context for generating this wiki page:
The AI Copilot Integration in Olive Studio provides an intelligent assistant layer designed to help Machine Learning engineers build, validate, and optimize Microsoft Olive recipes. It functions as a "local-first" advisor that bridges the gap between complex optimization parameters and the user's specific hardware/model constraints. Sources: ABOUT.md, OLIVE_MCP_PROMPT_FOR_DEVELOPER.md:14-18
The system is composed of three primary functional pillars: AI Audit for pipeline efficiency analysis, Assistant Chat for context-aware technical support, and a Configuration Engine supporting multiple LLM providers (Gemini, OpenAI, Anthropic, Mistral). This integration ensures that optimization passes like quantization and pruning are correctly configured before execution. Sources: src/components/features/GeminiSidebar.tsx:142-146, server.ts:25-30
Architecture and Data Flow
The AI Copilot operates by synchronizing the frontend UI state with a backend LLM provider. It uses a specialized "Workspace Context" to ensure the AI has full visibility into the current model source, hardware target, and enabled optimization passes. Sources: src/lib/aiWorkspaceContext.ts:4-33, src/components/features/GeminiSidebar.tsx:327-329
System Workflow
This diagram illustrates the flow from a user action to an AI-generated optimization suggestion.
The system utilizes an Express backend to proxy requests to various AI providers, ensuring API keys remain secure and are not exposed directly in the browser. Sources: server.ts:39-160, src/components/features/GeminiSidebar.tsx:189-211
Key Components
1. AI Workspace Context
The aiWorkspaceContext.ts module is critical for grounding the AI's responses. It transforms the complex UIState into a structured AiWorkspaceContext object that includes model metadata, hardware execution providers, and validation statuses. Sources: src/lib/aiWorkspaceContext.ts:4-33
| Field | Description |
|---|---|
modelSource | Type of input (HuggingFace, Local, Azure). |
hardware | Target execution provider (e.g., CUDA, TensorRT, CPU). |
activePassLabels | Human-readable list of enabled optimization passes. |
validation | Current critical errors or warnings from the pipeline. |
infrastructure | Cache settings and batch queue status. |
Sources: src/lib/aiWorkspaceContext.ts:4-33
2. Assistant Sidebar
The GeminiSidebar.tsx component serves as the primary UI for AI interaction. It features three tabs:
- Audit: Provides an efficiency score (0-100%) and actionable "Autofix" suggestions. Sources: src/components/features/GeminiSidebar.tsx:406-444
- Chat: A conversational interface pre-loaded with "Quick Queries" derived from the current workspace. Sources: src/components/features/GeminiSidebar.tsx:501-526
- Settings: Management of AI providers and API keys (stored in server memory). Sources: src/components/features/GeminiSidebar.tsx:563-652
3. Backend AI Handlers
The server implements specific logic to handle different LLM APIs, such as Gemini's system_instruction or Anthropic's message format. Sources: server.ts:60-142
Integration Points
Automated Optimization Recommendations
The QuantizationInspector.tsx uses a specific endpoint to fetch tailored quantization settings. When a user clicks "✨ Ask AI...", the system analyzes the model and hardware to recommend specific methods like AWQ, GPTQ, or QAT. Sources: src/components/features/recipe-graph/inspectors/QuantizationInspector.tsx:135-155
API Endpoints
| Endpoint | Method | Purpose |
|---|---|---|
/api/ai/provider | GET/POST | Manage active LLM provider and model settings. |
/api/ai/analyze-state | POST | Generates the Audit score and suggestions. |
/api/ai/chat | POST | Handles conversational queries with workspace context. |
/api/ai/validate | POST | Performs expert recipe validation for specific hardware. |
Sources: server.ts:919-1016
Supported Providers and Models
The system supports several mainstream AI providers, allowing users to bring their own API keys. Sources: src/components/features/GeminiSidebar.tsx:18-63
- Google Gemini: Models like
gemini-2.0-flash(standard) andgemini-2.5-pro. - OpenAI: Models including
gpt-4oandgpt-4o-mini. - Anthropic: Support for
claude-sonnet-4-6andclaude-haiku. - Mistral AI: Support for
mistral-large-latest. - OpenAI-Compatible: Allows connection to local providers like Ollama or LM Studio.
Sources: src/components/features/GeminiSidebar.tsx:18-63, server.ts:955-961
Conclusion
The AI Copilot Integration acts as a specialized compiler assistant within Olive Studio. By leveraging real-time workspace context, it provides high-fidelity optimization guidance that reduces the "stderr trace" trial-and-error cycle common in model optimization workflows. Sources: ABOUT.md, OLIVE_MCP_PROMPT_FOR_DEVELOPER.md:14-18