Adding New Model Providers
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The OpenLive system is designed to be provider-agnostic, supporting a wide range of LLM providers including Anthropic, OpenAI, and various OpenAI-compatible services like Groq, DeepSeek, and Ollama. The project uses a "harness" package to abstract the differences between wire protocols, allowing new providers to be added by defining their metadata and mapping them to one of the supported protocols: anthropic, openai (Responses API), or openai-chat (Chat Completions).
Adding a new provider involves registering its metadata in the harness registry, ensuring the correct protocol is selected, and optionally defining "quirks" to handle deviations from standard API formats. Once registered, the system automatically handles model discovery, API key management, and streaming interactions.
Sources: CHANGELOG.md:8-13, packages/harness/src/registry.ts:1-10
Provider Registry and Protocols
The central point for adding a new provider is BUILTIN_PROVIDERS in packages/harness/src/registry.ts. Each entry must implement the ProviderInfo interface.
Supported Protocols
The system categorizes providers into three primary wire protocols:
anthropic: Used for/messagesendpoints (Claude and MiniMax).openai: Used for/responsesendpoints (OpenAI Responses API and newer Ollama versions).openai-chat: The universal/chat/completionsdialect used by Groq, Together, Mistral, xAI, etc.
Sources: packages/harness/src/types.ts:63-68, packages/harness/src/registry.ts:12-100
Provider Configuration Schema
| Field | Type | Description |
|---|---|---|
id | string | Unique identifier for the provider. |
name | string | Display name shown in the UI. |
protocol | Protocol | One of anthropic, openai, or openai-chat. |
baseURL | string | The base URL for the API endpoint. |
envKeys | string[] | Environment variables used to auto-detect API keys. |
catalogId | string | (Optional) Maps to a models.dev key for enriched metadata. |
keyless | boolean | True for local servers (Ollama) that do not require authentication. |
quirks | ProviderQuirks | Specific wire-format deviations (e.g., noCacheControl, bearerAuth). |
Sources: packages/harness/src/types.ts:79-92, packages/harness/src/registry.ts:6-10
Implementation Flow
When a provider is added to the registry, the system follows a specific flow to surface it in the UI and make it functional for the agent.
The diagram above shows the lifecycle of a new provider from registration to being active in a user session.
Sources: packages/harness/src/registry.ts:12-110, packages/harness/src/models.ts:73-88, apps/web/src/components/settings/ModelsSettings.tsx:143-155
Model Discovery and Metadata
Providers typically support dynamic model discovery. The fetchModels function attempts to reach the provider's /models endpoint to retrieve available models.
Live Fetching vs. Snapshots
- Live Fetch: The system calls the provider API (Anthropic or OpenAI-compat) to get the current list.
- Metadata Enrichment: If a
catalogIdis provided, the system fetches metadata frommodels.devto determine features likevision,reasoning, andcontextWindow. - Offline Snapshot: If the API is unreachable, the system falls back to a hardcoded
MODEL_SNAPSHOTdefined in the registry.
Sources: packages/harness/src/models.ts:73-88, packages/harness/src/catalog.ts:46-73, packages/harness/src/registry.ts:114-142
Handling Provider Quirks
Some providers follow a protocol but require minor adjustments. The ProviderQuirks interface allows these to be handled without rewriting the adapter:
bearerAuth: ForcesAuthorization: Bearer <key>even on Anthropic-compatible endpoints (used by MiniMax).noCacheControl: Prevents sendingcache_controlblocks if the provider handles its own caching.noThinking: Disables reasoning/thinking parameters for models where it is always-on or unsupported.
Sources: packages/harness/src/types.ts:71-77, packages/harness/src/registry.ts:32-40
UI Integration
The frontend automatically populates the provider list from the BUILTIN_PROVIDERS export.
Configuration Components
ModelsSettings: The main settings pane where users select providers and save API keys.ProviderKey: Handles the encryption and storage of keys for a specific provider "kind".api.setProviderKey: An API utility that routes key updates to the backend/database.
Sources: apps/web/src/components/settings/ModelsSettings.tsx:21-70, apps/web/src/lib/api.ts:24-30, apps/web/src/app/api/providers/route.ts:16-25
Model Badges and Capabilities
When adding a provider, ensuring accurate metadata is critical for features like the camera loop. The UI uses the vision and reasoning flags to display badges and warn users if a selected model cannot "see" (required for vision-related tools).
Sources: apps/web/src/components/settings/ModelsSettings.tsx:75-90, packages/harness/src/types.ts:104-110
Summary
To add a new model provider to OpenLive, a developer must register it in packages/harness/src/registry.ts with the appropriate protocol and base URL. The system leverages inheritance of common protocols (openai-chat, anthropic, openai) to minimize boilerplate, while the ProviderQuirks and catalogId systems provide the flexibility needed for high-performance features like prompt caching and vision-capable model identification.