Model Adapters (Harness)
Relevant source files
The following files were used as context for generating this wiki page:
Model Adapters (Harness) is a centralized abstraction layer located in packages/harness that enables the OpenLive project to interact with multiple Large Language Model (LLM) providers through standardized interfaces. It handles the translation of canonical internal message formats into provider-specific wire protocols, manages streaming Server-Sent Events (SSE), and provides utilities for model discovery and capability enrichment.
The harness supports three primary protocols: anthropic (for Claude and MiniMax), openai (for the Responses API and Ollama), and openai-chat (the universal dialect used by Groq, Together, DeepSeek, and others). By isolating these complexities, the rest of the application can perform complex tasks like delegated web searches and real-time vision analysis without needing to implement provider-specific logic.
Sources: CONTRIBUTING.md:14, packages/harness/src/types.ts:60-65, services/agent/src/live/worker.ts:40-45
Core Architecture and Protocols
The architecture is built around a "Protocol" system that selects the appropriate adapter for a given API endpoint. This allows the system to treat disparate services as interchangeable components.
Supported Wire Protocols
| Protocol | Target Adapters / Services | Key Features |
|---|---|---|
anthropic | Anthropic Claude, MiniMax | Ephemeral prompt caching, adaptive thinking, signed reasoning blocks. |
openai | OpenAI Responses API, Ollama (v0.13.3+) | Native reasoning effort controls, high-performance streaming. |
openai-chat | Groq, DeepSeek, Mistral, xAI, etc. | Standardized Chat Completions interface for broad compatibility. |
Sources: packages/harness/src/types.ts:60-65, packages/harness/src/anthropic.ts:47-60
Message Normalization Flow
The harness converts internal Message objects (which can contain text, images, and tool calls) into the specific JSON structure required by the target provider. For example, the Anthropic adapter maps system messages to a top-level system field and handles "thinking" blocks required for replaying conversations with tools.
The diagram shows how internal message formats are routed through protocol-specific adapters to produce the final API payloads.
Sources: packages/harness/src/anthropic.ts:7-40, packages/harness/src/types.ts:35-43
Advanced Features and Optimization
Prompt Caching and Latency Reduction
To maintain sub-second response times in live voice loops, the harness implements aggressive caching strategies. In the anthropic adapter, ephemeral cache_control blocks are added to the system prompt and the last tool definition. This ensures that the context prefix is cached at approximately 10% of the normal input price and significantly faster read speeds.
Sources: packages/harness/src/anthropic.ts:47-58, services/agent/src/live/turn-runner.ts:117-125
Reasoning and Thinking Controls
The harness manages "Reasoning Effort" levels (low, medium, high, xhigh, max). These levels are mapped differently depending on the protocol:
- Anthropic: Uses a
thinking-tokenbudget oradaptivedisplay modes. - OpenAI: Maps directly to the
reasoning_effortparameter (limited to low/medium/high). - Worker Subagents: Often use "minimal" effort to ensure fast tool execution during background tasks.
Sources: packages/harness/src/types.ts:14-25, services/agent/src/live/worker.ts:48-50
Vision and Tool Support
The harness tracks model capabilities to prevent invalid requests. For instance, it identifies models that do not support tools (like certain Ollama vision models) and provides logic to route frames to dedicated vision models if the primary model is "blind."
This flow illustrates the logic used by LiveTurnRunner to handle vision capabilities before passing data to the harness.
Sources: services/agent/src/live/turn-runner.ts:178-210, apps/web/src/components/settings/ModelsSettings.tsx:140-155
Model Discovery and Management
The fetchModels function in packages/harness/src/models.ts is responsible for aggregating available models. It attempts to fetch live data from the provider's /models endpoint and enriches it with metadata from a local catalog or offline snapshot.
Model Capability Enrichment
| Field | Source / Logic |
|---|---|
contextWindow | Derived from provider payload or inputTokenLimit. |
vision | Detected via modalities (OpenAI) or architecture.input_modalities (OpenRouter). |
reasoning | Heuristic based on model ID or explicit metadata from the catalog. |
cost | Parsed from pricing (OpenRouter) or catalog metadata (USD per 1M tokens). |
Sources: packages/harness/src/models.ts:51-72, packages/harness/src/types.ts:85-98
Usage in Live Systems
The harness is primarily consumed by the LiveTurnRunner and Worker systems.
- Main Agent: Uses the harness to manage a long-running conversation history, often pruning old turns to stay within context limits while preserving tool-result parity.
- Worker Subagent: Spawns short-lived harness sessions to execute specific tasks like
web_searchorfetch_url. The worker is designed to be "non-spoken," meaning its text deltas are discarded while its tool activity is streamed to the UI.
Sources: services/agent/src/live/turn-runner.ts:128-140, services/agent/src/live/worker.ts:38-55
Implementation Example: Anthropic Stream
// From packages/harness/src/anthropic.ts:60-100
export async function* streamAnthropic(opts: {
baseURL: string,
apiKey?: string,
req: ChatRequest,
signal: AbortSignal,
quirks?: ProviderQuirks
}): AsyncGenerator<ProviderEvent> {
const { system, messages } = toAnthropic(opts.req.messages, opts.quirks?.noCacheControl);
// Implementation handles SSE line parsing and yields ProviderEvents
// such as 'text', 'reasoning', and 'tool_start'.
}
Sources: packages/harness/src/anthropic.ts:60-65, packages/harness/src/types.ts:100-110
The Model Adapters (Harness) serves as the critical bridge between the high-level agent logic and the diverse landscape of AI providers, ensuring that OpenLive remains performant, cost-effective, and protocol-agnostic.