Multi-Provider AI Client
Relevant source files
The following files were used as context for generating this wiki page:
The Multi-Provider AI Client is a core subsystem of the Oh My Pi project designed to provide a unified interface for interacting with various Large Language Model (LLM) providers. It abstracts the complexities of different API protocols, authentication methods, and message formats into a standardized SDK, allowing the project to seamlessly switch between or parallelize work across providers like OpenAI, Anthropic, Google Vertex, and GitLab Duo Workflow.
This system manages the lifecycle of AI sessions, including model discovery, credential resolution, stream management, and tool-call normalization. By utilizing a "dialect" system, it ensures that project-wide data structures are correctly translated to and from provider-specific formats, maintaining consistency across the entire coding harness.
Sources: packages/coding-agent/DEVELOPMENT.md:104-110, packages/ai/test/gitlab-duo-workflow-provider.test.ts:36-62
Core Architecture and Data Flow
The architecture is built around a centralized registry that maps model identifiers to specific provider implementations. When a request is made, the client resolves the necessary credentials, identifies the correct dialect for translation, and establishes a communication channel (typically via HTTP or WebSockets).
The diagram illustrates the flow of an AI request from initial client call to normalized output.
Sources: packages/coding-agent/DEVELOPMENT.md:107-108, packages/coding-agent/src/extensibility/plugins/legacy-pi-bundled-registry.ts:60-140
Key Components
| Component | Responsibility |
|---|---|
| Model Registry | Maps model IDs to providers and manages model capabilities (tokens, cost, context window). |
| Dialect System | Translates internal OMP messages/tools into provider-specific schemas (e.g., ChatML, Anthropic, OpenAI). |
| Auth Broker | Manages OAuth flows, API keys, and secure credential storage. |
| Stream Handler | Processes incremental deltas for text and tool calls during model inference. |
Sources: packages/coding-agent/DEVELOPMENT.md:55-58, packages/ai/test/gitlab-duo-workflow-provider.test.ts:50-60
Provider Implementation: GitLab Duo Workflow
The GitLab Duo Workflow provider is a specialized implementation that uses WebSockets for bi-directional communication. It demonstrates the client's ability to handle complex, stateful interactions beyond simple REST requests.
WebSocket Communication Lifecycle
The provider manages a WebSocket state machine that handles initial handshakes, ambient workflow creation, and terminal status detection.
The sequence diagram shows the specialized handshake and communication loop for the GitLab Duo Workflow provider.
Sources: packages/ai/test/gitlab-duo-workflow-provider.test.ts:121-140, packages/ai/test/gitlab-duo-workflow-provider.test.ts:665-685
Connection Parameters
The client builds specific headers and URLs to maintain compatibility with the GitLab ecosystem:
- Client Type:
node-websocket - Language Server Version:
8.104.0 - Headers: Includes
x-gitlab-root-namespace-idandauthorizationwith the Rails token.
Sources: packages/ai/test/gitlab-duo-workflow-provider.test.ts:135-150
Message and Tool Normalization
To maintain a provider-agnostic core, the client normalizes all inputs and outputs. This includes stripping internal metadata (like intent tracing) before sending to the model and re-injecting it into the assistant's response.
Dialect Translation Examples
- ChatML: Used as a "bare transcript" for certain providers, rendering user, assistant, and tool roles into a single goal string.
- MCP Schema: Tools are advertised under bare names (e.g.,
read,write) with official JSON schemas translated at the dialect level.
Sources: packages/ai/src/dialect/index.ts, packages/ai/test/gitlab-duo-workflow-provider.test.ts:184-200, packages/ai/test/gitlab-duo-workflow-provider.test.ts:250-280
Stall and Error Handling
The client includes proactive logic to detect stalled workflows. If consecutive tool-call boundaries carry byte-identical checkpoints, the client identifies a stall, stops the workflow server-side, and re-seeds a fresh workflow with the full conversation history.
Sources: packages/ai/test/gitlab-duo-workflow-provider.test.ts:1040-1060, packages/ai/test/gitlab-duo-workflow-provider.test.ts:1110-1130
Internal SDK Registry
The system exposes its capabilities through a bundled registry. This allows extension modules and other packages in the monorepo to access live module references for AI processing, especially in compiled-binary mode.
// Example of the static handles in the bundled registry
export const BUNDLED_PI_REGISTRY: Readonly<Record<string, any>> = {
"@oh-my-pi/pi-ai": bundledPiAi,
"@oh-my-pi/pi-ai/error": bundledPiAiError,
"@oh-my-pi/pi-ai/auth-broker": bundledPiAiAuthBroker,
"@oh-my-pi/pi-ai/dialect": bundledPiAiDialect,
// ... additional provider paths
};
Sources: packages/coding-agent/src/extensibility/plugins/legacy-pi-bundled-registry.ts:610-630
Summary
The Multi-Provider AI Client serves as the essential bridge between the Oh My Pi coding harness and diverse AI ecosystems. By centralizing authentication, dialect translation, and stream management, it enables robust features like subagent delegation and parallel task execution. Its ability to handle complex stateful protocols, such as GitLab Duo Workflow's WebSocket interface, ensures the project remains flexible and future-proof as new AI providers and protocols emerge.