Supported LLM Providers
Relevant source files
The following files were used as context for generating this wiki page:- packages/ai/src/providers/openai-completions.ts
- packages/ai/src/providers/anthropic.ts
- packages/ai/src/providers/google.ts
- packages/ai/src/types.ts
- packages/ai/test/stream.test.ts
- packages/coding-agent/src/main.ts
- packages/mom/src/agent.ts
- README.md
- packages/ai/test/empty.test.ts
- packages/ai/test/unicode-surrogate.test.ts
The @mariozechner/pi-ai package provides a unified, multi-provider API for interacting with various Large Language Models (LLMs). It is designed to abstract away provider-specific implementations, allowing applications like the coding-agent and mom Slack bot to use a consistent interface for features like text generation, tool calling, image input, and streaming.
The core of this system revolves around the stream and complete functions, which are extensively tested across all supported providers to ensure consistent behavior. This unified approach simplifies development and allows for seamless switching between different models and providers.
Sources: README.md:5, packages/coding-agent/src/main.ts:1, packages/mom/src/agent.ts:1-2, packages/ai/test/stream.test.ts:18
Core Concepts
The provider abstraction is built on a set of unified interfaces and a dynamic transport layer.
Unified Interfaces
The system defines a standard set of types to represent models, messages, and context, ensuring a consistent data structure regardless of the underlying provider.
The following diagram illustrates the main data structures.
This diagram shows the central Context object, which holds Message objects and a Model definition.
Sources: packages/ai/src/types.ts:5-167
Provider Transport
The ProviderTransport class is responsible for handling the communication with the LLM APIs. A key feature is its ability to dynamically resolve API keys at request time based on the current model's provider. This allows an agent to switch between models from different providers without manual key management.
The following sequence diagram shows how an API key is resolved when an agent makes a prompt.
Provider Overview
The platform supports a wide range of cloud and self-hosted LLM providers, often by leveraging common API compatibility layers. API keys are typically configured via environment variables.
| Provider Name | KnownProvider ID | Default Model (in coding-agent) | API Key Environment Variable |
|---|---|---|---|
| Anthropic | anthropic | claude-sonnet-4-5 | ANTHROPIC_API_KEY or ANTHROPIC_OAUTH_TOKEN |
| OpenAI | openai | gpt-5.1-codex | OPENAI_API_KEY |
google | gemini-2.5-pro | GEMINI_API_KEY | |
| Groq | groq | openai/gpt-oss-120b | GROQ_API_KEY |
| xAI | xai | grok-4-fast-non-reasoning | XAI_API_KEY |
| Cerebras | cerebras | zai-glm-4.6 | CEREBRAS_API_KEY |
| OpenRouter | openrouter | openai/gpt-5.1-codex | OPENROUTER_API_KEY |
| ZAI | zai | glm-4.6 | ZAI_API_KEY |
| Ollama | ollama | gpt-oss:20b (in tests) | (N/A - uses Base URL) |
Sources: packages/coding-agent/src/main.ts:40-47, 308-325, packages/ai/test/stream.test.ts:264-489
API Compatibility Layers
To manage the diversity of provider APIs, the system uses several compatibility layers, defined by the Api type. A given Provider is mapped to a specific Api implementation.
This diagram shows how multiple providers can be mapped to a single API compatibility layer, such as openai-completions.
Sources: packages/ai/src/types.ts:5, packages/ai/test/stream.test.ts
openai-completions
This is the most widely used compatibility layer, supporting providers that adhere to the OpenAI Completions API format. This includes:
- Groq
- xAI
- Cerebras
- OpenRouter
- Local Ollama instances
This layer includes an OpenAICompat configuration object to handle minor variations between providers, such as different field names for max_tokens or support for specific features like reasoning_effort.
Sources: packages/ai/test/stream.test.ts:311, 353, 374, 395, 456, packages/ai/src/types.ts:165-175
openai-responses
This layer is for newer OpenAI models that use the "Responses" API, such as gpt-5-mini. It supports features like reasoningEffort.
Sources: packages/ai/test/stream.test.ts:288
anthropic-messages
This layer supports Anthropic's native Messages API. It is also used by compatible providers like ZAI. It has a thinkingEnabled option to request reasoning from the model.
Sources: packages/ai/test/stream.test.ts:332, 416, packages/mom/src/agent.ts:8
google-generative-ai
This layer is dedicated to Google's Gemini family of models. It supports a thinking option that can be configured with a token budget.
Sources: packages/ai/test/stream.test.ts:265, 277
Testing and Validation
The reliability of the multi-provider system is ensured through a comprehensive end-to-end test suite located in packages/ai/test/. The stream.test.ts file is particularly important, as it runs a series of standardized tests against each supported provider to validate core functionality.
The following features are systematically tested for each provider:
| Feature | Description | Test Function Example | Source File |
|---|---|---|---|
| Basic Text Generation | Verifies simple request-response cycles. | basicTextGeneration | packages/ai/test/stream.test.ts:35-64 |
| Tool Calling | Ensures the model can correctly identify when to use a tool and parse its arguments. | handleToolCall | packages/ai/test/stream.test.ts:66-127 |
| Streaming | Validates that text_start, text_delta, and text_end events are correctly emitted. | handleStreaming | packages/ai/test/stream.test.ts:129-150 |
| Thinking/Reasoning | Tests the model's ability to provide step-by-step reasoning when requested. | handleThinking | packages/ai/test/stream.test.ts:152-181 |
| Image Input | Checks if vision-capable models can process and describe images. | handleImage | packages/ai/test/stream.test.ts:183-219 |
| Multi-turn Conversation | Simulates a longer conversation involving multiple tool calls and text responses. | multiTurn | packages/ai/test/stream.test.ts:221-262 |
In addition to feature validation, the test suite includes tests for robustness, such as handling empty messages (empty.test.ts) and correctly processing Unicode surrogate pairs in tool results to prevent serialization errors (unicode-surrogate.test.ts).
Sources: packages/ai/test/stream.test.ts, packages/ai/test/empty.test.ts, packages/ai/test/unicode-surrogate.test.ts
Summary
The pi-ai package provides a robust and extensible system for working with a diverse set of LLM providers. By using a unified API, compatibility layers, and a comprehensive test suite, it allows developers to build applications that are agnostic to the underlying model provider. This architecture simplifies development, promotes code reuse, and makes it easy to adopt new models and services as they become available.