Adding a New Provider

Relevant source files

The following files were used as context for generating this wiki page:

• providers/base.py
• providers/registry.py
• providers/custom.py
• conf/custom_models.json
• providers/shared/model_capabilities.py
• providers/CLAUDE.md

The Zen MCP Server is designed with a modular and extensible provider system to integrate with various AI model services. Providers act as a standardized abstraction layer, creating a uniform interface for the server to communicate with different AI APIs, such as those from Google, OpenAI, or custom local endpoints like Ollama. This architecture allows developers to easily add support for new AI models without altering the core server logic.

Each provider is responsible for declaring the models it supports, defining their capabilities (e.g., context window, vision support), and handling the specifics of API communication, including request formatting and response parsing. The system uses a central registry to manage and select the appropriate provider for a given model request, ensuring a flexible and scalable multi-model environment. For more high-level context, see the Providers Module documentation.

Sources: providers/base.py, providers/registry.py, providers/CLAUDE.md

Core Architecture

The provider system is built on three key components: the ModelProvider abstract base class, the ModelCapabilities data structure, and the ModelProviderRegistry.

ModelProvider Abstract Base Class

All providers must inherit from the ModelProvider class defined in providers/base.py. This class establishes the contract that every provider must follow.

The following diagram shows the structure of the ModelProvider abstract class and its relationship with a concrete implementation like CustomModelProvider.

This diagram illustrates that CustomModelProvider is a concrete implementation of the abstract ModelProvider interface.

Sources: providers/base.py:11-100, providers/custom.py:27

The key components to implement are:

Member	Type	Description
MODEL_CAPABILITIES	dict	A static dictionary mapping model names to their ModelCapabilities.
get_model_capabilities()	Method	Returns the ModelCapabilities for a specific model.
invoke_model()	Method	Sends a synchronous request to the model's API and returns a single ModelResponse.
invoke_model_streaming()	Method	Sends a request and streams the response back as an iterator of ModelResponse objects.

Sources: providers/base.py:21-100

ModelCapabilities Data Structure

The ModelCapabilities class defines the attributes and features of a specific AI model. This information is used by the server to understand how to interact with a model, what its limitations are, and what features it supports.

Field	Type	Description
model_name	str	The unique identifier for the model.
context_window	int	The maximum number of tokens the model can handle in a single prompt.
max_output_tokens	int	The maximum number of tokens the model can generate in a response.
supports_vision	bool	Indicates if the model can process image inputs.
supports_functions	bool	Indicates if the model supports function calling.
provider_type	ProviderType	The enum identifying the provider (e.g., GOOGLE, OPENAI, CUSTOM).

Sources: providers/shared/model_capabilities.py:11-31

ModelProviderRegistry

The ModelProviderRegistry is a singleton class that discovers, registers, and manages all available providers. It is the central authority for resolving a model name to a specific provider instance.

The registration process is automatic for providers with corresponding API keys set in the environment. The registry maintains a prioritized list to resolve which provider to use if a model is available from multiple sources.

The following flowchart shows how a provider is registered and selected.

This diagram shows the server registers providers at startup based on available credentials and then uses the registry to find the correct provider at request time.

Sources: providers/registry.py:21-120

How to Implement a New Provider

Adding a new provider involves creating a new class that implements the ModelProvider interface and making it discoverable by the registry. The CustomModelProvider serves as a clear example for supporting generic, OpenAI-compatible APIs.

Step 1: Create the Provider Class

Create a new file in the providers/ directory (e.g., my_provider.py). Inside this file, define a class that inherits from ModelProvider.

# providers/my_provider.py
from providers.base import ModelProvider
from providers.shared.model_capabilities import ModelCapabilities
from providers.shared.model_response import ModelResponse

class MyModelProvider(ModelProvider):
    # ... implementation ...

Step 2: Define Model Capabilities

Populate the static MODEL_CAPABILITIES dictionary. This can be hardcoded or loaded from a configuration file. The CustomModelProvider demonstrates loading from conf/custom_models.json.

Example conf/custom_models.json:

{
  "llama3-8b-8192": {
    "model_name": "llama3-8b-8192",
    "context_window": 8192,
    "max_output_tokens": 8192,
    "supports_vision": false,
    "supports_functions": false,
    "provider_type": "custom",
    "temperature_range": [0.0, 2.0]
  }
}

This JSON file defines the capabilities for a custom model, which is then loaded by the CustomModelProvider.

Sources: conf/custom_models.json, providers/custom.py:21-25

Step 3: Implement invoke_model

Implement the logic to call the external API. This typically involves:

1. Constructing the API request payload from the messages and other parameters.
2. Adding authentication headers.
3. Making the HTTP request.
4. Parsing the response and returning a ModelResponse object.

The CustomModelProvider provides a reference implementation for communicating with an OpenAI-compatible API endpoint.

Sources: providers/custom.py:44-86

Step 4: Register the Provider

The ModelProviderRegistry automatically discovers providers if their corresponding environment variables are set. For a custom provider, you would typically add logic to the registry's initialization to register it if its configuration is present.

The registry uses a priority system for provider selection: Native API > Custom Endpoint > OpenRouter.

Sources: providers/registry.py:28-48, providers/CLAUDE.md

Example: CustomModelProvider

The CustomModelProvider is a concrete example designed to connect to any OpenAI-compatible API, such as a local Ollama or vLLM instance.

Configuration

It is configured using environment variables and a JSON file.

Environment Variable	Description
CUSTOM_API_URL	The base URL of the custom API endpoint (e.g., http://localhost:11434).
CUSTOM_API_KEY	The API key, if required by the endpoint.
CUSTOM_MODEL_NAME	An optional override for the model name to be sent in the API request.

Sources: providers/custom.py:16-19

Invocation Flow

The following sequence diagram illustrates the process when a request is made to a model handled by the CustomModelProvider.

This diagram shows how the Zen MCP Server delegates the call to the CustomModelProvider, which then communicates with the external custom API.

Sources: providers/custom.py:44-86

Conclusion

The provider architecture in Zen MCP Server is a powerful feature that ensures the platform remains adaptable and future-proof. By adhering to the ModelProvider contract, developers can quickly integrate new AI services, whether they are commercial APIs or local, self-hosted models. The CustomModelProvider serves as an excellent template, demonstrating how to build a flexible provider that can be configured entirely through environment variables and JSON files, minimizing the need for code changes when adding new custom models.