AI Model Providers
Relevant source files
The following files were used as context for generating this wiki page:
- packages/model-bank/package.json
- packages/model-bank/src/aiModels/index.ts
- packages/model-bank/src/index.ts
- src/app/[variants]/(main)/ai-models/test/page.tsx
- locales/en-US/modelProvider.json
- scripts/seedBundledApps/index.ts
- packages/database/src/repositories/dataImporter/tests/fixtures/agentsToSessions.json
The AI Model Providers system is a modular framework within LobeChat designed to integrate a wide variety of Large Language Model (LLM) services. It acts as a centralized registry, defining the configurations, models, and capabilities for each supported AI provider, such as OpenAI, Anthropic, Google, and many others. This architecture allows the application to seamlessly switch between different models and providers, offering users a flexible and powerful chat experience.
The core of this system is the model-bank package, which exports a comprehensive list of all supported AI providers and their respective models. These configurations are then consumed by the main application, where they are used in various features, including the user-facing provider settings, agent configurations, and bundled applications.
Architecture Overview
The AI Model Provider system is designed around a central package, model-bank, which serves as a single source of truth for all provider and model information. The main application consumes this package to populate provider settings, model selection dropdowns, and to correctly route API requests to the chosen LLM service.
The following diagram illustrates the high-level architecture of the provider system.
This diagram shows that the model-bank package defines and exports provider information, which is then integrated into the main application's configuration. Various parts of the application, like the UI and agent settings, use this configuration to interact with external AI services through the API layer.
Sources: packages/model-bank/package.json, scripts/seedBundledApps/index.ts, packages/database/src/repositories/dataImporter/tests/fixtures/agentsToSessions.json
Model Bank
The model-bank package is a dedicated private package that centralizes the definitions for all supported AI model providers. It is structured to be easily extensible, allowing new providers to be added with minimal effort.
Sources: packages/model-bank/package.json, packages/model-bank/src/aiModels/index.ts
Provider Definitions
Each AI provider is defined in its own file within packages/model-bank/src/aiModels/. These definitions are then aggregated and exported from packages/model-bank/src/aiModels/index.ts. The package.json for model-bank exposes each provider as a separate export, allowing for modular access.
The following table lists the AI model providers available in the model-bank.
| Provider ID | Export Path |
|---|---|
ai21 | ./ai21 |
ai302 | ./ai302 |
anthropic | ./anthropic |
azure | ./azure |
bedrock | ./bedrock |
google | ./google |
groq | ./groq |
ollama | ./ollama |
openai | ./openai |
openrouter | ./openrouter |
qwen | ./qwen |
zhipu | ./zhipu |
| (... and many others) | (...) |
Sources: packages/model-bank/package.json:5-65
Aggregation and Export
The individual provider modules are imported and then exported from a central index file, creating a comprehensive object of all available providers.
// packages/model-bank/src/aiModels/index.ts
import AI21 from './ai21';
import AI302 from './ai302';
// ... other imports
import Zhipu from './zhipu';
export const LobeAI = {
ai21: AI21,
ai302: AI302,
// ... other providers
zhipu: Zhipu,
};
Sources: [packages/model-bank/src/aiModels/index.ts]
This LobeAI object is the primary export consumed by the rest of the application to access provider configurations.
Sources: packages/model-bank/src/index.ts:3-5
Provider Configuration UI
Users can manage AI provider settings, such as API keys and endpoints, through the "AI Provider" section in the application settings. The UI for these configurations is defined by localization files that provide labels, placeholders, and descriptions for each field.
Sources: [locales/en-US/modelProvider.json]
Example: Azure OpenAI Configuration
The configuration for Azure OpenAI requires several specific fields, as detailed in the modelProvider.json locale file.
| Field | UI Title | Description | Placeholder |
|---|---|---|---|
endpoint | Azure API Address | When checking resources from the Azure portal, you can find this value in the 'Keys and Endpoints' section | https://docs-test-001.openai.azure.com |
token | API Key | When checking resources from the Azure portal, you can find this value in the 'Keys and Endpoints' section. You can use KEY1 or KEY2 | Azure API Key |
azureApiVersion | Azure API Version | Azure API version, follow the format YYYY-MM-DD, check the latest version | - |
Sources: locales/en-US/modelProvider.json:2-20
The settings page also includes functionality to create a new custom AI provider, with fields for ID, name, logo, and proxy URL.
Sources: locales/en-US/modelProvider.json:52-90
Integration Testing
LobeChat includes a dedicated page for testing AI model provider integrations. This tool allows developers to verify that API keys, endpoints, and model routing are functioning correctly.
Sources: [src/app/[variants]/(main)/ai-models/test/page.tsx]
Test Flow
The test process involves two main steps: fetching the list of available models from the provider and then sending a test chat message to a specified model.
The following sequence diagram illustrates the integration test flow initiated from the test page.
Test Form
The test page provides a simple form to input the necessary credentials for testing a provider.
| Field | Label | Description |
|---|---|---|
provider | AI Provider | A dropdown to select the provider to test (e.g., OpenAI, Anthropic, Google, Azure). |
apiKey | API Key | A password input for the provider's API key. |
model | Model (Optional) | An input to specify a model to use for the chat completion test (e.g., gpt-3.5-turbo). |
Sources: src/app/[variants]/(main)/ai-models/test/page.tsx:96-121
Usage in Application
The provider and model identifiers are used throughout the application to configure AI-powered features.
- Bundled Apps: Pre-configured applications specify a
providerandmodelin their configuration. For example, the "Code Reviewer" app uses theanthropicprovider and theclaude-3-5-sonnet-20241022model.
Sources: scripts/seedBundledApps/index.ts:43-51 - Agents: Each agent in LobeChat is associated with a specific
providerandmodel. This is evident in data structures where an agent's configuration includes these fields, such asprovider: "openai"andmodel: "gpt-4o-mini".
Sources: [packages/database/src/repositories/dataImporter/tests/fixtures/agentsToSessions.json]