LLM Backends & Integration

Relevant source files

The following files were used as context for generating this wiki page:

• vibe/core/llm/backend/factory.py
• vibe/core/llm/backend/mistral.py
• vibe/core/config.py
• CLAUDE.md
• tests/backend/test_backend.py

The LLM Backends & Integration system provides a unified abstraction layer for interacting with various Large Language Model (LLM) providers. It enables the QQcode agent to communicate seamlessly with different APIs—such as Mistral AI, Anthropic, and OpenAI-compatible providers—while maintaining a consistent interface for request handling, response streaming, and tool execution.

This system is built on a factory pattern that instantiates specific backend drivers based on configuration. These backends handle the complexities of vendor-specific SDKs, authentication, and data mapping, ensuring that the core agent logic remains decoupled from the underlying API implementations.
Sources: CLAUDE.md, vibe/core/llm/backend/factory.py

Backend Architecture

The architecture relies on a clear separation between configuration, provider-specific logic, and the unified interface used by the agent.

Core Components

Component	Description
Backend Enum	Defines supported backend types: MISTRAL, GENERIC, and ANTHROPIC.
BACKEND_FACTORY	A mapping that associates Backend enum values with their respective implementation classes.
ProviderConfig	A Pydantic model storing API base URLs, keys, and specific backend requirements.
MistralBackend	A specialized backend using the mistralai SDK for native Mistral AI integration.
GenericBackend	A flexible backend designed for OpenAI-compatible APIs (OpenRouter, Llamacpp, etc.).

Sources: vibe/core/config.py:143-146, vibe/core/llm/backend/factory.py:10-15

Backend Initialization Flow

The following diagram illustrates how the system selects and initializes a backend implementation based on the active model configuration.

The system resolves the provider from the active_model and uses the BACKEND_FACTORY to instantiate the correct class.
Sources: vibe/core/config.py:440-454, vibe/core/llm/backend/factory.py:10-15

Data Mapping & Transformation

Each backend utilizes a "Mapper" class to translate internal QQcode data structures into the format expected by the provider's API.

Mistral Mapping Example

The MistralMapper class handles the conversion of internal LLMMessage and AvailableTool objects into mistralai SDK types.

• Messages: Maps roles (system, user, assistant, tool) and handles tool call IDs and content.
• Tools: Converts tool definitions into mistralai.Tool and mistralai.Function objects.
• Usage: Parses token counts from the SDK response into a unified LLMUsage structure.

Sources: vibe/core/llm/backend/mistral.py:27-105

Request/Response Sequence

Supported Providers & Configurations

The system supports a wide range of providers out-of-the-box, configured via vibe/core/config.py.

Default Provider Configurations

Provider	API Base URL	Default Backend	API Style
Mistral	https://api.mistral.ai/v1	MISTRAL	OpenAI-like
Anthropic	https://api.anthropic.com	ANTHROPIC	Anthropic
OpenRouter	https://openrouter.ai/api/v1	GENERIC	OpenAI
Llamacpp	http://127.0.0.1:8080/v1	GENERIC	OpenAI
Qwen	https://dashscope.aliyuncs.com/...	GENERIC	OpenAI

Sources: vibe/core/config.py:189-230

Compatibility Validation

The configuration system performs validation to ensure the selected backend matches the API endpoint. For example, using Backend.MISTRAL with a non-Mistral API base URL will trigger a WrongBackendError.
Sources: vibe/core/config.py:91-98, vibe/core/config.py:539-567

Streaming & Tool Execution

Backends support both standard completions and asynchronous streaming. Streaming is critical for the CLI/TUI to provide real-time feedback.

Streaming Implementation

The complete_streaming method returns an AsyncGenerator[LLMChunk, None]. Each chunk may contain:

• Partial message content.
• Incremental tool call information (name and arguments).
• Token usage (usually in the final chunk).
• Finish reasons (e.g., stop, tool_calls).

Sources: vibe/core/llm/backend/mistral.py:204-262, tests/backend/test_backend.py:129-181

Tool Call Handling

Tool calls are structured as FunctionCall objects within a ToolCall list. Backends must handle the serialization of tool arguments, which may arrive as JSON strings or dictionary objects depending on the provider.
Sources: vibe/core/llm/backend/mistral.py:93-105, vibe/core/llm/backend/mistral.py:175-181

Conclusion

The LLM Backend system provides a robust and extensible foundation for QQcode's AI capabilities. By abstracting provider-specific logic into dedicated backend classes and using a centralized factory, the project can support diverse models while maintaining strict type safety and a consistent developer experience across the codebase.
Sources: CLAUDE.md, vibe/core/llm/backend/factory.py