Provider-Specific Implementations

Relevant source files

The following files were used as context for generating this wiki page:

• litellm/llms/anthropic/chat/transformation.py
• litellm/llms/vertex_ai/gemini/transformation.py
• litellm/llms/bedrock/chat/invoke_handler.py
• ARCHITECTURE.md
• AGENTS.md
• litellm/llms/chatgpt/common_utils.py

The Provider-Specific Implementations within LiteLLM serve as a modular translation layer that enables a unified interface for over 100 Large Language Model (LLM) providers. This system standardizes various input formats (such as OpenAI's chat completions) into the specific API requirements of providers like Anthropic, Google Vertex AI, and AWS Bedrock, while ensuring that the resulting outputs are returned in a consistent OpenAI-compatible format.

This architecture allows developers to switch between different models and infrastructure backends with minimal code changes. Each provider implementation is isolated within its own directory under litellm/llms/, containing logic for request transformation, response parsing, and error handling tailored to that specific vendor's requirements.

Sources: ARCHITECTURE.md, AGENTS.md

Architectural Overview

The core of the translation system relies on a standardized request/response flow. When a call is made to the LiteLLM SDK, it identifies the intended provider and routes the request through a specific Config class that inherits from a base implementation.

The diagram shows how LiteLLM standardizes different provider logics into a single request and response pipeline.
Sources: ARCHITECTURE.md

Key Components

Component	Location	Responsibility
BaseConfig	llms/base_llm/	The base class providing the template for all provider-specific configurations.
Transformation	llms/{provider}/chat/	Logic to convert OpenAI formatted parameters into provider-specific payloads.
InvokeHandler	llms/{provider}/chat/	Manages the actual network call and handles authentication protocols like AWS IAM or OAuth.
StreamingHandler	litellm_core_utils/	General logic for handling chunked responses, customized by provider-specific parsers.

Sources: ARCHITECTURE.md, AGENTS.md

Anthropic Implementation

The Anthropic implementation focuses on translating OpenAI's tool calling and message formats into the Anthropic Messages API. It handles complex logic for image processing and tool use blocks, which are distinct from OpenAI's implementation.

Request Transformation

The AnthropicConfig class handles the conversion of parameters like max_tokens, temperature, and stop_sequences. A significant part of this implementation involves flattening OpenAI's nested tool results and assistant tool calls into Anthropic's block-based format.

This sequence illustrates the transformation steps required to communicate with Anthropic's specific message-based architecture.
Sources: litellm/llms/anthropic/chat/transformation.py

Google Vertex AI (Gemini) Implementation

The Vertex AI implementation supports the Gemini family of models. It is responsible for mapping OpenAI parameters to the generateContent requirements of the Google Cloud platform.

Logic Details

Vertex AI implementation handles:

• Safety Settings: Mapping OpenAI-style content filtering to Google's safetySettings.
• Generation Config: Converting top_p, top_k, and max_output_tokens.
• Project Authentication: Utilizing environment-based credentials for Google Cloud projects.

Sources: litellm/llms/vertex_ai/gemini/transformation.py

AWS Bedrock Implementation

AWS Bedrock implementations use the BedrockLLM class and specific handlers like AWSEventStreamDecoder for streaming. Unlike direct HTTP providers, Bedrock often requires signing requests with AWS SigV4.

Invoke and Streaming Logic

The Bedrock implementation differentiates between standard invokes and event-stream based streaming. The decoder is responsible for identifying exception types within the AWS binary stream format and mapping them to LiteLLM's standard exceptions.

The flow demonstrates the decoding logic for AWS Bedrock's event stream, identifying whether a packet is a data chunk or a service exception.
Sources: litellm/llms/bedrock/chat/invoke_handler.py:126-170

ChatGPT/Codex CLI Implementation

A specialized implementation exists for the ChatGPT/Codex CLI, which uses OAuth-based device flows. This module provides constants for device code URLs and verification endpoints, as well as specific system instructions for the coding agent.

Authentication Helpers

The implementation includes functions like get_chatgpt_user_agent() and get_chatgpt_default_headers() to ensure requests are correctly identified by the ChatGPT backend.

Sources: litellm/llms/chatgpt/common_utils.py:12-25

Default Instructions

The module defines CHATGPT_DEFAULT_INSTRUCTIONS, a comprehensive system prompt that guides the model on:

• File editing constraints (ASCII preferred, code comments usage).
• Planning tool usage (skipping for easy tasks).
• Terminal command execution (using date or rg).

Sources: litellm/llms/chatgpt/common_utils.py:28-100

Conclusion

The provider-specific implementations in LiteLLM abstract the complexity of interacting with diverse LLM ecosystems. By standardizing transformations into isolated modules, the system maintains a consistent developer experience while supporting specialized vendor features like Anthropic's tool blocks, Bedrock's event streams, and Google's safety configurations. This modularity is essential for the LiteLLM SDK's role as a unified gateway.

Sources: ARCHITECTURE.md, AGENTS.md