Proxy API Endpoints
Relevant source files
The following files were used as context for generating this wiki page:
LiteLLM provides a Proxy Server (LLM Gateway) that acts as a unified interface for over 100 LLM providers. It translates standard OpenAI-format inputs into provider-specific completion, embedding, and image generation endpoints while adding critical enterprise features such as authentication, rate limiting, budget management, and observability. Sources: AGENTS.md:1-12, ARCHITECTURE.md:1-10
The Proxy API is organized into several functional categories: core LLM endpoints for chat and completions, management endpoints for administrative tasks (keys, teams, users), and passthrough endpoints for direct provider API access. This architecture ensures that the proxy can handle high-throughput scenarios while providing robust control over LLM usage across an organization. Sources: ARCHITECTURE.md:100-120, GEMINI.md:40-50
LLM Interaction Endpoints
These endpoints form the primary interface for users and applications to interact with language models. They generally follow the OpenAI API specification to ensure compatibility with existing SDKs. Sources: ARCHITECTURE.md:10-20, AGENTS.md:80-90
Chat and Completion Requests
The core request flow for chat completions involves several stages: authentication via API keys or JWT, budget/rate limit checks, routing via a Load Balancer, and finally, the SDK call to the actual LLM provider. Sources: ARCHITECTURE.md:15-35
The sequence above illustrates how an incoming request is validated and transformed before reaching the provider. Sources: ARCHITECTURE.md:15-45
Core LLM Endpoint Categories
| Endpoint Path | Purpose | Documentation Reference |
|---|---|---|
/v1/chat/completions | Standard chat interface (GPT-style) | ARCHITECTURE.md:110 |
/v1/messages | Anthropic-style messages API | ARCHITECTURE.md:111 |
/v1/embeddings | Vector embedding generation | ARCHITECTURE.md:118 |
/v1/images/generations | Image generation requests | ARCHITECTURE.md:114 |
/v1/rerank | Reranking documents for search | ARCHITECTURE.md:118 |
Management Endpoints
Management endpoints allow administrators to control access and track costs within the LiteLLM ecosystem. These are typically restricted to master-key holders or specific admin roles. Sources: litellm/proxy/management_endpoints/key_management_endpoints.py, ARCHITECTURE.md:120-130
Key and Team Management
Admin APIs facilitate the creation and rotation of virtual keys, team assignments, and the definition of organizational hierarchies. Sources: litellm/proxy/management_endpoints/key_management_endpoints.py, ARCHITECTURE.md:105-110
- Virtual Keys: Managed in
key_management_endpoints.py, allowing per-key budgets. - Team Management: Defined in
LiteLLM_TeamTablewithin the PostgreSQL schema. - SSO/Auth: Support for JWT, OAuth2, and Proxy CLI authentication.
Sources: ARCHITECTURE.md:165-175, AGENTS.md:85-90
Passthrough and Specialized Endpoints
For features not yet standardized in the OpenAI format or for direct provider access, the proxy includes passthrough capabilities. Sources: litellm/proxy/pass_through_endpoints/pass_through_endpoints.py, ARCHITECTURE.md:110-125
Provider-Specific Passthrough
The /vertex-ai/* and /gemini/* paths allow developers to use LiteLLM's authentication and logging while accessing unique features of Vertex AI or Google AI Studio. Sources: ARCHITECTURE.md:112-113
LiteLLM Skills
A specialized module for database-backed skills storage. It allows users to upload "skills" (as ZIP files) that are then converted into tools and injected into the system prompt for any LLM provider, including those that do not natively support certain tool formats. Sources: litellm/llms/litellm_proxy/skills/README.md:1-20
The skills system provides a way to standardize agentic capabilities across heterogeneous model deployments. Sources: litellm/llms/litellm_proxy/skills/README.md:10-30
Infrastructure and Persistence
The Proxy API relies on external infrastructure to maintain state and performance: Sources: ARCHITECTURE.md:150-165
- Redis Cache: Used for high-speed rate limit counters, TPM/RPM tracking, and API key caching.
- PostgreSQL (Prisma): Stores persistent data including
LiteLLM_VerificationToken(keys),LiteLLM_SpendLogs(billing), andLiteLLM_UserTable. - InternalUsageCache: A dual-layer cache (in-memory + Redis) for proxy-level rate limiting.
Sources: ARCHITECTURE.md:155-180
Conclusion
Proxy API endpoints in LiteLLM serve as the bridge between standardized client requests and a diverse array of LLM backends. By centralizing authentication, routing, and management into a unified gateway, it enables enterprise-grade control over AI usage. Whether through standard completion endpoints, administrative management APIs, or specialized skill injection, the proxy ensures that LLM interactions are secure, tracked, and optimized. Sources: ARCHITECTURE.md:5-15, AGENTS.md:80-100