Kubernetes and Helm Deployment
Relevant source files
The following files were used as context for generating this wiki page:
LiteLLM provides a community-maintained Helm chart for deploying the LiteLLM Proxy into Kubernetes clusters. This deployment method is recommended for production environments, offering scalability, integrated secret management, and optional persistence through database sidecars.
The Helm chart automates the creation of essential Kubernetes primitives, including Deployments, Services, ConfigMaps, and Secrets, while supporting advanced features like horizontal pod autoscaling (HPA) and KEDA-based scaling. Sources: deploy/charts/litellm-helm/README.md:3-5, deploy/charts/litellm-helm/values.yaml:148-185
Architecture and Components
The LiteLLM deployment architecture consists of a primary proxy container supported by configuration maps for model definitions and secrets for sensitive credentials.
The diagram shows the relationship between the LiteLLM Proxy container and its Kubernetes-native configuration sources. Sources: deploy/charts/litellm-helm/templates/deployment.yaml:49-160, deploy/charts/litellm-helm/README.md:104-120
Core Components
- LiteLLM Proxy Container: The main application logic running the proxy server. Sources: deploy/charts/litellm-helm/templates/deployment.yaml:49-55
- Init Containers: Used for database migrations (via Prisma) and setting up necessary dependencies before the main app starts. Sources: deploy/charts/litellm-helm/templates/deployment.yaml:45-48
- ConfigMap: Houses the
config.yamlwhich defines themodel_listandgeneral_settings. Sources: deploy/charts/litellm-helm/values.yaml:120-132 - Secrets: Manage sensitive information such as the
PROXY_MASTER_KEYand database connection strings. Sources: deploy/charts/litellm-helm/values.yaml:104-118
Configuration Management
Deployment configuration is primarily managed through values.yaml, which allows for detailed tuning of the proxy environment.
Proxy Deployment Settings
| Parameter | Description | Default |
|---|---|---|
replicaCount | Number of proxy pods to run | 1 |
image.repository | Docker image to use (e.g., ghcr.io/berriai/litellm-database) | ghcr.io/berriai/litellm-database |
service.port | TCP port for the proxy service | 4000 |
proxyConfigMap.create | Whether to generate a ConfigMap from proxy_config | true |
masterkeySecretName | Name of existing secret for master key | "" |
Sources: deploy/charts/litellm-helm/values.yaml:5-132, deploy/charts/litellm-helm/README.md:15-46
Environment Variables and Secrets
The chart allows injecting environment variables through three mechanisms:
- Direct
envVars: Defined as a map invalues.yaml. Sources: deploy/charts/litellm-helm/values.yaml:263 environmentSecrets: A list of existing Kubernetes secrets to be exported to the pod. Sources: deploy/charts/litellm-helm/values.yaml:70-74environmentConfigMaps: A list of existing ConfigMaps to be exported. Sources: deploy/charts/litellm-helm/values.yaml:77-81
Database and Persistence
LiteLLM requires a database to store budgets, virtual keys, and spend data. The Helm chart supports two primary modes for database integration.
The flow shows how LiteLLM connects to its data persistence layer depending on the configuration. Sources: deploy/charts/litellm-helm/values.yaml:209-247, deploy/charts/litellm-helm/templates/deployment.yaml:60-90
Database Deployment Options
- Standalone PostgreSQL: The chart can deploy a single-node PostgreSQL instance using the Bitnami subchart. This is enabled via
db.deployStandalone. Sources: deploy/charts/litellm-helm/README.md:94-96 - Existing Instance: Users can point to an existing external database by setting
db.useExisting: trueand providing the endpoint details. Sources: deploy/charts/litellm-helm/README.md:82-90 - Migration Job: A Kubernetes Job is used to run Prisma schema migrations. It can be controlled via
migrationJobsettings and supports ArgoCD/Helm hooks. Sources: deploy/charts/litellm-helm/values.yaml:251-260, deploy/charts/litellm-helm/README.md:104-120
Health and Monitoring
Standard Kubernetes probes are configured to ensure high availability and automatic recovery.
Probes
The deployment template defines three probe types:
- Liveness: Checks if the container is still running (
/health/liveliness). - Readiness: Checks if the container is ready to serve traffic (
/health/readiness). - Startup: Used for slow-starting applications to avoid premature liveness failures.
Sources: deploy/charts/litellm-helm/templates/deployment.yaml:126-154, deploy/charts/litellm-helm/values.yaml:94-118
Prometheus Integration
If serviceMonitor.enabled is set to true, the chart creates a ServiceMonitor resource for the Prometheus Operator. This resource targets the /metrics/ path on the proxy service port. Sources: deploy/charts/litellm-helm/templates/tests/test-servicemonitor.yaml:27-32, deploy/charts/litellm-helm/values.yaml:313-316
Summary
Deploying LiteLLM with Helm provides a robust production-ready foundation. By managing configuration through ConfigMaps and sensitive data through Secrets, the chart ensures compliance with Kubernetes best practices. Key capabilities like integrated database migrations via Prisma Jobs and optional Prometheus monitoring make it suitable for enterprise-grade LLM gateway requirements. Significance within the project includes providing a standard, repeatable deployment path for DevOps teams. Sources: deploy/charts/litellm-helm/README.md:3-8, deploy/charts/litellm-helm/templates/deployment.yaml:1-5