Secret Management
Relevant source files
The following files were used as context for generating this wiki page:
Secret Management in LiteLLM provides a unified interface for securely storing, retrieving, and managing sensitive credentials such as API keys and database URLs. It supports integration with external enterprise-grade providers, including AWS Secrets Manager, HashiCorp Vault, and CyberArk.
The system is designed to abstract the complexities of different cloud and on-premise secret storage solutions, providing a consistent API for both synchronous and asynchronous operations. This ensures that sensitive information is never hardcoded or exposed in configuration files, facilitating secure deployments across various environments.
Sources: litellm/secret_managers/main.py, cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md:144-148
Architecture and Components
The architecture follows a provider-based pattern where a central management utility delegates calls to specific secret manager implementations.
Component Overview
- Secret Manager Client: A global client instance (
litellm.secret_manager_client) that handles secret operations. - BaseSecretManager: An abstract base class defining the required interface for all providers.
- AWSSecretsManagerV2: Implementation for AWS-native secret storage using
boto3andaiobotocore. - HashiCorpVault: Implementation for HashiCorp Vault, supporting KV (Key-Value) version 2.
System Flow
The following diagram illustrates the flow of secret retrieval through the global client.
The application interacts with the global secret_manager_client, which routes the request to the configured backend.
Sources: litellm/secret_managers/main.py, litellm/secret_managers/aws_secret_manager_v2.py
Supported Providers
LiteLLM supports several major secret management platforms. Configuration is typically handled via KeyManagementSettings or environment variables.
| Provider | Implementation Class | Primary Environment Variables |
|---|---|---|
| AWS Secrets Manager | AWSSecretsManagerV2 | AWS_REGION_NAME, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY |
| HashiCorp Vault | HashiCorpVault | VAULT_ADDR, VAULT_TOKEN |
| Google Secret Manager | Referenced in docs | GCP_PROJECT_ID |
Sources: litellm/secret_managers/aws_secret_manager_v2.py:101-105, litellm/secret_managers/hashicorp_secret_manager.py:27-30, tests/litellm_utils_tests/test_aws_secret_manager.py:24-28
AWS Secrets Manager Integration
The AWSSecretsManagerV2 implementation provides advanced support for AWS features, including IAM role assumption and cross-account access.
Authentication Methods
AWS integration supports multiple authentication schemes:
- Environment Credentials: Standard AWS credentials set in the environment.
- IAM Role Assumption: Using
aws_role_nameto assume a specific role for operations. - IAM Roles for Service Accounts (IRSA): Specifically for EKS environments using
aws_web_identity_token. - AWS Profiles: Specifying a named profile from the local AWS config.
Primary Secret Functionality
A "Primary Secret" allows LiteLLM to treat a single JSON-formatted AWS secret as a container for multiple individual keys. When a primary_secret_name is provided during a read operation, the system fetches the JSON object and returns the value associated with the specified key.
HashiCorp Vault Integration
The HashiCorpVault implementation interacts with the Vault API to manage secrets stored in KV v2 engines.
Operations
The implementation supports core CRUD operations:
- Read: Fetches the latest secret value from a specific path.
- Write: Stores a secret value at a path.
- Delete: Soft-deletes a secret at a path.
Configuration
Vault requires a base URL (vault_addr) and an authentication token (vault_token). These can be passed during initialization or read from the environment.
Sources: litellm/secret_managers/hashicorp_secret_manager.py:10-60
Global Initialization
The system is typically initialized via a static method that configures the global client based on provided settings.
Key Management Settings
The KeyManagementSettings object allows for granular control over the secret manager configuration, including region names, role ARNs, and custom STS endpoints for AWS.
Initialization Flow
Conclusion
Secret Management in LiteLLM provides an essential security layer by decoupling sensitive configuration from application logic. By supporting enterprise backends like AWS and HashiCorp Vault, it enables secure key rotation, centralized auditing, and fine-grained access control through IAM roles and Vault policies. This modular approach allows LiteLLM to scale from simple local development to complex multi-cloud enterprise deployments.