Guardrails and Content Filtering
Relevant source files
The following files were used as context for generating this wiki page:
- litellm/proxy/guardrails/guardrail_endpoints.py
- policy_templates.json
- cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py
- ui/litellm-dashboard/src/components/guardrails/content_filter/ContentFilterManager.test.tsx
- ui/litellm-dashboard/src/components/view_logs/GuardrailViewer/BedrockGuardrailDetails.tsx
- ui/litellm-dashboard/src/components/guardrails/guardrail_info.test.tsx
Guardrails and content filtering in LiteLLM provide a comprehensive safety layer designed to monitor, mask, or block sensitive and inappropriate data in both LLM inputs and outputs. This system allows developers to enforce compliance with regulatory requirements (like GDPR or PDPA), protect Personally Identifiable Information (PII), and filter out harmful or off-topic content before it reaches the model or the end user.
The architecture supports multiple guardrail types, including built-in content filters, Presidio-based PII detection, and integration with external providers like AWS Bedrock Guardrails. These are implemented as hooks that can be configured to run in pre_call (before the LLM request) or post_call (after the LLM response) modes.
Architecture and Execution Flow
The LiteLLM Proxy integrates guardrails into its request processing pipeline. When a request is received, the proxy evaluates the configured guardrail policies and applies the necessary actions—BLOCK, MASK, or ANONYMIZE—based on the detection results.
Guardrail Hook Lifecycle
The following diagram illustrates how guardrails are executed during the lifecycle of an LLM call within the LiteLLM Proxy.
The system differentiates between input validation (Pre-Call) and output validation (Post-Call). Violations in Pre-Call mode prevent the request from ever reaching the LLM, saving costs and preventing potential jailbreaks. Sources: litellm/proxy/guardrails/guardrail_endpoints.py, policy_templates.json
Guardrail Types and Configurations
LiteLLM supports several guardrail implementations, each defined by a specific set of parameters and logic.
1. LiteLLM Content Filter
A built-in filter that uses prebuilt patterns, regex, or keyword matching to detect specific entities. It is highly configurable via the dashboard or configuration files.
| Configuration Field | Type | Description |
|---|---|---|
guardrail | string | Set to litellm_content_filter. |
patterns | array | List of prebuilt or regex patterns to detect. |
blocked_words | array | List of specific keywords to trigger a BLOCK action. |
action | string | The action to take on match (BLOCK, MASK). |
severity_threshold | string | Sensitivity level (e.g., high, medium, low). |
Sources: ui/litellm-dashboard/src/components/guardrails/content_filter/ContentFilterManager.test.tsx:10-25, policy_templates.json:20-50
2. Bedrock Guardrails
Integration with AWS Bedrock's guardrail API. This allows LiteLLM to leverage AWS's native topic policies, content policies, and sensitive information policies.
The diagram shows the structure of data returned from a Bedrock-compatible guardrail server, providing granular details on which policy was triggered. Sources: cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py:20-100, ui/litellm-dashboard/src/components/view_logs/GuardrailViewer/BedrockGuardrailDetails.tsx
Policy Templates
LiteLLM provides extensive pre-configured policy templates for common regulatory and safety scenarios. These templates bundle multiple guardrails into a single "Policy."
Available Policy Categories
- PII Protection: Specific versions for Australia (AU), EU (GDPR Art. 32), and Singapore (PDPA).
- Regulatory Compliance: Includes UAE Federal Decree-Laws and Singapore MAS AI Risk Management.
- Content Safety: NSFW filtering across multiple languages (English, French, German, Spanish).
- Security: Prompt injection protection, SQL injection blocking, and MCP (Model Context Protocol) server validation.
- Industry-Specific: Aviation operations security and healthcare claims agent safety.
Example: Advanced PII Protection (Australia)
Derived from policy_templates.json, this policy includes the following guardrail definitions:
au-pii-tax-identifiers: Masks TFN, ABN, and Medicare numbers.au-pii-passports: Masks Australian passport numbers.credentials-api-keys: Blocks AWS access keys, GitHub tokens, and Slack tokens.network-infrastructure-pii: Masks IPv4 and IPv6 addresses.
Sources: policy_templates.json:2-350
Guardrail API Endpoints
The system exposes management endpoints to interact with guardrail configurations dynamically.
| Endpoint | Method | Description |
|---|---|---|
/beta/litellm_basic_guardrail_api | POST | Basic API for applying guardrails to a list of texts/images. |
/config | GET | Retrieves the current guardrail configuration. |
/config/update | POST | Updates the active guardrail settings. |
Sources: cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py:280-330, litellm/proxy/guardrails/guardrail_endpoints.py
Request Structure for LiteLLM Basic Guardrail API
{
"texts": ["User input text here"],
"input_type": "request",
"additional_provider_specific_params": {},
"litellm_call_id": "optional-id"
}
Sources: cookbook/mock_guardrail_server/mock_bedrock_guardrail_server.py:285-298
Management and UI Integration
The LiteLLM Dashboard provides a ContentFilterManager to visualize and edit guardrail settings. It allows administrators to:
- Add or remove prebuilt PII patterns.
- Define custom regex patterns.
- Manage blocked keyword lists.
- Toggle between
MASKandBLOCKactions.
Changes made in the UI are formatted into a standardized API payload before being sent to the proxy server for persistence. Sources: ui/litellm-dashboard/src/components/guardrails/content_filter/ContentFilterManager.test.tsx, ui/litellm-dashboard/src/components/guardrails/guardrail_info.test.tsx
Conclusion
The Guardrails and Content Filtering system in LiteLLM acts as a robust intermediary layer that ensures LLM interactions remain safe and compliant. By combining prebuilt policy templates with flexible regex and keyword matching, it addresses a wide range of concerns from simple profanity filtering to complex multi-national data protection regulations. The integration with external providers like AWS Bedrock further extends these capabilities for enterprise environments.