Observability and Logging Integrations
Relevant source files
The following files were used as context for generating this wiki page:
Observability and Logging Integrations in LiteLLM provide a standardized framework for tracking LLM interactions, costs, performance metrics, and traces across various third-party providers. This system acts as a middleware layer that intercepts requests and responses to provide unified logging regardless of the underlying LLM provider.
The architecture relies on a callback-based system where CustomLogger instances handle events such as successful API calls, failures, and budget tracking. This allows for seamless integration with platforms like DataDog, Langfuse, and Prometheus, while also supporting custom internal spend tracking and cost management.
Sources: litellm/integrations/custom_logger.py, AGENTS.md:21-27, litellm/integrations/websearch_interception/ARCHITECTURE.md:120-130
Core Logging Architecture
The logging system is built around the CustomLogger class, which defines the interface for all observability integrations. These loggers are triggered at various stages of the LLM request lifecycle.
The Logger Lifecycle
The following diagram illustrates how the CustomLogger interacts with the request flow:
The system supports both synchronous and asynchronous logging methods to ensure that observability does not block the primary request path.
Sources: litellm/integrations/custom_logger.py:34-150
Key Integration Classes
| Class | Description | Primary File |
|---|---|---|
CustomLogger | Base class defining the interface for all logging callbacks. | litellm/integrations/custom_logger.py |
DataDogLogger | Specialized logger for sending metrics and traces to DataDog. | litellm/integrations/datadog/datadog.py |
LangfuseLogger | Integration for Langfuse for tracing and prompt management. | litellm/integrations/langfuse/langfuse.py |
ProxyTrackCostCallback | Internal callback for calculating and storing spend in the Proxy DB. | litellm/proxy/hooks/proxy_track_cost_callback.py |
Supported Integrations
LiteLLM supports a wide array of observability providers. These are categorized into Logging, Guardrails, and Spend Tracking.
Observability Providers
- DataDog: Used for infrastructure-level monitoring and custom LLM metrics.
- Langfuse: Provides deep tracing of LLM chains and prompt versioning.
- Prometheus: Standard for monitoring RPS, latency, and error rates.
- PostHog: Integrated for observability and user behavior tracking.
- Arize Phoenix: Specialized in LLM observability and evaluation.
Sources: cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md:104-125
DataDog Integration Logic
The DataDogLogger translates LiteLLM metadata into DataDog tags and metrics. It tracks specific fields like model, api_base, and user to allow for granular filtering in DataDog dashboards.
Spend Tracking and Cost Management
A critical component of LiteLLM's observability is the ability to track costs in real-time. This is handled by the ProxyTrackCostCallback.
Cost Tracking Workflow
- Response Interception: After a successful LLM call, the callback receives the
ModelResponse. - Cost Calculation: The system uses
completion_costto calculate the price based on tokens used and model-specific pricing. - Metadata Association: Costs are attributed to specific
api_key,user_id, orteam_id. - Database Persistence: The calculated cost is pushed to the Proxy database for budget enforcement.
Sources: litellm/proxy/hooks/proxy_track_cost_callback.py:15-60, cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md:155-165
Agentic Loop Observability
LiteLLM includes advanced interception for "Agentic Loops," such as WebSearch tool execution. This requires specialized logging to track the internal search calls made on behalf of the user.
WebSearch Interception Diagram
This diagram shows how search execution is logged and managed internally:
The WebSearchInterceptionLogger acts as a specialized CustomLogger that implements async_should_run_agentic_loop() and async_run_agentic_loop().
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:144-180
Configuration and Implementation
Developers can implement custom observability by inheriting from CustomLogger.
Custom Logger Interface
class CustomLogger:
def log_pre_call_hook(self, model, messages, kwargs, model_response):
pass
def log_success_event(self, kwargs, response_obj, start_time, end_time):
pass
def log_failure_event(self, kwargs, response_obj, start_time, end_time):
pass
Sources: litellm/integrations/custom_logger.py:48-93
Logging Configuration Parameters
| Parameter | Type | Description |
|---|---|---|
litellm.success_callback | List | List of logger instances to trigger on success. |
litellm.failure_callback | List | List of logger instances to trigger on failure. |
litellm.callbacks | List | Universal list for both success and failure logging. |
litellm.set_verbose | Boolean | Enables detailed debug logging for development. |
Sources: tests/llm_translation/test_bedrock_completion.py:46-52, litellm/integrations/websearch_interception/ARCHITECTURE.md:200-210
Summary
Observability and Logging Integrations form a robust backbone for the LiteLLM Proxy, ensuring that every LLM interaction is measurable and accountable. By utilizing the CustomLogger framework, LiteLLM allows for extensive customization while providing production-ready integrations for industry-standard tools like DataDog and Langfuse. This system is essential for production LLM deployments that require strict budget management and performance monitoring.