Database Integrations
Relevant source files
The following files were used as context for generating this wiki page:
Logfire provides specialized database integrations designed to offer deep visibility into the data layer of Python applications. These integrations are built upon OpenTelemetry standards, allowing Logfire to capture database queries, execution times, and connection telemetry as rich spans and metrics. By profiling database queries, Logfire enables developers to identify performance bottlenecks and understand the data flow between application logic and various storage backends.
The scope of database support covers popular SQL and NoSQL technologies, including SQLAlchemy, Psycopg, Asyncpg, MySQL, SQLite3, Redis, and PyMongo. These integrations are managed as optional dependencies within the Logfire SDK, ensuring a modular footprint that only includes the necessary instrumentation for a project's specific stack.
Sources: README.md:16-24, pyproject.toml:70-80, mkdocs.yml:143-151
Integration Architecture
Database integrations in Logfire function as an opinionated wrapper around OpenTelemetry (OTel) instrumentation. When a user calls a specific instrumentation function (e.g., instrument_sqlalchemy()), Logfire configures the underlying OTel instrumentation to capture spans. These spans are then processed through Logfire's internal pipeline, which includes scrubbing sensitive data and adding Python-centric attributes.
Data Flow and Span Processing
The following diagram illustrates how database calls are captured and transformed into Logfire spans:
The flow ensures that raw SQL queries or database commands are captured by OpenTelemetry, sanitized by Logfire's scrubber, and sent to the platform as structured telemetry.
Sources: README.md:25-27, logfire/_internal/config.py:640-660, logfire/_internal/config.py:720-740
Supported Database Clients and ORMs
Logfire supports a wide range of database technologies through specific optional dependencies. These are defined in the project's configuration to allow for targeted installation.
| Integration | Optional Dependency Group | Backend Type |
|---|---|---|
| SQLAlchemy | sqlalchemy | SQL ORM |
| Psycopg (v2/v3) | psycopg, psycopg2 | PostgreSQL |
| Asyncpg | asyncpg | PostgreSQL (Async) |
| MySQL | mysql | MySQL |
| SQLite3 | sqlite3 | SQLite |
| Redis | redis | Key-Value / NoSQL |
| PyMongo | pymongo | MongoDB / NoSQL |
Sources: pyproject.toml:70-80, mkdocs.yml:143-151
Key Implementation Features
SQL Object Representation
Logfire provides rich display and serialization for database objects, particularly for ORMs like SQLAlchemy. When logging objects retrieved from the database, Logfire captures the fields while handling lazy-loading relationships by marking them as <deferred> to avoid unintended database hits during the telemetry capture process.
# Example of how SAModel (SQLAlchemy) is represented in telemetry
# SAModel(id=1, name='test name', models2=[...])
attributes = {
'var': '{"models2":"<deferred>","id":1,"name":"test name"}',
'logfire.json_schema': '{"properties":{"var":{"title":"SAModel","x-python-datatype":"sqlalchemy"}}}'
}
Sources: tests/test_json_args.py:530-555, README.md:18-20
Configuration and Global Logic
Database integrations are typically activated during the global configuration phase or through explicit instrumentation calls. The logfire.configure() function sets up the TracerProvider and MeterProvider, which are required for database instrumentation to export data correctly.
| Parameter | Type | Description |
|---|---|---|
scrubbing | ScrubbingOptions | Controls redaction of sensitive data in database queries. |
service_name | str | Identifies the service performing the database operations. |
sampling | SamplingOptions | Determines the percentage of database spans to record. |
Sources: logfire/_internal/config.py:410-450, logfire/_internal/config.py:600-610
Advanced Telemetry Capture
Logfire extends standard OTel database spans with additional metadata to improve debugging.
Relationship Mapping
For ORM integrations like SQLAlchemy and SQLModel, Logfire identifies the specific Python class associated with the telemetry. This is reflected in the logfire.json_schema attribute, which identifies the x-python-datatype as sqlalchemy or PydanticModel.
The diagram shows the relationship between a generic database span and the specific Python metadata attached to ORM-based queries.
Sources: tests/test_json_args.py:545-560, logfire/_internal/config.py:615-625
Connection Lifecycle Metrics
Logfire captures metrics related to database connection pools and active requests. This is primarily configured through the MetricsOptions class, which handles instrument types like UpDownCounter for http.server.active_requests, which can correlate with database-heavy web operations.
Sources: logfire/_internal/config.py:255-275, logfire/_internal/config.py:760-770
Conclusion
Database integrations in Logfire serve as a bridge between low-level database drivers and high-level Python application observability. By combining OpenTelemetry instrumentation with Logfire's specialized handling of Python objects (like SQLAlchemy models) and robust scrubbing for SQL statements, the system provides a detailed and safe view into data layer performance and behavior. This integration is essential for diagnosing slow queries and understanding the impact of database interactions on overall application latency.