Query Client & DB API
Relevant source files
The following files were used as context for generating this wiki page:
The Query Client and DB API components in Pydantic Logfire provide a structured interface for programmatically querying telemetry data using standard SQL. This system allows users to export and analyze data stored in the Logfire platform through various formats including JSON, CSV, and Apache Arrow. By leveraging standard SQL, the system enables integration with existing BI tools and data processing libraries such as Polars.
The Query Client is primarily implemented as an experimental feature, offering both synchronous and asynchronous implementations. It handles authentication via read tokens, manages HTTP communication with the Logfire API, and provides typed results for data manipulation.
Sources: README.md:16-18, logfire/experimental/query_client.py:1-25
Architecture and Core Components
The query system is built on a generic base class that abstracts the common logic for request construction and response handling, allowing for specialized synchronous and asynchronous client implementations.
Class Hierarchy
The architecture utilizes a Generic base class constrained by the httpx.BaseClient type to ensure consistent behavior across sync and async variants.
The diagram shows the relationship between the internal base client and the public-facing sync and async query clients.
Sources: logfire/experimental/query_client.py:53-233
Key Data Structures
The Query Client defines several TypedDict structures to ensure type safety for JSON-format query results.
| Structure | Purpose | Fields |
|---|---|---|
ReadTokenInfo | Metadata about the authenticated token. | organization_name, project_name |
ColumnDetails | Metadata for a single result column. | name, datatype, nullable |
ColumnData | Column-oriented data including values. | name, datatype, nullable, values |
QueryResults | Standard column-oriented results. | columns (List of ColumnData) |
RowQueryResults | Row-oriented results for standard JSON processing. | columns (List of ColumnDetails), rows (List of dicts) |
Sources: logfire/experimental/query_client.py:35-50
Query Execution Flow
Executing a query involves constructing SQL parameters, setting appropriate Accept headers for the desired format, and handling API-specific error codes.
Request Parameters
The client automatically constructs query parameters to filter data and control the output format.
This flowchart illustrates the internal process of converting a Python method call into an API request.
Sources: logfire/experimental/query_client.py:65-79
Error Handling
The system defines specific exceptions for common failure modes in the query lifecycle:
QueryExecutionError: Raised for 400 Bad Request status codes, typically indicating SQL syntax errors or invalid logic on the server.QueryRequestError: Raised for 422 Unprocessable Entity codes, indicating invalid request parameters.InfoRequestError: Raised when the metadata endpoint/v1/read-token-inforeturns an incomplete response.
Sources: logfire/experimental/query_client.py:19-32, logfire/experimental/query_client.py:81-86
Supported Data Formats
The Query Client supports four primary data exchange formats, each optimized for different consumption patterns.
Format Comparison
| Method | Accept Header | Return Type | Use Case |
|---|---|---|---|
query_json | application/json | QueryResults | Column-oriented data processing in Python. |
query_json_rows | application/json | RowQueryResults | Traditional row-by-row data handling. |
query_arrow | application/vnd.apache.arrow.stream | pyarrow.Table | High-performance data analysis (Polars integration). |
query_csv | text/csv | str | Exporting to spreadsheets or text processing. |
Sources: logfire/experimental/query_client.py:108-160, tests/cassettes/test_query_client/test_read_sync.yaml
Performance Considerations
- Timeouts: The Query Client uses a default timeout of 30.0 seconds (
DEFAULT_TIMEOUT), as SQL queries against large datasets typically take longer than standard API requests. - Compression: Evidence from API recordings shows that the Logfire backend uses
gzipcontent encoding for query results to reduce bandwidth for large exports.
Sources: logfire/experimental/query_client.py:16, tests/cassettes/test_query_client/test_read_sync.yaml:73
Internal Infrastructure and Configuration
The Query Client relies on internal utilities for authentication and URL resolution.
URL and Token Resolution
The Query Client infers the Logfire API base URL from the read_token provided during initialization. This is handled by a regional lookup mechanism.
# Regional URL Mapping
REGIONS = {
'us': {'base_url': 'https://logfire-api.pydantic.dev'},
'eu': {'base_url': 'https://logfire-api-eu.pydantic.dev'},
}
Sources: logfire/_internal/config.py:1181-1191, logfire/experimental/query_client.py:91-95
Authentication
The read_token is passed in the authorization header for all requests. While LogfireClient (internal) uses standard sessions, the LogfireQueryClient explicitly updates headers during instantiation.
# Internal Header setup in _BaseLogfireQueryClient
headers = client_kwargs.pop('headers', {})
headers['authorization'] = read_token
self.client: T = client(timeout=timeout, base_url=base_url, headers=headers, **client_kwargs)
Sources: logfire/experimental/query_client.py:57-61, logfire/_internal/client.py:38
Summary
The "Query Client & DB API" module serves as the primary bridge for extracting processed telemetry from Logfire. By wrapping httpx and providing native support for pyarrow, it enables sophisticated data analysis workflows. The separation of LogfireQueryClient and AsyncLogfireQueryClient ensures compatibility with both traditional scripts and modern asynchronous frameworks, while the regional URL inference ensures seamless connectivity based on user authentication tokens.
Sources: logfire/experimental/query_client.py, logfire/_internal/config.py