Structured Logging Engine

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_core/_utilities/_logging_config_parts/logging_config_part_01.py
• src/flext_core/_utilities/_logging_config_parts/logging_config_part_02.py
• src/flext_core/_utilities/_logging_config_parts/logging_config_part_03.py
• src/flext_core/_loggings_parts/flextlogger_part_04.py
• tests/unit/test_loggings_full_coverage.py
• tests/unit/test_coverage_loggings.py

The Structured Logging Engine in FLEXT-Core is a robust, asynchronous logging system built upon structlog. It is designed to provide high-performance, structured output (JSON or Console) while ensuring context propagation across different application layers. The engine employs an MRO-based mixin architecture to maintain a clean facade while providing comprehensive logging capabilities, including performance tracking and service infrastructure integration.

Key goals of the engine include idempotent configuration, asynchronous log writing to prevent I/O blocking in the main execution thread, and level-based context filtering. This allows developers to enrich log entries with domain-specific metadata, correlation IDs, and execution metrics without significant overhead.

Architecture and Core Components

The logging engine is structured into specialized parts that handle configuration, asynchronous writing, and high-level integration.

Asynchronous Log Writer

To prevent logging operations from stalling the main application thread, the engine uses a dedicated _AsyncLogWriter. This component inherits from io.TextIOBase and utilizes a internal queue.Queue to buffer log messages. A background daemon thread processes this queue and writes messages to the target stream (typically sys.stdout or sys.stderr).

The writer includes a shutdown mechanism registered via atexit to ensure all remaining messages in the queue are flushed before the process terminates.
Sources: logging_config_part_01.py:32-153

Processor Chain

The engine assembles a standard structlog processor chain that handles context merging, timestamping, and rendering.

Processor	Description
merge_contextvars	Merges thread-local and context-local variables into the event dict.
add_log_level	Adds the severity level to the log entry.
level_based_context_filter	Custom FLEXT processor that filters context variables based on the active log level.
TimeStamper	Injects ISO-formatted timestamps.
StackInfoRenderer	Renders stack trace information if present.
JSONRenderer / ConsoleRenderer	Final output formatting based on configuration.

Sources: logging_config_part_02.py:100-120

Configuration and Bootstrapping

Configuration is managed through FlextUtilitiesLoggingConfig, which ensures that structlog is configured exactly once (the "once pattern").

Idempotent Setup Flow

The configure_structlog method resolves parameters from settings models or direct arguments, builds the processor chain, and initializes the async writer if enabled.

Context Filtering Logic

The level_based_context_filter allows developers to include detailed context that only appears at specific log levels. Keys prefixed with _level_ are parsed (e.g., _level_debug_query) and only included if the current logger level meets the requirement.
Sources: logging_config_part_02.py:38-75

High-Level Integration Helpers

The engine provides specialized helpers for common enterprise logging tasks through the Integration and PerformanceTracker classes.

Performance Tracking

The PerformanceTracker is a context manager that automatically calculates the duration of a block of code and logs the result. It captures success/failure status and exception details.

# Usage Example
with u.PerformanceTracker(logger, "data_sync_operation"):
    perform_sync()

Sources: flextlogger_part_04.py:27-72, test_loggings_full_coverage.py:133-146

Service Infrastructure Setup

The setup_service_infrastructure method standardizes how services initialize logging. It binds service names, versions, and automatically generates a truncated correlation ID (e.g., flext-abcd12345678) to enable request tracing across distributed systems.
Sources: flextlogger_part_04.py:77-101

API Surface and Usage Patterns

The public API is designed for ease of use while maintaining strict technical standards.

Method	Source File	Description
bind(**kwargs)	test_loggings_full_coverage.py	Returns a new logger with the provided context keys bound.
unbind(*keys, safe=True)	test_loggings_full_coverage.py	Removes specific keys from the logger context.
exception(msg, ...)	test_coverage_loggings.py	Logs an error with full stack trace info, accepting explicit exception objects.
clear_scope(scope)	test_coverage_loggings.py	Clears context variables for a specific scope (app, request, op).
track_domain_event(...)	flextlogger_part_04.py	Emits a structured log entry specifically for DDD domain events.

Formatting and Templates

The engine preserves standard Python logging templates (e.g., logger.info("User %s active", username)) while allowing additional structured keyword arguments.
Sources: test_coverage_loggings.py:91-99

Summary

The Structured Logging Engine provides a high-performance foundation for observability in FLEXT-Core. By combining asynchronous I/O, strict context propagation, and idempotent configuration, it ensures that logs are both rich in metadata and non-intrusive to application performance. The system's ability to filter context by level and track performance metrics out-of-the-box makes it a central component for debugging and monitoring clean-architecture applications.