Lazy Initialization Generation
Relevant source files
The following files were used as context for generating this wiki page:
Lazy Initialization Generation is a specialized codegen system within flext-infra designed to implement PEP 562 compliant lazy loading for Python packages. The system auto-discovers exports from sibling .py files and generates clean, high-performance __init__.py files that defer the cost of importing heavy modules until their symbols are actually accessed.
The primary goal of this system is to maintain a clean public API surface for large Python projects while optimizing startup time and resource usage. It processes package directories bottom-up, ensuring that child package exports are correctly bubbled up to parent packages and eventually to the root API facades.
Sources: src/flext_infra/codegen/lazy_init.py:1-24, src/flext_infra/_constants/codegen_lazy.py:1-12
Core Architecture
The system is built as a composite service using mixins to separate concerns such as planning, registry management, and file I/O. The primary entry point is the FlextInfraCodegenLazyInit class, which orchestrates the discovery and generation process across a workspace.
The architecture utilizes a FlextInfraRopeWorkspace to perform static analysis of the codebase without executing it, allowing for safe and fast discovery of classes, functions, and constants.
Sources: src/flext_infra/codegen/lazy_init.py:27-41, src/flext_infra/codegen/_lazy_init_generation.py:24-30
Generation Workflow
The generation process follows a strict sequence to ensure consistency and prevent export collisions.
1. Discovery and Indexing
The system scans the workspace using predefined glob patterns (e.g., src/**/__init__.py, tests/**/__init__.py) to identify package directories. It then indexes modules and their top-level symbols.
Sources: src/flext_infra/_constants/codegen_lazy.py:32-38, src/flext_infra/codegen/lazy_init.py:75-81
2. Duplicate Detection
Before generation, the system performs a workspace-wide check for duplicate class names. Duplicates are forbidden across src/ modules but allowed within local project scopes for tests/ or examples/.
Sources: src/flext_infra/codegen/lazy_init.py:108-140
3. Planning and Processing
The system processes directories bottom-up (sorted by the number of path parts in reverse). For each directory, a LazyInitPlan is built, which determines whether to skip, remove, or write a new initializer.
Key Components and Logic
Lazy Initializer Rendering
The generated __init__.py content includes several critical sections:
- Autogen Header: A standard warning header
"# AUTO-GENERATED FILE — Regenerate with: make gen". - PEP 562 Helpers: Imports from
flext_core.lazysuch asbuild_lazy_import_mapandinstall_lazy_exports. - Type Checking Block: An
if TYPE_CHECKING:block containing imports for IDE support and static analysis. - Lazy Map: A dictionary mapping module paths to exported symbol names.
Sources: src/flext_infra/_constants/codegen_lazy.py:15-18, src/flext_infra/codegen/_lazy_init_generation.py:168-181, tests/unit/codegen/lazy_init_service_tests.py:34-45
Registry and Typing Stubs
For complex packages, the system generates split registries (_exports_lazy_part_*.py) to avoid massive __init__.py files. It also generates .pyi typing stubs to provide full static analysis support for the lazily exported symbols.
| Artifact | Purpose | Logic |
|---|---|---|
__init__.py | Entry point | Uses install_lazy_exports for runtime access. |
_exports.py | Registry Root | Consolidates split parts in the _constants directory. |
_exports_lazy_part_XX.py | Split Map | Contains subsets of lazy imports (default part size: 32). |
__init__.pyi | Typing Stub | Provides static analysis hints for the package. |
Sources: src/flext_infra/_constants/codegen_lazy.py:20-31, src/flext_infra/codegen/_lazy_init_generation_registry.py:22-68
Configuration and Constraints
The system relies on several constants to enforce project-wide conventions:
| Constant | Value | Description |
|---|---|---|
MAX_LINE_LENGTH | 88 | Max line length for generated code. |
ROOT_EXPORTS_DIR | _constants | Location for generated lazy registries. |
DUPLICATE_CLASS_MIN_LEN | 8 | Min length to trigger duplicate check. |
NON_PUBLIC_LAZY_ROOTS | examples, scripts, tests | Roots treated as private plumbing. |
Sources: src/flext_infra/_constants/codegen_lazy.py:14-42, src/flext_infra/codegen/lazy_init.py:133-134
Implementation Example: Registry Generation
The FlextInfraCodegenLazyInitGenerationRegistryMixin manages the creation of secondary export files. If a package is flext_core or if a registry is required by the plan, it triggers the creation of parts.
# From src/flext_infra/codegen/_lazy_init_generation_registry.py:65-71
files = FlextInfraCodegenGeneration.generate_registry_files(
plan.context.current_pkg,
registry.name,
plan.lazy_map,
plan.child_packages_for_lazy,
plan.excluded_lazy_names,
registry_filename=registry_filename,
)
Sources: src/flext_infra/codegen/_lazy_init_generation_registry.py:65-71
Conclusion
Lazy Initialization Generation provides a deterministic, bottom-up approach to managing Python package entry points. By leveraging static analysis and PEP 562, it ensures that even extremely large project structures remain maintainable, performant, and friendly to modern development tools. The use of split registries and typing stubs further bridges the gap between runtime performance and developer productivity.