Lazy Import Optimization

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_core/lazy.py
• src/flext_core/_lazy_parts/flextlazy_part_01.py
• src/flext_core/__init__.py
• tests/unit/test_lazy_exports.py
• tests/unit/test_lazy_exports_merge.py
• tests/_utilities/__init__.py

Lazy Import Optimization in FLEXT-Core provides a high-performance, PEP 562-compliant mechanism to defer module loading until attributes are actually accessed. This system is critical for maintaining a lightweight "thin facade" architecture, reducing startup time, and preventing circular dependency issues in a large-scale enterprise framework. By utilizing dynamic __getattr__ and __dir__ overrides, the framework can expose a massive API surface (such as the flext_core root package) without incurring the performance penalty of importing every underlying module at initialization.

The core of this system is the FlextLazy container, which manages runtime caches for modules, import maps, and installation metadata. It supports advanced features like recursive map merging, relative path resolution, and automated bootstrap hooks for type-checking engines like Beartype.
Sources: src/flext_core/lazy.py, src/flext_core/_lazy_parts/flextlazy_part_01.py:44-55, src/flext_core/init.py:149-160

Architecture and Core Components

The lazy import system is built around a centralized registry and localized injection into module namespaces.

The FlextLazy Container

The FlextLazy class acts as the canonical API container and runtime engine. It utilizes Pydantic for internal configuration and manages five distinct caches to optimize repeated lookups and merges.

Cache Name	Type	Purpose
module_cache	dict[str, ModuleType]	Stores successfully loaded module objects to prevent redundant importlib calls.
child_lazy_cache	dict[str, LazyImportDict]	Stores raw _LAZY_IMPORTS maps extracted from child packages.
child_merge_cache	dict[tuple[str, ...], LazyImportDict]	Stores results of complex multi-package map merges.
normalized_map_cache	dict[tuple[str, int], LazyImportDict]	Stores maps after relative paths have been resolved to absolute paths.
install_cache	dict[str, tuple]	Tracks installation state of modules to ensure idempotency.

Sources: src/flext_core/_lazy_parts/flextlazy_part_01.py:44-71

Data Flow for Attribute Resolution

When a user accesses an attribute on a lazily-configured module, the PEP 562 __getattr__ function triggers the following flow:

The system ensures that once a symbol is resolved, it is injected into the module's globals(), making subsequent accesses as fast as a standard local variable lookup.
Sources: tests/unit/test_lazy_exports.py:126-140, src/flext_core/_lazy_parts/flextlazy_part_01.py:144-162

Implementation Details

Map Building and Merging

The framework provides utilities to construct complex import maps from multiple sources. build_lazy_import_map creates a flat mapping where keys are the public names and values are either module paths (strings) or attribute pairs (tuples).

_LAZY_IMPORTS = build_lazy_import_map(
    {
        ".case_factories": ("TestsFlextUtilitiesCaseFactoriesMixin",),
        "flext_tests": ("c", "r", "t", "u"),
    },
)

Sources: tests/_utilities/init.py:9-16, src/flext_core/_lazy_parts/flextlazy_part_01.py:186-200

Recursive Merging

merge_lazy_imports (and FlextLazy.merge) allows a parent package to aggregate lazy exports from its children. This is used extensively in the flext_core root and test suites to provide unified access points.

Installation and Idempotency

install_lazy_exports is the primary entry point for enabling lazy loading in a module. It injects __getattr__ and __dir__ into the target globals().

• publish_all: If True, it populates __all__ with the keys of the lazy map.
• public_exports: An optional whitelist to restrict which lazy symbols are visible in dir() or __all__.
• Idempotency: The install_cache prevents re-installing the logic if the map has not changed, which is vital during complex reload scenarios.

Sources: src/flext_core/lazy.py, tests/unit/test_lazy_exports.py:179-195

Diagnostics and Performance

The system includes observability features through cache_stats, which exposes the current sizes of all internal caches. This allows developers to monitor memory usage and ensure that the lazy engine is functioning as expected.

Cache Statistics Structure

Field	Description
module_cache	Count of unique modules imported via the lazy engine.
child_lazy_cache	Count of scanned child packages.
normalized_map_cache	Count of unique module/map combinations processed.

The lazy.reset() method can be used to clear all caches, which is primarily used during testing to ensure isolation between test cases.
Sources: src/flext_core/_lazy_parts/flextlazy_part_01.py:90-100, tests/unit/test_lazy_exports_merge.py:145-163

Conclusion

Lazy Import Optimization in FLEXT-Core is a robust system that balances the need for a comprehensive, easily accessible API with the performance requirements of enterprise applications. By centralizing import logic and utilizing sophisticated caching and merging strategies, it ensures that the framework remains responsive even as the number of internal modules grows.