Code Generation Pipeline
Relevant source files
The following files were used as context for generating this wiki page:
The Code Generation Pipeline is a centralized orchestration system within flext-infra designed to enforce namespace governance, automate boilerplate creation, and ensure architectural consistency across a workspace. It integrates multiple specialized tools—Census, Scaffolder, Fixer, and Lazy-Init—into a unified workflow that can be executed in a single pass.
The pipeline operates by discovering governed projects, auditing them for namespace violations, scaffolding missing architectural components, and automatically fixing eligible violations. This process ensures that projects adhere to the FLEXT project class standards, particularly regarding the placement of constants, models, and protocols.
Sources: src/flext_infra/codegen/pipeline.py:1-40, tests/unit/codegen/pipeline_tests.py:100-140
Pipeline Architecture and Workflow
The pipeline is structured as a series of stages managed by the FlextInfraCodegenPipeline class. It maintains an internal state using CodegenPipelineState to share data between stages, such as discovered projects and audit reports.
Execution Stages
- Project Discovery: Identifies all projects within the workspace that are governed by codegen automation.
- Initial Census: Audits the workspace to establish a baseline of namespace violations.
- Scaffolding: Generates missing base modules (e.g.,
constants.py,models.py) insrc/andtests/directories. - Auto-Fix: Automatically resolves eligible namespace violations, such as moving
Finalconstants orTypeVardeclarations to their correct modules. - Lazy-Init Refresh: Regenerates PEP 562 lazy-import
__init__.pyfiles to reflect structural changes. - Final Census: Performs a post-fix audit to verify improvements and report remaining manual actions.
The following diagram illustrates the data flow through these stages:
Key Components
Scaffolder
The FlextInfraCodegenScaffolder is responsible for generating the standard FLEXT module structure. It ensures that every package has the necessary facade modules and test files.
- Primary Modules:
constants.py,typings.py,protocols.py,models.py,utilities.py. - Test Modules: Scaffolds corresponding test files in the
tests/directory with correct class prefixes.
Sources: src/flext_infra/codegen/scaffolder.py:1-50, tests/unit/codegen/pipeline_tests.py:15-25
Census and Fixer
The FlextInfraCodegenCensus tool scans the workspace for rule violations (e.g., NS-001, NS-002). The FlextInfraCodegenFixer then attempts to resolve these by relocating symbols to the correct files based on their kind.
Sources: src/flext_infra/_models/codegen.py:20-40, src/flext_infra/_constants/cli_routes.py:100-140
Data Structures
The pipeline relies on several Pydantic models to ensure typed communication between stages.
| Model | Purpose |
|---|---|
CodegenPipelineState | Stores discovered_projects, reports_before, reports_after, and stage results. |
CensusViolation | Represents a single rule violation with module, rule, message, and fixable status. |
ScaffoldResult | Tracks files_created and files_skipped during the scaffolding stage. |
AutoFixResult | Aggregates violations_fixed, violations_skipped, and files_modified. |
Sources: src/flext_infra/_models/codegen.py:20-100, src/flext_infra/_models/codegen.py:430-460
CLI Integration
The pipeline is exposed via the flext-infra codegen pipeline command. It supports apply and dry-run modes to control whether changes are written to the disk.
Summary
The Code Generation Pipeline provides a deterministic approach to maintaining workspace health. By automating the transition from loose definitions to structured, namespace-compliant modules, it reduces architectural debt and ensures that all projects within the FLEXT ecosystem remain byte-identical in their structural implementation when run twice.
Sources: src/flext_infra/codegen/pipeline.py, tests/unit/codegen/pipeline_tests.py:140-155