Code Census Analysis
Relevant source files
The following files were used as context for generating this wiki page:
Code Census Analysis is a comprehensive inspection pipeline within the FLEXT infrastructure designed to discover, classify, and analyze Python objects across a multi-project workspace. Its primary purpose is to provide a unified inventory of symbols (classes, functions, constants) and detect architectural violations, such as unused code, cross-project duplicates, or misplaced modules based on defined tiers.
The system integrates with the refactoring engine to identify "Removal Candidates" and provide auto-fix recommendations for identified violations. This ensures the workspace maintains a clean, non-redundant, and properly tiered structure.
Sources: src/flext_infra/_models/census.py:1-12, src/flext_infra/refactor/_census_collect_helpers.py:126-140
Architecture and Data Flow
The census process operates by scanning selected modules within a workspace, indexing their contents via Rope-based semantic analysis, and applying a set of rules to generate project-specific and workspace-wide reports.
The Analysis Pipeline
The data flow follows a sequence from configuration to collection, then to project-level assembly, and finally workspace aggregation.
The pipeline begins by resolving which families (e.g., constants, utilities) and rules should be active, which dictates whether a full object inventory or just lightweight module scans are required.
Sources: src/flext_infra/refactor/_census_collect_helpers.py:126-160, src/flext_infra/refactor/_census_collect.py:102-140
Core Components
| Component | Responsibility |
|---|---|
ScanConfig | Defines filters for symbol kinds, violation rules, and whether to resolve object references or local scopes. |
Census.Object | Data contract representing a discovered Python symbol, including its kind, tier, and reference counts. |
ProjectReport | Aggregates discovered objects, violations, and proposed fixes for a single project. |
WorkspaceReport | The final artifact containing cross-project duplicates and aggregated metrics for the entire workspace. |
RemovalCandidate | Identifies objects eligible for deletion based on "unused" or "test_only" criteria. |
Sources: src/flext_infra/_models/census.py:40-300, src/flext_infra/refactor/_census_collect_helpers.py:144-160
Data Models
The system relies on structured Pydantic models to ensure consistency across the refactoring and analysis tools.
Census Object Model
The m.Infra.Census.Object model is the atomic unit of the census. It captures the semantic fingerprint and usage metrics of a symbol.
class Object(mm.AbsoluteFilePathTextMixin, ...):
name: Annotated[t.NonEmptyStr, m.Field(description="Object identifier")]
kind: Annotated[str, m.Field(description="class/function/method/constant/...")]
references_count: Annotated[t.NonNegativeInt, m.Field(description="Total refs")]
runtime_references_count: t.NonNegativeInt
test_references_count: t.NonNegativeInt
fingerprint: str = ""
Sources: src/flext_infra/_models/census.py:40-120
Violation and Fix Contracts
Detected issues are stored as Violation objects, which may include metadata for automated resolution.
| Field | Description |
|---|---|
kind | Type of violation (e.g., misplaced, duplicate, unused, wrong_tier). |
severity | Level of importance (defaults to WARNING). |
fixable | Boolean indicating if a Fix action is available. |
fix_action | The identifier for the recommended auto-fix (e.g., move_to_tier). |
Sources: src/flext_infra/_models/census.py:180-210
Rule Analysis and Classification
The census identifies specific architectural issues by applying rules during the module scan.
Violation Types
The system evaluates several categories of rules to maintain code quality:
- Duplicate Detection: Identifies symbols defined in multiple projects. If values are identical, they are grouped as a
DuplicateGroup. - Tier Validation: Checks if an object's location matches its
expected_tiervsactual_tier. - Reference Analysis:
- Unused: Objects with zero references outside their definition.
- Test-Only: Objects referenced exclusively from the
tests/directory.
Sources: src/flext_infra/refactor/_census_project.py:75-120, src/flext_infra/_models/census.py:245-270
Removal Candidates
When objects are identified as unused or test_only, they are flagged as RemovalCandidate items. These candidates include reference sites that block or support their deletion, allowing developers to review aggressive-removal suggestions.
Sources: src/flext_infra/refactor/_census_project.py:120-135, src/flext_infra/_models/census.py:130-175
Integration with Quality Gates
Code Census Analysis is a critical component of the FlextInfraCodegenQualityGate. It provides the "after" metrics used to determine if a codegen or refactor operation was successful.
The Quality Gate specifically checks for:
- Namespace Compliance: Total violations must be zero.
- MRO Validity: No MRO failures.
- Duplication Reduction: Groups of duplicate constants must be resolved.
- Type Safety: Integration with external tools like
PyreflyandRuff.
Sources: src/flext_infra/codegen/constants_quality_gate.py:38-75, src/flext_infra/codegen/constants_quality_gate.py:195-230
Conclusion
Code Census Analysis serves as the "source of truth" for the state of symbols within a FLEXT workspace. By combining deep semantic analysis with architectural rules, it enables the infrastructure to not only report on technical debt (like unused code and duplicates) but also to enforce strict tiering and namespace standards through automated Quality Gates. This system is foundational for maintaining large-scale Python monorepos where manual symbol tracking is unfeasible.