Test Enforcement Discovery
Relevant source files
The following files were used as context for generating this wiki page:
- src/flext_tests/_fixtures/_enforcement_parts/discovery.py
- src/flext_tests/_fixtures/_enforcement_parts/build.py
- src/flext_tests/_fixtures/_enforcement_parts/hooks.py
- src/flext_tests/_fixtures/_enforcement_parts/validators.py
- tests/unit/test_enforcement_dispatcher.py
- src/flext_tests/_models/validator.py
Test Enforcement Discovery is a specialized subsystem within the FLEXT ecosystem designed to automatically identify, validate, and report architectural and quality violations during the test execution lifecycle. It operates as a pytest plugin integration that resolves the workspace environment, discovers applicable enforcement rules, and dynamically injects validation items into the test collection.
The system ensures that projects within a FLEXT workspace adhere to predefined standards by scanning for MRO (Method Resolution Order) violations, forbidden test patterns (e.g., restricted mock usage), and infrastructure inconsistencies. It supports both a passive warning mode and a strict enforcement mode that promotes warnings to test failures.
Sources: tests/unit/test_enforcement_plugin.py:1-13, src/flext_tests/_models/validator.py:76-107
Workspace and Target Discovery
The discovery process begins by identifying the FLEXT workspace root and the specific targets within that workspace that require validation. This mechanism ensures that enforcement only triggers when running within a valid ecosystem structure.
Workspace Root Resolution
The system identifies a workspace root by walking up the directory tree from a given path until it encounters a specific set of markers:
- An
AGENTS.mdfile. - A
flext-core/directory. - A
flext-tests/directory.
If these markers are missing, the dispatcher remains inactive.
Sources: tests/unit/test_enforcement_dispatcher.py:18-34, tests/unit/test_enforcement_dispatcher.py:80-91
Validator Target Collection
During the pytest collection phase, the system identifies "validator targets." These are specific paths (files or directories) that correspond to the items currently being collected by pytest. This prevents the system from scanning the entire workspace if only a subset of tests is being executed.
The following flowchart illustrates the high-level discovery and item injection flow:
The diagram shows how the system resolves the workspace before determining which rules and targets are relevant for the current session.
Sources: src/flext_tests/_fixtures/_enforcement_parts/hooks.py:18-32, src/flext_tests/_fixtures/_enforcement_parts/build.py:28-40
Enforcement Configuration
The behavior of the discovery system is governed by the EnforcementDispatcherConfig. This model tracks whether enforcement is active, which rules are included or excluded, and whether violations should be treated as strict failures.
Configuration Model
| Field | Type | Description |
|---|---|---|
active | bool | Whether the dispatcher is active for the current session. |
strict | bool | If true, promotes runtime warnings to test failures. |
include | frozenset[str] | Optional allow-list of enforcement rule IDs. |
exclude | frozenset[str] | Optional block-list of enforcement rule IDs. |
workspace_root | Path | None | The resolved FLEXT workspace root. |
warning_counter | MutableIntMapping | Counts of captured runtime warnings. |
Sources: src/flext_tests/_models/validator.py:76-107
Rule Dispatching and Validation
Once targets and rules are identified, the system dispatches validation tasks to specific detectors based on the "source kind" defined in the rule specification.
Supported Detectors
- Infra Detector (
flext_infra_detector): Scans workspace reports for missing files or structural project violations. - Tests Validator (
flext_tests_validator): Dynamically imports and executes methods fromFlextTestsValidatorto check for patterns like monkeypatch usage or forbidden imports in Markdown code blocks.
Validation Logic
The system merges violations by mapping them to specific "projects" based on their relative path to the workspace root. If a violation's file path cannot be resolved to a specific sub-project, it is categorized under the generic "workspace" segment.
The sequence shows how the build logic delegates the actual scanning to specialized validator methods.
Sources: src/flext_tests/_fixtures/_enforcement_parts/validators.py:38-75, src/flext_tests/_fixtures/_enforcement_parts/build.py:65-76
Runtime Monitoring and Summary
The enforcement system integrates with pytest hooks to monitor warnings and provide a terminal summary of the enforcement status.
Warning Capture
The pytest_warning_recorded hook tracks violations emitted as FlextMroViolation warnings. If strict mode is disabled, these are tallied; if strict mode is enabled, they are promoted to failures.
Sources: src/flext_tests/_fixtures/_enforcement_parts/hooks.py:35-51, tests/unit/test_enforcement_plugin.py:59-73
Terminal Summary
At the end of a test session, the system prints a summary section titled flext-enforce. This summary includes:
- The total number of active rules.
- A breakdown of rules per "source kind" (e.g.,
runtime_warning,flext_tests_validator). - The total count of runtime warnings captured during the run.
Sources: src/flext_tests/_fixtures/_enforcement_parts/hooks.py:60-84
Conclusion
Test Enforcement Discovery provides a robust framework for maintaining architectural integrity in the FLEXT ecosystem. By combining static file-system discovery with dynamic pytest integration, it ensures that standards are enforced consistently across diverse projects while providing developers with clear, categorized feedback on violations.