Testing Services & Hooks
Relevant source files
The following files were used as context for generating this wiki page:
Testing services and hooks within the FLEXT infrastructure provide a robust framework for validating code changes, ensuring refactoring safety, and maintaining high-quality gates across the workspace. These services integrate directly with pytest and custom linting logic to enforce "protected edits," where modifications to source code are automatically rolled back if they introduce new linting errors or break existing functional tests.
The system relies on a combination of standard pytest configurations, custom regex patterns for parsing tool output, and specialized utility classes that manage the execution and verification of test suites during automation tasks like dependency modernization and large-scale refactoring.
Protected Edit Validation Flow
The core of the testing hook system is the "Protected Edit" mechanism. This system ensures that any automated change to the codebase (e.g., during a refactor census or a make mod dependency update) is verified against a baseline of existing tests and linting rules.
Safety Execution Logic
When a protected edit is initiated, the system captures a "snapshot" of the current state, applies the change, and then runs a verification suite. If the verification fails, the system automatically restores the original source.
The logic differentiates between functional failures and "no tests collected" scenarios. If a file contains no collectable tests (detected via AST parsing), the system performs a py_compile check instead to ensure syntax validity.
Sources: src/flext_infra/_utilities/protected_edit_apply.py:88-142, src/flext_infra/_utilities/protected_edit_apply.py:180-244
Key Validation Methods
| Method | Description |
|---|---|
_pytest_failure | Runs pytest on a specific file with -x (fail-fast) and --tb=short options. |
_file_contains_tests | Uses ast.parse to detect test_ functions or Test classes. |
lint_snapshots | Collects current linting errors from multiple tools as a baseline. |
protected_file_edit | High-level wrapper that applies an edit_fn and performs rollback on failure. |
Sources: src/flext_infra/_utilities/protected_edit_apply.py:114-142, src/flext_infra/_utilities/protected_edit_apply.py:180-210
Testing Infrastructure Configuration
The project utilizes pyproject.toml as the primary configuration authority for testing behaviors. This includes standard markers, environment variables, and reporting thresholds.
Pytest Configuration
The [tool.pytest.ini_options] section defines the standard execution environment for all FLEXT projects:
- Strict Markers: Enforces explicit marker registration to prevent typos.
- Markdown Docs: Enables testing of code snippets within markdown files.
- Ignored Paths: Standardizes which directories (like
_utilitiesor_models) are skipped during collection to avoid unnecessary overhead.
Sources: pyproject.toml:442-458, src/flext_infra/deps/tool_config.yml:450-466
Standard Test Markers
The infrastructure defines a set of canonical markers used to categorize tests across the workspace:
unit: Isolated logic tests.integration: Tests covering multiple components.slow: Tests that exceed standard execution budgets.docker: Tests requiring containerized environments.e2e: Full end-to-end flows.
Sources: pyproject.toml:462-475, src/flext_infra/deps/tool_config.yml:441-449
Refactor Census & Test Hooks
The refactor census module includes specialized hooks to identify "test-only" candidates. These are functions or classes that are only referenced within the tests/ directory and are candidates for removal or relocation if they no longer serve the production codebase.
Test-Only Detection Flow
The census service scans the project's object inventory and filters references by their file paths. If all reference sites for a symbol reside in files within a tests/ path, the symbol is flagged with a test_only violation.
Diagnostic & Pattern Matching Hooks
The testing system includes hooks for parsing and acting upon the output of various diagnostic tools. These are primarily defined via regular expressions in the constants and patterns modules.
Mypy & Markdown Patterns
The system uses specific patterns to capture feedback from external tools during test execution:
- Mypy Hints: Patterns like
note: Hint: "python3 -m pip install (.*)"are used to suggest missing stub packages. - Markdown Links: Patterns like
\[(.*?)\]\((.*?)\)validate that documentation links are well-formed duringmake docsandmake test.
Sources: tests/unit/test_infra_patterns_core.py:12-25, tests/unit/test_infra_patterns_core.py:44-58
Quality Gate Thresholds
Quality gates are enforced through coverage requirements that vary by project class. These thresholds are defined in the tool configuration and checked during the val (validate) phase.
| Project Class | Minimum Coverage % |
|---|---|
core | 45% |
domain | 30% |
platform | 25% |
integration | 20% |
app | 15% |
Sources: src/flext_infra/deps/tool_config.yml:327-333
Summary
Testing Services & Hooks in flext-infra represent a defensive engineering layer. By combining AST-based test detection, transactional source writes with rollback capabilities, and standardized quality gate configurations, the system ensures that automated modernization and refactoring tools cannot degrade the health of the workspace. These services transform testing from a passive check into an active participant in the code generation and maintenance lifecycle.