Testing Strategy & Configuration
Relevant source files
The following files were used as context for generating this wiki page:
The flext-quality project implements a multi-layered testing strategy to validate code quality analysis, domain business rules, and ecosystem integration. This system ensures that the unified orchestration platform for Claude Code tooling maintains high reliability and performance across unit, integration, and end-to-end boundaries.
The test suite enforces a minimum 90% coverage requirement across all source modules, including branch coverage for conditional logic and function coverage for public APIs. Tests must be deterministic and isolated, with unit tests targeted to complete in under 100ms and integration tests in under 5 seconds.
Sources: tests/README.md:1-26, tests/README.md:104-124
Test Architecture and Categories
The project organizes tests into a hierarchical structure that separates concerns by scope and dependency levels. This structure allows for targeted execution during development and comprehensive validation in CI/CD pipelines.
Hierarchy and Execution Flow
The diagram shows the logical categorization of tests from isolated unit checks to full system end-to-end journeys.
Sources: tests/README.md:28-40, tests/README.md:42-70
Key Test Types
| Category | Location | Focus |
|---|---|---|
| Unit | tests/unit/ | Isolated component testing with comprehensive mocking of domain logic and algorithms. |
| Integration | tests/integration/ | Validation of interactions between components and real dependencies like Ruff or MyPy. |
| E2E | tests/e2e/ | Complete user journey testing, including CLI workflows and multi-format report creation. |
| Helpers | tests/helpers/ | Utilities and shared functions for assertions and test data generation. |
Sources: tests/README.md:42-70
Configuration Management Testing
The system utilizes FlextQualityConfigManager to handle YAML-based configurations for audit rules, style guides, and validation settings. Testing for this module focuses on caching behavior, default fallbacks, and real YAML override logic.
Configuration Lifecycle
The FlextQualityConfigManager ensures that configurations are resolved efficiently through memoization.
This sequence demonstrates the caching mechanism where repeated lookups return the same cached configuration object to minimize filesystem I/O.
Sources: tests/unit/test_docs_config_manager.py:27-33, tests/unit/test_docs_config_manager.py:59-65
Quality Rules and Validation Testing
The FlextQualityRulesEngine and FlextQualityValidators are tested against real files to ensure that pattern matching and tier-based validation function correctly. The rules engine supports lazy loading of rules from YAML files.
Validation Logic Components
- Pattern Validator: Surfaces violations based on configured regex or string patterns.
- Tier Validator: Enforces architectural boundaries (e.g., preventing a tier-0
constants.pyfrom importing from services). - Forbidden Pattern Validator: Flags specific suppression markers (e.g.,
# type: ignore) as violations.
Sources: tests/unit/test_rules_engine_validators.py:157-220
Rule Enforcement Workflow
This flow illustrates how the engine validates a single file or directory, ensuring context (like "stage": "ci") is attached to every produced violation.
Sources: tests/unit/test_rules_engine_validators.py:75-115
CLI and API Facade Testing
The FlextQuality facade and FlextQualityCli represent the public surface of the project. Testing ensures that the observable contract—including return types (r[T]), status snapshots, and command construction—remains stable.
CLI Command Composition
The CLI builds specialized command sequences for different quality gates:
- Check: Combines
ruff(linting) andbasedpyright(type-checking). - Validate: Extends
Checkwithbandit(security) andcoverage(test results).
Sources: tests/unit/test_cli.py:46-80
API Status Contract
| Key | Description |
|---|---|
name | The service identifier. |
version | The current package version. |
settings | A snapshot of the active configuration. |
hooks_registered | List of active integration hooks. |
Sources: tests/unit/test_api.py:52-65
Tooling and Environment Configuration
The test execution environment is managed via pyproject.toml and Makefile. The project uses pytest with specific plugins for benchmarking, coverage, and timeout management.
Pytest Metadata
- Strict Markers: Enforced via
--strict-markersto prevent typos in test categorization. - Fail Under: The coverage report is configured to fail the build if coverage drops below 90%.
- Timeouts: Default test timeout is 10 seconds, with a
flext_slow_timeout_secondsoverride of 30 seconds for complex integration tests.
Sources: pyproject.toml:500-545, tests/README.md:144-162
Summary
The testing strategy of flext-quality emphasizes strict architectural boundaries and comprehensive coverage. By combining isolated unit tests for domain logic with path-based integration tests for the rules engine and CLI, the project ensures that quality gates—such as Ruff, Bandit, and basedpyright—are orchestrated correctly. Configuration is validated through a dedicated manager that supports environment-specific overrides and caching, maintaining performance without sacrificing flexibility.