Testing Strategy & Make Contract
Relevant source files
The following files were used as context for generating this wiki page:
The testing strategy and Make contract for the flext-oracle-wms project establish a rigorous quality framework designed for enterprise-grade Oracle Warehouse Management System (WMS) integration. This framework ensures that all code meets strict quality gates, including high coverage targets and automated validation through canonical commands. The strategy encompasses unit, integration, end-to-end, and performance testing, all managed through a standardized interface that enforces "Continuous Green" development.
Sources: tests/README.md:1-26, src/README.md:144-149, AGENTS.md:54-61
Canonical Make Contract
The project utilizes a strict "Make Contract" where all build, check, test, and validation actions are executed through a predefined set of make verbs. This approach prevents ad-hoc tool invocations that might bypass safety guards or environment locks.
Primary Make Verbs
The following table details the core commands available in the Make contract for maintaining project health:
| Command | Purpose | Description |
|---|---|---|
make test | Full Test Execution | Runs the complete test suite with coverage enforcement (min 90%). |
make val | Quality Validation | Executes a complete quality gate: linting, type checking, security scans, and tests. |
make check | Static Analysis | Runs linters and formatters to ensure code style compliance. |
make lint | Code Quality | Specifically runs Ruff with zero-tolerance rules for code quality. |
make type-check | Type Safety | Executes MyPy in strict mode; targeting 95%+ type coverage. |
make security | Security Scan | Runs Bandit and pip-audit for security vulnerability scanning. |
make mod | Dependency Management | Regenerates managed pyproject sections for dependency standardization. |
Sources: src/README.md:151-158, tests/README.md:118-125, pyproject.toml:1-5, AGENTS.md:118-128
Execution Workflow
The following diagram illustrates the typical validation flow executed by a developer using the Make contract before pushing changes:
Sources: AGENTS.md:54-61, src/README.md:151-158
Testing Architecture
The project employs a multi-tiered testing architecture categorized by pytest markers. This ensures comprehensive coverage from individual components to real-world Oracle WMS Cloud connections.
Test Categories and Markers
| Marker | Type | Scope |
|---|---|---|
unit | Unit Test | Fast, mock-based testing of individual components with no external dependencies. |
integration | Integration Test | Testing component interactions and mock Oracle WMS server integration. |
declarative | Integration Test | Specific tests for the declarative API integration and catalog. |
real_connection | E2E Test | Tests requiring real Oracle WMS credentials to verify live API connectivity. |
performance | Benchmark | Performance and benchmarking tests with timing assertions. |
slow | Optimization | Categorizes tests with long execution times for CI/CD optimization. |
Sources: tests/README.md:73-95, pyproject.toml:474-486
Coverage Requirements
The project enforces strict coverage thresholds through CI quality gates:
- Overall Coverage: 90% minimum.
- Unit Test Coverage: 95% minimum for core modules.
- Integration Coverage: 80% minimum.
- Branch Coverage: 85% minimum for critical paths.
Sources: tests/README.md:144-152, src/README.md:104-110
Testing Standards & Patterns
The repository mandates specific engineering standards for test development to ensure reliability and maintainability.
The P0 Rule: Validation by Construction
Tests must remain valid regardless of changes in config/*.yaml or settings. Hardcoding or freezing current values in tests is strictly prohibited. Expected values must be read from the same Typed Single Source of Truth (SSOT) that production uses.
Sources: AGENTS.md:14-25
Test Structure (AAA Pattern)
All tests are expected to follow the Arrange, Act, Assert (AAA) pattern for clarity. Tests must use descriptive names and comprehensive assertions to verify complete behavior, including both success and failure paths.
Dependency Management
The test suite utilizes several specialized tools to maintain environment isolation:
- HTTP Mocking:
responsesis used for API testing without live instances. - Data Generation:
factory-boyandfakercreate realistic test data. - Time Control:
freezegunhandles time-dependent logic. - Mocking:
pytest-mockis used for dependency injection and mockingt.JsonValuetypes.
Sources: tests/README.md:218-235, pyproject.toml:113-118
Infrastructure and Environment
Tests are configured to support both mock and real execution modes via environment variables.
| Variable | Possible Values | Description |
|---|---|---|
ORACLE_WMS_TEST_MODE | mock, real | Switches between mock server and live Oracle instance. |
ORACLE_WMS_LOG_LEVEL | debug, info, error | Controls logging verbosity during test execution. |
ORACLE_WMS_TEST_TIMEOUT | Integer (e.g., 30) | Sets the timeout in seconds for API requests during tests. |
ORACLE_WMS_BASE_URL | URL | Target URL for real connection integration tests. |
Sources: tests/README.md:128-138
The project also maintains a dedicated Mock Oracle WMS Server within the core module (src/flext_oracle_wms/mock_server.py) to facilitate development and testing in environments without valid Oracle WMS credentials.
Sources: src/flext_oracle_wms/README.md:38, src/flext_oracle_wms/README.md:92
Conclusion
The flext-oracle-wms testing strategy is anchored in the "Continuous Green" principle, where no feature is considered complete until it passes all canonical quality gates. By strictly adhering to the Make Contract and enforcing high coverage through standardized AAA test patterns, the project maintains a production-ready status with 150+ comprehensive tests that validate both internal logic and external API contracts.
Sources: tests/README.md:286-292, AGENTS.md:54-61