Testing & Benchmarking
Relevant source files
The following files were used as context for generating this wiki page:
- AGENTS.md
- CONTRIBUTING.md
- openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/tasks.md
- openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md
- openspec/changes/archive/spec-kit-old/002-replace-rng-currently/002-replace-rng-currently.md
- openspec/changes/archive/spec-kit-old/README_OLD.md
- thirdparty/crates/zerocopy-0.8.48/agent_docs/reviewing.md
Testing and benchmarking are foundational to the DeepCausality project, ensuring the reliability and performance of its computational causality algorithms. The project maintains a high standard of quality, targeting a sustained code coverage rate of approximately 95% to 97%. Testing encompasses unit, integration, and contract testing, often following a Test-Driven Development (TDD) cycle to ensure all logic is verified before implementation.
Sources: AGENTS.md:144, openspec/changes/archive/spec-kit-old/README_OLD.md:92
Testing Architecture and Conventions
The project employs a structured approach to testing that mirrors the source code organization. Every crate contains a tests folder that replicates the exact structure of the src folder. This consistency ensures that every component has a corresponding test suite located in a predictable path.
Test Folder Structure
| Source File Path | Corresponding Test File Path |
|---|---|
src/errors/mod.rs | tests/errors/mod.rs |
src/types/type_name.rs | tests/types/type_name_tests.rs |
src/traits/mod.rs | tests/traits/mod.rs |
Sources: AGENTS.md:144-156
The following diagram illustrates the relationship between the source tree and the test tree:
Diagram shows the structural mirroring between functional source code and its corresponding test modules.
Sources: AGENTS.md:144-158
Red-Green-Refactor Cycle
A strict TDD cycle is enforced for new features. Tests must be written and observed to fail (the "Red" phase) before any implementation is performed. Git commits are expected to demonstrate this sequence, showing tests existing prior to the functional code changes.
Sources: openspec/changes/archive/spec-kit-old/005-right-now-there/plan.md:60-64, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/tasks.md:7-12
Performance Benchmarking
Benchmarking is critical for algorithms where performance is a key requirement, such as the Maximum Relevance and Minimum Redundancy (mRMR) algorithm. The project utilizes criterion or similar profiling tools to quantitatively measure performance gains, particularly when implementing parallelization.
Benchmarking Targets
- Algorithm Performance: Measuring the execution time of core algorithms on representative datasets (e.g., sepsis case study data).
- Regression Monitoring: Ensuring that new features (such as returning feature scores) do not degrade performance beyond an acceptable budget (typically <10%).
- Parallelization Gains: Validating the effectiveness of parallel processing guarded by the
parallelfeature flag.
Sources: openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md:58-62, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:27-28, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/tasks.md:23-25
Flowchart of the performance validation process used to verify parallelization gains and regression budgets.
Sources: openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md:58-62
Specialized Testing Requirements
Random Number Generation (RNG) Testing
For internal crates like deep_causality_rand, the project mandates 100% code coverage. This includes testing all error variants, edge cases, and corner cases. Specifically, for RNG implementations, tests must verify:
- Zero External Dependencies: Ensuring only the standard library is used.
- Floating-Point Comparison: Using epsilon-based tolerance (e.g.,
(a - b).abs() < epsilon) rather than direct equality checks for numerical stability in parallel environments. - Edge Cases: Handling scenarios where
stdfeatures might be restricted.
Sources: openspec/changes/archive/spec-kit-old/002-replace-rng-currently/002-replace-rng-currently.md:11-15, openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md:50-54
Data and Algorithm Edge Cases
Algorithms must be tested against specific data anomalies to ensure robustness:
- Missing Data: Handling
NaNvalues by converting them toNoneand verifying that the system prevents bias from substitution. - Empty Inputs: Verifying that
MrmrError::InvalidInputorSampleTooSmallis returned appropriately. - Insufficient Samples: Testing that pairwise deletion in statistical calculations results in correct error propagation if sample sizes fall below thresholds.
Sources: openspec/changes/archive/spec-kit-old/005-right-now-there/spec.md:22-26, openspec/changes/archive/spec-kit-old/006-mrmr-feature-score/spec.md:22-26
Tooling and Commands
The project utilizes make and cargo for standardized testing and quality checks.
| Command | Purpose |
|---|---|
make test | Tests the entire mono-repo (Comprehensive but slow). |
cargo test -p <crate_name> | Preferred method for testing a specific crate during development. |
cargo test --features parallel | Validates parallel implementations of algorithms. |
make format && make fix | Lints and formats code to maintain standards before testing. |
make check | Scans the codebase for security vulnerabilities. |
Sources: AGENTS.md:113-134, openspec/changes/archive/spec-kit-old/007-name-parallelize-mrmr/spec.md:58-62
Review and Quality Assurance
Testing is supplemented by rigorous code reviews. For sensitive code, such as unsafe blocks or security-critical logic, auditors follow an "Analyze-First" mandate.
Review Checklist for Tests
- Do tests achieve the target 95%+ coverage?
- Are unit tests provided for all error enums?
- Does every
unsafeblock (where permitted) have a corresponding safety justification verified by tests? - Are benchmarks included for performance-critical path changes?
Sources: thirdparty/crates/zerocopy-0.8.48/agent_docs/reviewing.md:12-25, openspec/changes/archive/spec-kit-old/002-replace-rng-currently/002-replace-rng-currently.md:144-150
The DeepCausality testing and benchmarking framework is designed to facilitate high-velocity development without sacrificing numerical accuracy or system stability. By combining strict folder mirroring, TDD cycles, and performance budgets, the project ensures its causal reasoning engine remains reliable in complex, dynamic environments.