Developer Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• CONTRIBUTING.md
• README.md
• src/nu/autoload.rs
• tests/module_autoload_test.rs
• REVIEW.md
• CLAUDE.md

Introduction

The Numan project employs a multi-tiered testing strategy designed to ensure cross-platform reliability, data integrity, and safe integration with Nushell. The strategy encompasses unit testing, hermetic integration testing, and real-world acceptance testing using actual Nushell binaries. A core tenet of the strategy is the use of injectable seams to prevent side effects during lower-level testing while maintaining high coverage of complex logic like activation journals and lockfile mutations.

Testing is integrated directly into the development workflow and CI/CD pipeline, enforcing strict quality gates for formatting, linting, and behavioral correctness across Linux, macOS, and Windows. This ensures that features like Module Autoload and Plugin Lifecycle are verified against the specific constraints of the user's environment.

Sources: CONTRIBUTING.md:3-8, AGENTS.md:14-25, README.md:17-25

Testing Tiers

Numan categorizes tests into three distinct levels to balance execution speed with environmental accuracy.

1. Unit Testing

Unit tests are located inline within source modules or in tests/. They focus on pure logic, such as path rendering, version constraint matching, and metadata parsing. Developers are strictly prohibited from spawning real nu processes in unit tests to ensure they remain fast and hermetic.
Sources: CONTRIBUTING.md:46-48, AGENTS.md:162-164, REVIEW.md:40-41

2. Hermetic Integration Testing

Integration tests exercise multiple components (e.g., the install transaction or activation flow) using a tempfile::TempDir as a mock Numan root. These tests utilize "injectable seams" like FakeCandidateRunner to simulate Nushell's behavior without requiring a binary on the host system.
Sources: tests/module_autoload_test.rs:18-23, AGENTS.md:112-115

3. Real-Nu Acceptance Testing

Marked with the #[ignore] attribute, these tests require a real Nushell binary on the PATH. They verify the actual integration between Numan and Nushell, including plugin registration and vendor autoload loading. CI runs these tests on all supported operating systems.
Sources: CONTRIBUTING.md:27-30, AGENTS.md:148-155, README.md:415-417

Testable Seams and Mocking

To maintain hermeticity, Numan uses trait-based injection for components that interact with the external environment.

Candidate Runner Seam

The CandidateRunner trait abstracts the execution of <nu> -n <path> validation. This allows the testing suite to verify that Numan correctly handles both successful and failed Nushell validations without actually invoking the shell.

The diagram above shows the relationship between the real Nushell runner and the fake runner used for testing module activation.
Sources: src/nu/autoload.rs:198-258, tests/module_autoload_test.rs:18-23

Injectable Registrars

Activation and deactivation commands utilize injectable registrars to simulate plugin management.

• Plugins: execute_with_registrar(args, root, registrar)
• Modules: execute_with_candidate_runner(args, root, registrar, runner)
  Sources: AGENTS.md:112-120

Quality Gates and CI

Numan enforces a set of non-negotiable quality gates that must pass before any code is merged into the master branch.

Gate	Command	Description
Unit/Integration	cargo test	Runs the full suite of hermetic tests.
Linting	cargo clippy -- -D warnings	Enforces Rust best practices and Numan-specific rules (e.g., &Path usage).
Formatting	cargo fmt --check	Ensures consistent code style.
Acceptance	cargo test -- --ignored	Executes real-Nu integration tests (CI only).
MSRV	CI msrv job	Verifies compatibility with Minimum Supported Rust Version (1.88).

Sources: CONTRIBUTING.md:14-25, AGENTS.md:16-30, REVIEW.md:7-12

Data Integrity and Journaling

Testing strategies for state-mutating commands focus on crash recovery and atomic operations. Tests verify that:

1. Mutation Lock: Only one process can modify the root at a time, verified by acquire_mutation_lock.
2. Journal Reconciliation: If a process is interrupted, the next run reconciles the pending-activation.json or pending-autoload.json journals.
3. Snapshots: create_snapshot() is called before mutations, allowing for rollback verification in tests.

Sources: AGENTS.md:126-147, REVIEW.md:27-38, tests/module_autoload_test.rs:596-615

Developer Workflow Logic

The project follows a "test-first" approach where developers are encouraged to write a test, verify the failure, implement the fix, and then verify the pass.

This flowchart illustrates the standard Numan development cycle from branch creation to merging.
Sources: CONTRIBUTING.md:33-40, AGENTS.md:157-160

Conclusion

Numan’s testing strategy ensures that the package manager remains reliable across disparate platforms by isolating side effects through mocks while validating end-to-end behavior with ignored acceptance tests. By enforcing strict CI gates and journaling verification, Numan guarantees that user configurations remain safe even in the event of process crashes or environment drift.