Zero-Dependency Test Suite

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• test/cli_test.rb
• rake/docs_contract.rb
• action.yml
• test/site_test.js
• lib/mutineer/reporter.rb
• test/rspec_integration_test.rb
• rake/site_build.rb

The Zero-Dependency Test Suite ensures the stability and correctness of the Mutineer project while adhering to the core principle of using only the Ruby standard library (stdlib) and Prism. It validates mutation testing logic, CLI behavior, documentation consistency, and CI integrations without requiring external runtime gems.

Core Testing Philosophy

Mutineer maintains a strict "Zero runtime dependencies" policy. The test suite verifies that the gem functions correctly using only Prism and Ruby version 3.4 or higher. While runtime dependencies are forbidden, the project utilizes development-only dependencies such as minitest, rake, yard, playwright, and node.js for comprehensive validation.

Sources: AGENTS.md:5-7, AGENTS.md:43-48

The diagram shows the separation between runtime requirements and development-time validation tools.

CLI and Subprocess Validation

The suite uses Open3.capture3 to drive the bin/mutineer executable as a real subprocess. This method exercises flag parsing, validation exit codes, and operator listings in an environment that simulates real-world usage.

Subprocess Execution Logic

To maintain isolation during CLI tests:

1. The suite creates temporary directories using Dir.mktmpdir.
2. It copies fixtures (e.g., calculator.rb) into the isolated directory.
3. It executes the mutineer command within that specific path to prevent local .mutineer caches from interfering with the repository state.

Sources: test/cli_test.rb:14-17, test/cli_test.rb:63-68

CLI Exit Code Contract

The test suite asserts specific exit codes for various scenarios to maintain a stable machine-readable contract.

Code	Meaning	Source
0	Score ≥ threshold and no baseline regression	rake/docs_contract.rb:45
1	Score below threshold, baseline regression, or runtime error	rake/docs_contract.rb:46
2	Usage error, invalid flags, or unreadable baseline	rake/docs_contract.rb:47

Documentation and Contract Testing

Mutineer employs a "single-source" contract for exit codes and JSON schemas. A dedicated DocsContract module generates derived surfaces, such as README.md fragments, action.yml descriptions, and HTML schema references.

Drift Prevention

The test/docs_contract_test.rb file prevents documentation drift by:

• Comparing committed Markdown files against a fresh generation.
• Asserting that every contract surface (README, STABILITY, JSON schema) carries identical exit code definitions.
• Verifying that action.yml descriptions match the central contract.yml.

Sources: rake/docs_contract.rb:102-113, test/docs_contract_test.rb:23-32

The sequence diagram illustrates the automated verification process for documentation consistency.

Integration Testing Frameworks

The suite validates Mutineer's ability to interface with different Ruby test runners, primarily Minitest and RSpec.

RSpec Integration

Integration tests drive Mutineer::Runner.execute in standalone mode to verify:

• Strong Specs: Killing every mutant results in a 100.0% score.
• Weak Specs: Survivors are correctly identified and reported.
• IO Handling: Specs that reopen $stdout or swap it for StringIO do not break the coverage capture.
• Kill Matrix: Correct identification of blind and redundant tests.

Sources: test/rspec_integration_test.rb:11-20, test/rspec_integration_test.rb:116-126

GitHub Action Integration

The action.yml defines the CI interface, which is tested to ensure:

• Proper installation of the gem when use-bundler is false.
• Automatic scoping of mutants via the since input using GitHub event payloads.
• Temp file routing for JSON reports to prevent baseline clobbering.

Sources: action.yml:144-160, action.yml:211-225

Website and Browser Testing

For the documentation site, the project uses a combination of Node.js and Python for link checking and browser behavior validation.

• Link Integrity: test/site_links.py (invoked via Node) validates local links, unique anchors, and rejects root-absolute URLs.
• Theme and UI Logic: test/site_test.js uses Node's vm module to execute mutineer.js in a simulated browser context, testing theme switching, copy-to-clipboard feedback, and video autoplay behavior without a full browser dependency.
• Playwright Integration: Full browser tests are executed via Playwright for visual and functional verification of the HTML reports.

Sources: test/site_test.js:14-17, test/site_test.js:122-135, AGENTS.md:27-33

Summary of CI Gates

To protect the zero-dependency architecture, the following gates must pass before merging:

• Zero-Dep Suite: bundle exec rake test.
• Documentation: bundle exec rake yard:strict (100% coverage including private methods).
• Compatibility: Minitest 5 and Minitest 6 cross-version testing.
• Site Stability: site:build followed by Node and Playwright tests.
• Dogfooding: Rails integration and mutation score regression checks.

Sources: AGENTS.md:14-22, AGENTS.md:38-41