Minitest Integration

Relevant source files

The following files were used as context for generating this wiki page:

• lib/mutineer/test_runners/minitest.rb
• lib/mutineer/minitest_integration.rb
• lib/mutineer/minitest_integration/kill_recorder.rb
• lib/mutineer/minitest_integration/stop_at_first_failure.rb
• test/runner_test.rb
• lib/mutineer/reporter.rb

Minitest Integration provides the necessary hooks and extensions to execute Ruby test suites using the Minitest framework within the Mutineer mutation testing environment. It manages the execution of test files, captures failure data ("kills"), and implements performance optimizations such as early exit on failure and specific test ordering.

Sources: lib/mutineer/minitest_integration.rb, test/runner_test.rb:52-95

Core Components

The integration consists of several specialized components that interface with Minitest's reporter and plugin systems.

Component Overview

Component	Responsibility
MinitestRunner	Manages the high-level execution of Minitest.run.
KillRecorder	An extension of Minitest::AbstractReporter that logs failing tests.
StopAtFirstFailure	Logic that halts the suite as soon as a mutant is successfully killed.

Sources: lib/mutineer/test_runners/minitest.rb, lib/mutineer/minitest_integration/kill_recorder.rb, lib/mutineer/minitest_integration/stop_at_first_failure.rb

Kill Recorder

The KillRecorder acts as a Minitest reporter. It identifies exactly which test file and specific test case name caused a mutant to fail. This information is critical for building the Kill Matrix and generating detailed reports.

Sources: lib/mutineer/minitest_integration/kill_recorder.rb, lib/mutineer/reporter.rb:321-340

Stop at First Failure

To improve performance, Mutineer implements a "fail-fast" mechanism. Once a single test failure is detected in a mutant run, the mutant is considered "killed." The integration uses Minitest.after_run or internal process signals to stop further test execution for that specific mutant.

Sources: lib/mutineer/minitest_integration/stop_at_first_failure.rb, test/runner_test.rb:89-92

Execution Flow

The mutation runner follows a specific sequence when executing Minitest suites to ensure that test results are captured accurately and efficiently.

The Minitest integration hooks into the Minitest.run lifecycle to monitor test outcomes in real-time.

Sources: test/runner_test.rb:52-95, lib/mutineer/minitest_integration/stop_at_first_failure.rb

Optimization and Determinism

Seed Management

Mutineer ensures deterministic test execution by pinning the Minitest seed. For "stop at first failure" runs, it typically uses a specific constant seed to ensure that test order remains consistent across different mutant attempts.

• Default Seed: 0x4D54 is used for STOP_AT_FIRST_FAILURE_SEED.

Sources: test/runner_test.rb:92, lib/mutineer/minitest_integration/stop_at_first_failure.rb

Test Ordering

Mutineer orders Minitest files to minimize execution time. It prioritizes:

1. Tests paired by naming convention (e.g., app/models/user.rb pairs with test/models/user_test.rb).
2. Tests with the lowest historical execution time based on capture timings.

Sources: lib/mutineer/coverage_map.rb:116-125

Reporting Integration

Minitest results are aggregated by the Reporter class into multiple formats.

The KillRecorder provides the metadata used in JSON reports to identify the specific test cases that killed a mutant.

Sources: lib/mutineer/reporter.rb:120-135, lib/mutineer/minitest_integration/kill_recorder.rb

Usage Example in Tests

The integration is verified by observing how the Runner handles failures and stops execution.

# test/runner_test.rb:89-92
result = Mutineer::Runner.run(mutation, source_file: src, coverage_map: map)

assert_predicate result, :killed?, "expected killed, got #{result.status}"
refute_path_exists marker, "the mutant run must stop at the first failing test"

This ensures that no unnecessary tests are run after a kill is confirmed.

Sources: test/runner_test.rb:89-92