Test Isolation & File Swapping
Relevant source files
The following files were used as context for generating this wiki page:
Mutineer maintains test integrity by isolating mutation side effects from the original source code and the parent process. It uses file swapping to replace source code with mutants during execution and employs process-level isolation to ensure that crashes or global state changes in one mutant do not affect the overall run.
Overview of Isolation Strategies
Mutineer provides different strategies to isolate and execute mutations. These strategies determine how the code is loaded and how tests run against it.
Reload and Redefine Strategies
The system supports two primary isolation strategies:
- Reload (Whole-file): Loads the entire modified source file into the runtime.
- Redefine (Surgical): Redefines specific methods or constants within the existing runtime.
Sources: lib/mutineer/cli.rb:44-45, lib/mutineer/reporter.rb:141-143
Process Isolation
To prevent a mutant from corrupting the parent process, Mutineer forks a child process for every mutant execution. This ensures that:
- Global state changes remain in the child.
- Process crashes do not stop the runner.
- Timeouts can be enforced per mutant.
Sources: lib/mutineer/cli.rb:46-51, lib/mutineer/kill_channel.rb:28-30
File Swapping Logic
The file swapping mechanism physically replaces the source file on disk with a mutated version before the test runs. This is critical for strategies like reload where the Ruby interpreter needs to read the file from disk.
The Swap Lifecycle
The swap process follows a strict sequence to ensure the original file is always restored, even if the process is interrupted.
The diagram shows the lifecycle of a file swap during a mutation test.
Sources: lib/mutineer/file_swap.rb, lib/mutineer/isolation.rb
Safety Mechanisms
- Backup Files: The original source is moved to a
.bakextension before the mutant is moved into the primary path. - Atomic Renames: Renaming is used to minimize the window where no file exists at the source path.
- Cleanup: The system ensures restoration occurs during both normal completion and exception handling.
Sources: lib/mutineer/file_swap.rb, lib/mutineer/cli.rb:259-261
Inter-Process Communication (Kill Channel)
When running with --matrix, the child process communicates test results back to the parent using the KillChannel. This protocol uses a line-based JSON format over an IO pipe.
Communication Flow
The sequence diagram illustrates the communication protocol between the parent process and the mutated child process.
Sources: lib/mutineer/kill_channel.rb:5-25
Channel Events
| Event | Meaning |
|---|---|
start | The recorder is armed and starting the run. |
pass | A test passed against the mutant. |
kill | A test failed or errored (mutant killed). |
parallel | Marks the transition to parallel test execution. |
skip | Marks a region that a non-matrix run would skip. |
cleanup | Tests finished; suite hooks (like after(:suite)) are starting. |
end | The suite returned and all tests were observed. |
Sources: lib/mutineer/kill_channel.rb:5-25, lib/mutineer/kill_channel.rb:62-95
Daemon Mode and DB Isolation
For complex applications like Rails, Mutineer supports a --daemon mode. This boots the application once in a persistent process and then forks per mutant.
- Persistent Boot: The app environment boots once, reducing overhead.
- DB Isolation: Supports per-worker database isolation for SQLite, PostgreSQL, and MySQL.
- Forking: Uses
forkto create workers from the warmed-up daemon process.
Sources: lib/mutineer/cli.rb:62-68, lib/mutineer/cli.rb:216-224
Summary
The isolation and file swapping systems provide a robust environment for mutation testing. By combining physical file replacement with process-level isolation and a structured communication protocol, Mutineer ensures that individual mutants are tested accurately without compromising the stability of the test runner or the original source code.