RSpec Integration
Relevant source files
The following files were used as context for generating this wiki page:
Mutineer integrates with the RSpec testing framework to perform mutation analysis on Ruby applications. It identifies RSpec suites, maps source code to specific examples, and executes targeted tests against generated mutants. The integration supports complex RSpec features like shared examples and custom I/O handling.
Framework Detection and Configuration
Mutineer identifies the testing framework through automatic detection or explicit user configuration.
Automatic Detection
Mutineer auto-detects RSpec by inspecting test file names and directory structures. If test files end with _spec.rb or are located within a spec/ directory, the system sets the framework to rspec. If a tie occurs (e.g., equal numbers of _spec.rb and _test.rb files), Mutineer defaults to minitest.
Sources: test/config_test.rb:495-502, test/cli_test.rb:927-940
Explicit Configuration
You can override automatic detection using the --framework rspec CLI flag or by setting framework: rspec in the .mutineer.yml configuration file. Explicit configuration via CLI takes precedence over configuration files.
Sources: test/cli_test.rb:880-900, action.yml:45-48
Execution Lifecycle
The RSpec integration follows a multi-phase execution flow to ensure efficiency and accuracy.
The diagram shows the standard flow of a mutation run where RSpec is the identified test runner.
Sources: test/rspec_integration_test.rb:81-98, test/coverage_map_test.rb:99-114
Smoke Check
Mutineer executes the unmutated RSpec suite before starting mutation analysis. If any example fails in the unmutated state, Mutineer raises a Mutineer::SmokeCheckError and aborts. This prevents false positive reports where existing test failures are interpreted as successful mutant kills.
Sources: test/rspec_integration_test.rb:81-98
Phase A: Coverage Mapping
During Phase A, Mutineer runs the RSpec suite to map source lines to the specific examples that exercise them. This mapping ensures that in Phase B, only relevant examples run against a specific mutant.
Sources: test/coverage_map_test.rb:99-114
Phase B: Mutation Analysis
Mutineer generates mutants and runs the covering RSpec examples. A mutant is "killed" if the RSpec examples fail. It "survives" if the examples pass despite the code change.
Sources: test/rspec_integration_test.rb:23-38
Example Identification and Scoping
RSpec integration utilizes unique identifiers to manage test execution at a granular level.
Example IDs
RSpec assigns unique IDs to examples, such as [1:4:1]. Mutineer uses these IDs to differentiate between examples that share identical descriptions, such as parameterized shared examples or examples in nested groups.
Sources: test/rspec_integration_test.rb:116-131, lib/mutineer/reporter.rb:304-315
Scoped Execution
When running in scoped mode (e.g., using --since), Mutineer limits RSpec execution to mutants generated on changed lines. The JSON report marks these runs with summary.scoped: true.
Sources: lib/mutineer/reporter.rb:147-152, test/cli_test.rb:533-547
Handling Shared Examples and Environments
Mutineer handles common RSpec patterns to maintain reporting accuracy.
- Shared Examples: Mutineer tracks shared examples separately for each inclusion point using their scoped IDs.
- I/O Redirection: The integration handles specs that reopen
$stdoutor$stderr(e.g., usingto_stdout_from_any_process) without corrupting the coverage data or results. - Load-time Execution: Code that prints to standard output during load time is handled to prevent interference with the JSON capture channel.
Sources: test/rspec_integration_test.rb:43-70, test/rspec_integration_test.rb:133-145
Reporting and Matrix Analysis
When the --matrix flag is active, Mutineer provides a detailed breakdown of the RSpec suite's relationship with mutants.
| Metric | Description |
|---|---|
| Blind Tests | RSpec examples that ran against mutants but never participated in a kill. |
| Redundant Tests | RSpec examples that killed a mutant which was also killed by at least one other test. |
| Killed By | A list of RSpec example indices that successfully caught a specific mutation. |
Sources: test/rspec_integration_test.rb:99-110, lib/mutineer/reporter.rb:294-315
The sequence shows how Mutineer drives RSpec for individual mutant verification and collects results for the final report.
Sources: test/rspec_integration_test.rb:23-38, lib/mutineer/reporter.rb:304-315
Conclusion
RSpec Integration allows Mutineer to provide deep mutation analysis for projects using the RSpec framework. By leveraging RSpec's granular example IDs and handled I/O environments, Mutineer ensures that mutation scores accurately reflect the strength of the RSpec suite. The integration identifies specific test weaknesses, such as blind or redundant examples, enabling developers to refine their testing strategy.