Stability Contract

Relevant source files

The following files were used as context for generating this wiki page:

• STABILITY.md
• rake/docs_contract.rb
• test/docs_contract_test.rb
• rake/site_docs.rb
• lib/mutineer/reporter.rb
• rake/site_build.rb

The Stability Contract defines the compatibility promises Mutineer makes to its users. It ensures that automated systems, such as CI/CD pipelines and AI coding agents, can rely on stable interfaces across minor version releases. Mutineer follows Semantic Versioning, where breaking changes are reserved for major version updates.

Sources: STABILITY.md:1-6

Contract Enforcement Architecture

Mutineer uses a "single-source" strategy to manage documentation and contract surfaces. A central YAML configuration file contains the canonical definitions for exit codes and descriptions. The DocsContract module splices these definitions into various files to prevent drift.

The DocsContract module acts as a generator that ensures consistency across all user-facing documentation and configuration files.
Sources: rake/docs_contract.rb:7-46, test/docs_contract_test.rb:11-20

SemVer Coverage

Mutineer promises that a minor release will not remove or change the meaning of established interfaces. A minor release may add new flags, keys, or inputs.

Category	Coverage Scope	Current Stable Identifier
CLI flags	All flags in mutineer --help	Version 1.6.0
Exit codes	Standard codes 0, 1, and 2	Version 1.6.0
JSON schema	Stability of keys and meanings	schema_version 1.8
Mutant ids	Consistency of ID generation	summary.id_format 2
Config keys	Valid keys in .mutineer.yml	Version 1.6.0
Action inputs	Input parameters in action.yml	Version 1.6.0

Sources: STABILITY.md:8-18, lib/mutineer/reporter.rb:23

Exit Codes

The exit code contract is critical for CI/CD integration. The Reporter class calculates these codes based on the mutation score and the presence of errors or regressions.

Code	Meaning
0	Score ≥ threshold (or no gate) and no baseline regression.
1	Score below --threshold, OR nothing could be scored and something broke, OR more than one mutant produced no verdict exceeding 10% of attempts, OR a --baseline regression, OR a runtime error.
2	Usage or invalid-flag error (e.g., mistyped flag, bad path, unreadable baseline).

Sources: STABILITY.md:25-30, lib/mutineer/reporter.rb:104-121

Completeness Gate Logic

The Reporter implements a "broken share limit" to prevent misleading success in unstable environments. If more than 10% of attempted mutants (or more than 1 floor mutant) produce no verdict (errors, timeouts, or uncapturable), the run fails even if the surviving mutants meet the threshold.

JSON Report Schema

The JSON report represents a machine-readable contract for AI agents and automation. The schema version is tracked independently and follows specific rules for breaking changes.

• Stable Versioning: The schema_version remains at 1.x for non-breaking additions.
• Id Format: The id_format: 2 indicates that mutant IDs include project-relative file paths for stability across renames or moves.
• Additive Changes: New keys like matrix or baseline are added without bumping the major version.

Sources: STABILITY.md:14-22, lib/mutineer/reporter.rb:141-205, rake/docs_contract.rb:112-114

Configuration and CI Integration

Mutineer maintains a stable list of configuration keys and GitHub Action inputs.

Supported Configuration Keys

The following keys are guaranteed to be supported in .mutineer.yml for the current major version:
operators, jobs, threshold, only, require, boot, rails, since, framework, verbose, ignore, baseline, fail_fast, matrix, test_command, daemon, timeout, capture_timeout, cache_dir.

Sources: STABILITY.md:37-41

GitHub Action Inputs

The GitHub Action supports these inputs:
sources, test, since, threshold, baseline, baseline-epsilon, operators, framework, strategy, jobs, rails, format, output, extra-args, working-directory, use-bundler, version.

Sources: STABILITY.md:48-52

Drift Detection and Site Build

To maintain the Stability Contract, the project includes automated tests and Rake tasks that verify documentation consistency.

1. Stale File Check: MutineerSiteDocs.stale_files identifies committed files that do not match the current contract definitions.
2. Site Build: The SiteBuild module assembles the published documentation, ensuring the production-ready section from README.md is accurately spliced into the homepage.
3. Alternate Twins: Every HTML page in the documentation has a corresponding Markdown twin to support LLM ingestion.

Sources: rake/docs_contract.rb:120-132, rake/site_build.rb:71-92, rake/site_docs.rb:30-46

Planned Changes for 2.0

Mutineer documents future breaking changes to allow users to prepare for major updates.

• Strict Validation: Unknown configuration keys and operator names will trigger errors instead of warnings.
• Empty Scans: Full scans that find zero mutants will result in failure.
• Exit Code 3: A new exit code will be introduced specifically for "untrustworthy" runs where the environment or harness is compromised.
• Default Parallelism: Rails integration will default to parallel execution using the reload strategy.

Sources: STABILITY.md:54-91

The Stability Contract serves as the foundation for Mutineer's reliability in automated environments, balancing the need for continuous improvement with the requirement for stable integration interfaces.