Plan Review Validation
Relevant source files
The following files were used as context for generating this wiki page:
Plan Review Validation is a mandatory gated stage within the Roughly development pipeline that verifies an implementation plan against the actual codebase before any code is modified. Its primary purpose is to check for technical soundness, completeness, accuracy of assumptions, and potential overengineering. By identifying issues at the planning stage, it prevents context overflow and implementation failures, which are common failure modes for long AI sessions.
Sources: skills/review-plan/SKILL.md:10-15, README.md:102-105
The validation process is handled by a dedicated review-plan subagent, which is dispatched as a blocking call by the orchestrator (either the build or fix pipeline). The subagent performs up to three automated iterations of questioning and investigation, ultimately returning a structured verdict of PASS or NEEDS REVISION.
Sources: skills/review-plan/SKILL.md:21-25, README.md:65-68
Orchestration and Dispatch
Plan Review Validation occurs at Stage 4 of both the Build Pipeline and the Fix Pipeline. It is an automated gate that must be passed before implementation can proceed.
Dispatch Flow
The orchestrator (e.g., roughly:fix) first verifies that a plan file exists and contains at least one task. It then dispatches the review-plan subagent using the sonnet model. Unlike other skills, review-plan has disable-model-invocation: true, meaning it is intended for programmatic dispatch rather than direct user interaction.
Sources: skills/fix/SKILL.md:120-130, skills/review-plan/SKILL.md:1-5, tests/fixtures/review-plan/README.md:65-70
The following diagram illustrates the sequence of dispatching a plan review within a pipeline:
Verification Dimensions
The review-plan subagent evaluates implementation plans across three core dimensions: Completeness, Assumptions, and Overengineering. For every iteration, the subagent generates 3-5 specific verification questions to guide its investigation.
Sources: skills/review-plan/SKILL.md:31-35
1. Completeness
This dimension ensures the plan fully covers the requirement and avoids "surfacing-failure traps."
- Edit Site Enumeration: Every file range or line number must appear as a separate numbered entry. Rejecting "confirm during edit" footnotes is mandatory unless "structural uniformity" is explicitly cited.
- Asymmetry Checks: Flagging cases where Acceptance Criteria (AC) verification commands search a broader scope than the enumerated file list without explicit acknowledgment.
- Co-location Hazards: Detecting
grep -Fcusage on paragraph-dense prose where multiple matches might land on the same line, resulting in incorrect counts.
Sources: skills/review-plan/SKILL.md:37-75
2. Assumptions
Validates the technical foundations of the plan against the current state of the repository.
- Signal Source Naming: Any task performing runtime detection (e.g., checking a branch name or file content) must name the specific observable signal source.
- Invariant Guards: Ensuring new invariant guards are defensive rather than introducing new structural rules that contradict existing ACs.
- Joint Satisfiability: Verifying that two ACs targeting overlapping surfaces do not create a structural impossibility (e.g., one AC forbidding modification while another requires it).
Sources: skills/review-plan/SKILL.md:95-125, agents/epic-reviewer.md:32-35
3. Overengineering
Prevents unnecessary complexity by checking if:
- Tasks could be simplified.
- Existing utilities or patterns should be reused instead of creating new abstractions.
- The plan builds for a scale that does not yet exist.
Sources: skills/review-plan/SKILL.md:130-135
Verdicts and Output
The validation concludes with a mandatory verdict block. The subagent is prohibited from pausing for human input during its investigation.
Output Structure
The final output must include:
- Iterations: Count (1 to 3).
- Verdict: Either
PASSorNEEDS REVISION. - Findings: Classified as ✅ Confirmed, ⚠️ Concern, or ❌ Blocker.
- Suggested Edits: Specific revisions mapped to Task IDs for plans that need revision.
Sources: skills/review-plan/SKILL.md:144-165
Verdict Logic Table
| Finding Type | Iteration Count | Resulting Action |
|---|---|---|
| All ✅ | Any | Stop iterating; return PASS. |
| ⚠️ or ❌ | < 3 | Generate new questions; investigate again. |
| ⚠️ or ❌ | = 3 | Produce final output; return NEEDS REVISION. |
Sources: skills/review-plan/SKILL.md:139-142
Non-Interactive Validation (--ci)
When the pipeline is invoked with the --ci flag, the Plan Review Validation behavior changes to facilitate automation.
- Build Pipeline: Stage 4 runs the review-plan subagent. If it returns
NEEDS REVISION, the orchestrator attempts an automated revision loop (up to 2 attempts) before failing. - Fix Pipeline: Stage 4 runs the review-plan subagent but exits immediately with a non-zero status on the first
NEEDS REVISIONverdict, without attempting a revision loop. - Override Protocol: In CI mode, human gates are bypassed, but the
NEEDS REVISIONverdict acts as a hard failure. In interactive mode, a human may "override" a failing verdict by explicitly typing "override."
Sources: skills/fix/SKILL.md:133-140, README.md:210-215
Conclusion
Plan Review Validation serves as the primary safeguard in the Roughly ecosystem against "hallucinated" implementation plans. By forcing a dedicated subagent to cite code evidence (file paths and line regions) for every claim made in a plan, the system ensures that the orchestrator never initiates an implementation that is fundamentally flawed or technically impossible.
Sources: skills/review-plan/SKILL.md:167-170, README.md:65-68