Pre-Merge Checks & Testing
Relevant source files
The following files were used as context for generating this wiki page:
Opificia AI employs a multi-layered verification system to ensure that AI-generated changes meet repository standards before they are merged. The system combines deterministic hygiene checks, automated test execution within sandboxes, and human-in-the-loop sign-offs to maintain code quality and security.
Sources: AGENTS.md, CONTRIBUTING.md, apps/server/src/worker/effects.service.ts:167-175
Automated Verification Flow
The verification process begins after an agent completes its implementation task. The EffectsService manages the transition from implementation to checking.
The diagram above shows the sequence of events from implementation completion to pull request opening.
Sources: apps/server/src/worker/effects.service.ts:133-228, apps/server/src/github/git.ts:75-115
Deterministic Hygiene Checks
The checker.ts module performs a suite of deterministic checks on the ChangeSummary produced by the agent. These checks verify that the agent followed security and repository policies.
Core Validation Rules
| Rule | Description |
|---|---|
protected_path | Prevents changes to sensitive files like CI workflows or policy files. |
secret | Detects potential API keys (e.g., sk-ant-) or private keys in added lines. |
hidden_unicode | Identifies dangerous characters like Right-to-Left Override (U+202E) used for obfuscation. |
size | Enforces limits on the number of files changed and lines added per file. |
commit | Ensures only the factory's bot account is the author/committer. |
history | Prevents merge commits or changes not based on the target branch. |
submodule | Detects the addition of git submodules. |
Sources: apps/server/src/checks/checker.ts, apps/server/src/checks/checker.test.ts:32-108
Verification Logic
The system validates that the agent does not include human Signed-off-by lines, as sign-offs are restricted to humans. It also checks for "tag characters" (U+E0000 block) that could be used to smuggle hidden text.
Sources: apps/server/src/checks/checker.test.ts:40-45, apps/server/src/checks/checker.test.ts:98-103
Testing Architecture
Testing occurs in two distinct environments: the agent sandbox and the server-side verification.
Sandbox Testing
Agents are instructed to run tests within their local sandbox environment before submitting their work.
- Read relevant code and existing tests.
- Implement changes and add new tests.
- Execute the test suite (e.g.,
pnpm test). - Fix any failures before replying with the final PR description.
Sources: plugins/harness-claude/src/prompts.ts:102-111, AGENTS.md:65-71
Server-Side Validation
The server uses loadResult to extract the agent's changes from a git bundle. It then invokes checkChange with DEFAULT_LIMITS and DEFAULT_PROTECTED_PATHS.
The sequence diagram illustrates how the EffectsService coordinates the loading and checking of agent results.
Sources: apps/server/src/worker/effects.service.ts:133-175, apps/server/src/github/git.ts:75-115
User Interface & Visibility
The web application displays the status of these checks to human reviewers.
Checks Card
The ChecksCard component in the web UI summarizes the verification results. It displays:
- Status: A "Checks passed" or "Checks failed" header with corresponding icons (
ShieldCheckorShieldAlert). - Stats: Number of files changed, additions, and deletions.
- Findings: A detailed list of failures, mapping rule IDs to human-readable labels and icons (e.g.,
Lockfor protected paths).
Sources: apps/web/src/run/conversation.tsx:162-195
Commit & PR Titles
Pull request titles must follow Conventional Commits (e.g., feat(server): ...). The conventionalTitle function ensures that agent-proposed subjects are formatted correctly before the PR is opened on GitHub.
Sources: CONTRIBUTING.md:162-175, apps/server/src/worker/effects.service.ts:219-224
Human Sign-off (DCO)
Final validation is performed by humans through the Developer Certificate of Origin (DCO) process.
- Agent Restriction: Agents NEVER add
Signed-off-bylines. - Maintainer Approval: Maintainers approve the PR on GitHub.
- Sign-off by Approval: If a maintainer includes
/signoffin their approving review, a dedicated workflow adds theSigned-off-byline to the commits and pushes them.
Sources: AGENTS.md:32-35, CONTRIBUTING.md:103-120
Summary
The Pre-Merge Checks & Testing system ensures that every AI-generated contribution is hygienic, safe, and aligned with repository policies. By automating the detection of secrets, obfuscation, and unauthorized file changes, Opificia maintains a secure software supply chain while requiring human approval for final integration.