Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
The Testing Strategy for opcode encompasses a multi-layered approach involving automated end-to-end (E2E) validation, unit test generation through autonomous agents, and specialized security audits. The strategy prioritizes technical accuracy and real-world authenticity by transitioning from mocked environments to executing actual Claude CLI commands with bypass permissions.
This strategy ensures that the integration between the Tauri backend, the React frontend, and the external Claude binary is robust. It covers critical areas such as process isolation, network connectivity, and database integrity, while maintaining a clean, production-ready codebase with zero warnings or ignored tests.
Sources: src-tauri/tests/TESTS_COMPLETE.md:1-5, src-tauri/tests/TESTS_TASK.md:1-10, CONTRIBUTING.md:46-52
Automated Backend Testing
The backend testing suite, primarily located within the Rust environment, has evolved to provide comprehensive validation of core system commands and process management.
Real-World CLI Validation
Instead of relying on mock implementations, the current suite executes actual claude commands. This approach validates the real behavior of the Claude CLI, including its output formatting and exit codes. To facilitate this in a test environment, the --dangerously-skip-permissions flag is utilized, bypassing manual approval prompts.
The workflow for a real Claude execution test is as follows:
Core Test Fixes and Optimizations
The test suite maintains a high standard of quality through the resolution of common concurrent and platform-specific issues:
- Mutex Poisoning Prevention: Replaced
std::sync::Mutexwithparking_lot::Mutexto prevent poisoning during thread panics. - Database Integrity: Added missing tables (
agents,agent_runs) and resolved foreign key constraints in the test database schema. - Process Isolation: Implemented
create_test_binary_with_depsto handlelibcdependencies during test binary generation. - Platform Awareness: Assertions are designed to handle platform-specific limitations across macOS and Linux.
Sources: src-tauri/tests/TESTS_TASK.md:12-30
Autonomous Unit Test Generation
Opcode utilizes a specialized "Unit Tests Bot" agent to automate the creation and maintenance of the test suite. This agent operates through a six-phase autonomous workflow.
Test Generation Workflow
The agent spawns specialized sub-agents to handle different aspects of the testing lifecycle:
| Phase | Agent Role | Key Responsibilities |
|---|---|---|
| 1 | Codebase Analyzer | Extract frameworks, test patterns, and directory structures. |
| 2 | Test Planner | Identify testable modules and prioritize scenarios (critical vs low). |
| 3 | Test Writer | Generate tests using the project's preferred framework and style. |
| 4 | Test Verifier | Execute suite, identify flaky tests, and measure coverage. |
| 5 | Coverage Optimizer | Generate additional tests for uncovered branches and edge cases. |
| 6 | Documentation Writer | Summarize metrics, suite structure, and CI/CD instructions. |
Sources: cc_agents/unit-tests-bot.opcode.json:15-102
Quality Criteria for Generated Tests
The autonomous system adheres to specific quality standards to ensure maintainability:
- Isolation: Each test must have a single purpose and be independent of execution order.
- Patterns: Consistent use of Arrange-Act-Assert or Given-When-Then patterns.
- Coverage Targets: Requirement of 100% coverage for critical paths and >80% overall code coverage.
- Mocking: External dependencies must be mocked appropriately to ensure determinism.
Sources: cc_agents/unit-tests-bot.opcode.json:104-124, cc_agents/unit-tests-bot.opcode.json:157-164
Security Testing Strategy (SAST)
Security testing is handled by an AI-powered Static Application Security Testing (SAST) agent that identifies vulnerabilities through contextual analysis and data flow tracking.
Vulnerability Assessment Methodology
The security scanner follows a systematic approach combining OWASP Top 10 patterns with STRIDE threat modeling.
Severity Classification
Findings are prioritized based on exploitability and business impact according to the following matrix:
| Severity | Criteria Examples |
|---|---|
| Critical | Remote Code Execution (RCE), Auth Bypass, Full Data Breach. |
| High | SQL Injection, Privilege Escalation, Sensitive Data Exposure. |
| Medium | Stored/Reflected XSS, CSRF on sensitive actions, Session issues. |
| Low | Missing security headers, weak configurations, minor logic flaws. |
Sources: cc_agents/security-scanner.opcode.json:200-225
Contributor Requirements
Contributors are required to maintain the integrity of the testing strategy by adhering to strict coding and security standards.
- Testing Requirement: New functionality must include corresponding tests.
- Linting: Rust code must pass
cargo clippyand be formatted viacargo fmt. - Validation: All inputs from the frontend must be validated on the backend.
- Database Security: Prepared statements are mandatory for all database operations to prevent injection.
- Sensitive Data: Tokens and passwords must never be logged.
Sources: CONTRIBUTING.md:46-52, CONTRIBUTING.md:65-72
Summary
The Testing Strategy for opcode integrates real-world command validation with autonomous AI agents to ensure a high level of code quality and security. By mandating 100% critical path coverage and employing specialized scanners for vulnerabilities, the project maintains a production-ready state suitable for complex AI-assisted development tasks. The transition from mocks to real Claude CLI execution marks a significant milestone in ensuring the technical accuracy of the system.
Sources: src-tauri/tests/TESTS_COMPLETE.md:44-48, cc_agents/unit-tests-bot.opcode.json:10-14