Testing & QA
Relevant source files
The following files were used as context for generating this wiki page:
Testing and Quality Assurance (QA) within Clawdbot are managed through a multi-layered strategy encompassing unit tests, integration tests, live model evaluations, and automated coverage tracking. The project utilizes Vitest for TypeScript logic and Swift's native testing frameworks for macOS-specific components, enforcing strict coverage thresholds to maintain code reliability across diverse environments.
Sources: CLAUDE.md:73-83, AGENTS.md:73-83
Testing Frameworks and Tooling
Clawdbot employs different testing stacks depending on the module and platform:
- TypeScript/Node.js: The primary framework is Vitest for unit and integration testing.
- macOS/Swift: Uses native Swift testing suites for IPC and UI component validation.
- Coverage: V8 coverage is used for TypeScript, targeting a 70% threshold across lines, branches, functions, and statements.
Command Reference
| Task | Command | Description |
|---|---|---|
| Run all tests | pnpm test | Executes the Vitest suite. |
| Coverage report | pnpm test:coverage | Runs tests and generates V8 coverage metrics. |
| Live tests | pnpm test:live | Runs tests requiring real API keys (CLAWDBOT_LIVE_TEST=1). |
| Docker E2E | pnpm test:docker:onboard | Executes onboarding end-to-end tests in Docker. |
| Linting | pnpm lint | Runs oxlint for static analysis. |
Sources: CLAUDE.md:73-83, AGENTS.md:58-64
Core Testing Strategies
Unit and Colocated Testing
Tests are typically colocated with their source files using the *.test.ts naming convention. This ensures that logic changes in modules such as src/agents/system-prompt.ts or src/slack/format.ts are immediately verified by adjacent test files.
The diagram shows the relationship between source files and their colocated tests.
Sources: CLAUDE.md:74-76, src/slack/format.test.ts:1-5
Live and Environment-Specific Testing
Clawdbot distinguishes between "pure" logic tests and those requiring external environments. Live tests use CLAWDBOT_LIVE_TEST=1 or LIVE=1 to include provider-specific evaluations. Docker-based testing is used for gateway and onboarding flows to ensure consistency across deployment environments.
Sources: CLAUDE.md:78-81, AGENTS.md:78-81
Domain-Specific Validation
Agent and Tool Validation
Testing for agents includes verifying system prompt generation and tool path resolution. For example, the createClawdbotCodingTools suite validates that workspace directories are correctly resolved for file operations like read, write, and edit.
This sequence illustrates how tool path resolution is validated during testing.
Sources: src/agents/pi-tools.create-clawdbot-coding-tools.adds-claude-style-aliases-schemas-without-dropping-f.test.ts:10-25
macOS IPC and Helper Testing
The macOS application undergoes testing for IPC communication, UI metrics, and shell execution. The LowCoverageHelperTests.swift suite covers critical utility functions including:
AnyCodablepayload handling for IPC.ShellExecutortimeouts and output draining.PortGuardianlistener parsing for identifying conflicting processes on port 18789.
Sources: apps/macos/Tests/ClawdbotIPCTests/LowCoverageHelperTests.swift:10-15, apps/macos/Tests/ClawdbotIPCTests/LowCoverageHelperTests.swift:85-105
Skill and Documentation Validation
Skills must undergo a validation and packaging process using scripts/package_skill.py. This script checks:
- YAML frontmatter format and required fields (name, description).
- Directory structure (scripts, references, assets).
- Description quality and completeness.
Sources: skills/skill-creator/SKILL.md:215-230, skills/skill-creator/scripts/init_skill.py:210-225
Continuous Integration (CI) and Gating
Clawdbot uses prek install to set up pre-commit hooks that run the same checks as the CI environment. This "full gate" process includes linting, building, and testing, which must be performed locally before pushing changes or landing Pull Requests.
The flow represents the mandatory local verification gate before code is committed.
Sources: CLAUDE.md:58-61, AGENTS.md:92-95
Summary
The Testing & QA system ensures that both the core TypeScript logic and platform-specific implementations (macOS, Docker) meet performance and security standards. By combining automated coverage thresholds with live model testing and manual skill validation, the project maintains high reliability in its multi-agent and cross-channel messaging architecture.