Testing Strategy & Vitest
Relevant source files
The following files were used as context for generating this wiki page:
The project employs a robust testing strategy centered on Test-Driven Development (TDD) and high-fidelity mocking to validate complex asynchronous background tasks. The primary test runner is Vitest, which is used to execute unit and integration tests across the codebase, particularly for Trigger.dev task definitions, sandbox provisioning logic, and AI content generation pipelines.
This strategy ensures that the system's core components—including coding agents, media generation via fal.ai, and GitHub repository management—function correctly before deployment. Developers and coding agents are instructed to follow a strict Red-Green-Refactor cycle for all changes.
Sources: AGENTS.md:38-44, CLAUDE.md:38-44
Core Testing Principles
The repository enforces a "Red-Green-Refactor" TDD workflow. This approach requires writing a failing test that reproduces a bug or describes new behavior before any implementation code is written.
| Principle | Description |
|---|---|
| Red | Write a failing test first to define the expected behavior. |
| Green | Write the minimum code required to make the test pass. |
| Refactor | Clean up the code while ensuring all tests remain passing. |
| Bug Regression | For production bugs, the test case must be derived from the exact error message in logs. |
Sources: AGENTS.md:38-44, CLAUDE.md:38-44
Vitest Implementation & Mocking Strategy
The project relies heavily on vi.mock to isolate units under test, especially when dealing with external SDKs like @trigger.dev/sdk/v3, @fal-ai/client, and Vercel Sandboxes.
Mocking External Services
Testing Trigger.dev tasks requires mocking the schemaTask and task wrappers to extract and manually execute the run function. This allows for asserting that specific metadata, tags, and sub-tasks are triggered with correct payloads.
The diagram shows the isolation of tests from external side effects using Vitest's mocking capabilities.
Sources: src/tasks/tests/createContentTask.test.ts:7-20, src/tasks/tests/codingAgentTask.test.ts:5-15
Key Mocking Patterns
- Trigger.dev SDK: Mocking
tags.add,metadata.set, andschemaTaskto track execution flow. - Sandbox Environments: Mocking
runCommandandSandbox.createto verify terminal commands without spawning real virtual environments. - Media APIs: Mocking
fal.storage.uploadand generation functions to return static URLs during testing.
Sources: src/tasks/tests/codingAgentTask.test.ts:17-45, src/sandboxes/tests/runClaudeCodeAgent.test.ts:25-30
Content Pipeline Testing
The content generation system uses a dual-layered testing approach: automated unit tests for logic and a local CLI-based pipeline tester for manual verification of AI-generated media.
Local Pipeline Tester
Located at src/content/testPipeline.ts, this utility allows developers to run individual steps of the content creation process (image generation, upscaling, audio selection, ffmpeg rendering) without invoking the full Trigger.dev infrastructure. It maintains state in a .pipeline-state.json file.
The local tester facilitates iterative development by persisting progress between generation steps.
Sources: src/content/testPipeline.ts:1-25, src/content/testPipeline.ts:316-335
Automated Task Validation
Tests for background tasks (e.g., codingAgentTask, setupSandboxTask) focus on the orchestration of multiple steps.
Sequence Verification
Tests ensure that operations occur in the correct order, such as configuring Git authentication before running an AI coding agent, or stopping a sandbox in a finally block to prevent resource leaks.
This sequence ensures that environment preparation always precedes agent execution and cleanup always follows.
Sources: src/tasks/tests/codingAgentTask.test.ts:114-123, src/tasks/tests/updatePRTask.test.ts:113-116
Summary of Test Utilities
| Utility File | Purpose |
|---|---|
src/content/testPipeline.ts | Local CLI tool for step-by-step media generation testing. |
src/schemas/__tests__/ | Zod schema validation to ensure input/output contracts. |
src/sandboxes/__tests__/ | Verification of shell command execution and sandbox state. |
src/tasks/__tests__/ | Integration tests for Trigger.dev background workers. |
Sources: src/content/testPipeline.ts:12-20, src/schemas/tests/contentCreationSchema.test.ts:4-15, src/sandboxes/tests/provisionSandbox.test.ts:47-59
The testing strategy within the tasks repository prioritizes deterministic results for non-deterministic AI processes through extensive mocking and clear lifecycle management of sandbox environments. Significant emphasis is placed on ensuring that every logical path—including failure modes like missing API keys or missing files—is covered by the Vitest suite.
Sources: src/tasks/tests/createContentTask.test.ts:110-120, src/sandboxes/tests/runClaudeCodeAgent.test.ts:153-165