Testing Strategies

Relevant source files

The following files were used as context for generating this wiki page:

• test/router.test.ts
• test/run-context.test.ts
• AGENTS.md
• package.json
• src/surface/slack.ts
• src/main.ts

Introduction

Testing in the pi-digby project is designed to ensure the reliability of bot interactions, event routing, and message lifecycle management. The strategy focuses on automated unit testing using the Vitest framework, combined with strict linting and type checking to maintain high code quality standards.

The testing suite specifically targets core components like the SlackSurface, which manages message updates and streaming footers, and the Router, which handles Slack event distribution based on channel types and message content. These tests rely heavily on mock implementations of external transports (e.g., Slack clients) to simulate complex interaction scenarios without requiring live API connections.

Sources: test/run-context.test.ts, test/router.test.ts, package.json:28

Unit Testing Framework

The project utilizes vitest as its primary test runner and tsgo for type checking. Testing is integrated into the development lifecycle through a pre-commit hook that enforces code quality checks.

Core Testing Configuration

Tool	Purpose	Command
Vitest	Main test runner for unit and integration tests.	npm run test
Biome	Linting and formatting enforcement.	npm run check
tsgo	Fast TypeScript type checking without full emission.	npx tsgo --noEmit

Sources: package.json:8-11, AGENTS.md:7-13

Component Testing Strategies

Surface and Message Lifecycle Testing

The SlackSurface component is tested to ensure that message transitions (thinking, progress, response, and terminal states) are handled correctly and serialized to prevent race conditions. These tests use a MessageTransport mock to record and verify the sequence of Slack API calls.

Key scenarios tested include:

• Footer Rendering: Verifying that the footer correctly displays step counts and transitions from "streaming" to actual dollar costs upon resolution.
• Idempotency: Ensuring that multiple calls to terminal operations like resolve() or reject() do not result in duplicate Slack updates.
• Error Handling: Validating that reject() correctly appends error indicators to the existing message content.
• Suppression: Confirming that suppress() deletes both the main message and any threaded detail messages.

Sources: test/run-context.test.ts:46-130, src/surface/slack.ts:133-176

Event Router Testing

The Router tests focus on the logic for processing different Slack event types (DMs, mentions, and channel messages). This ensures the bot only responds to valid triggers while correctly logging other activity.

Testing coverage includes:

• Trigger Logic: Verifying that runs are triggered for DMs and App Mentions, but ignored for the bot's own messages.
• Always-on Channels: Testing that messages in specifically configured channels trigger runs even without direct mentions.
• Busy State: Ensuring the router handles requests when a run is already in progress by posting "Still thinking" messages.
• Mention Stripping: Validating that bot user IDs are removed from the text before being passed to the handler.

Sources: test/router.test.ts:63-230, src/main.ts:168-185

Development Rules and Verification

The project enforces strict development rules to ensure that code changes do not break existing functionality. Developers are required to run npm run check before committing code, which combines Biome linting and TypeScript type checking.

• No any types: The use of any is restricted to maintain type safety.
• Conventional Commits: Standardized commit messages (e.g., feat:, fix:) are used for clarity.
• Strict Mode: TypeScript is configured in strict mode to catch potential errors early.

Sources: AGENTS.md:3-17, biome.json

Conclusion

The testing strategy for pi-digby provides comprehensive coverage of the bot's communication and routing layers. By leveraging Vitest for modular unit tests and strict CI/CD checks via Biome and tsgo, the project ensures that message flows remain consistent and that interaction logic is protected against regressions.