Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
The MacParakeet testing strategy is designed to ensure regression safety and validate behavior at service boundaries while maintaining high development velocity. The project follows the philosophy of writing tests that are deterministic, fast, and produce clear error messages, prioritizing integration tests over exhaustive unit coverage. The strategy spans multiple layers, from pure logic unit tests to complex integration flows involving AI services, speech-to-text (STT) scheduling, and database persistence.
Sources: spec/09-testing.md:1-8, AGENTS.md:88-92
Test Categories and Architecture
The project categorizes tests based on the scope of the components being verified. This layered approach ensures that failures are isolated to specific domains.
Unit and Logic Testing
Unit tests focus on pure logic, models, and data transformations. These tests use XCTest and are strictly decoupled from external dependencies like the network or database. Key areas covered include transcript word merging, text processing stages (capitalization, punctuation), and dictation stop-decision logic.
Sources: spec/09-testing.md:12-23, spec/09-testing.md:278-285
Database Testing
Database tests verify CRUD operations, queries, and migrations. To ensure speed and isolation, these tests utilize in-memory SQLite via GRDB. Each test creates a fresh, isolated database environment.
Sources: spec/09-testing.md:25-45, spec/09-testing.md:268-276
Integration and Flow Testing
Integration tests validate service boundaries and multi-component workflows. MacParakeet relies heavily on protocol-based dependency injection (DI) to inject mock implementations of complex services like transcription or LLM providers.
This diagram illustrates the separation between production dependencies and the mock-based integration testing architecture.
Sources: spec/09-testing.md:47-66, spec/09-testing.md:287-295
Characterization Testing
Specialized characterization tests are used to "pin" existing behavior before major refactors. These tests capture current observable states, such as settings selections and state machine transitions, to ensure that architectural changes do not introduce silent regressions in complex view models.
Sources: plans/active/2026-06-15-settings-engine-characterization-tests.md:12-25
Specialized Testing Domains
Certain features require dedicated testing strategies due to their complexity or hardware dependencies.
| Domain | Focus Area | Verification Method |
|---|---|---|
| STT Scheduler | ADR-016 scheduling, priority, backpressure | Protocol mocks for STT runtime; execution order assertions. |
| Meeting Recording | State transitions (idle → recording → stopping) | Mocks for MeetingAudioCapturing and MeetingMicrophoneCapturing. |
| LLM / Transforms | Prompt construction, local-only metadata | In-memory SQLite; protocol mocks for LLM provider calls. |
| CLI | Command parsing, machine-readable JSON output | CLITests target; throwaway database paths for smoke runs. |
| Onboarding | First-run sequences, model download status | OnboardingViewModelTests walking the step list. |
Sources: spec/09-testing.md:79-138, plans/active/2026-06-15-settings-engine-characterization-tests.md:78-95
Developer and Agent Workflow
The testing strategy is integrated into the daily development loop for both human contributors and AI coding agents.
The "Inner Loop"
Agents are encouraged to use a focused "inner loop" for rapid iteration. This is facilitated by the scripts/dev/check.sh script, which performs a debug build and runs filtered tests.
This diagram shows the sequence of automated checks performed during active development.
Sources: scripts/dev/check.sh:1-15, spec/10-ai-coding-method.md:74-88
Verification Protocol
Before declaring work complete, the following standard verification commands are used:
- Full Suite:
swift test(Deterministic; ~1-2 minutes). - Parallel Run:
swift test --parallel(To reduce wall-clock time). - Focused Filter:
swift test --filter <ModuleName>. - Local CI:
scripts/dev/ci_local.sh.
Sources: AGENTS.md:28-36, spec/09-testing.md:154-165, spec/10-ai-coding-method.md:80-88
Test Quality and Exclusions
To maintain a fast and reliable suite, certain types of tests are explicitly avoided.
Excluded Areas
- SwiftUI View Tests: Considered brittle and low value; replaced by testing ViewModels and state logic.
- Hardware-Dependent Audio: Replaced by testing processing logic with fixture data (e.g., short WAV clips in
Tests/Fixtures/). - Flaky Tests: Intermittent failures are fixed or deleted rather than retried.
Sources: spec/09-testing.md:143-152, spec/09-testing.md:200-205
Quality Rules
All tests must be:
- Deterministic: No
sleep()or random data without fixed seeds. - Fast: Individual tests should run in under 1 second.
- Isolated: No network calls; everything external is mocked.
- Clear: Names describe scenarios, e.g.,
testImportVTTCreatesMemoryWithCorrectTimestamps.
Sources: spec/09-testing.md:183-198, plans/completed/2026-04-tech-debt-audit.md:214-225
Summary
MacParakeet's testing strategy balances rigorous verification of core logic with a pragmatic approach to UI and hardware testing. By leveraging in-memory databases and protocol-based mocking, the project ensures that the most critical paths—dictation, transcription, and meeting recording—remain stable across architectural changes. The integration of scripts like check.sh into the developer workflow ensures that tests remain a proactive tool rather than a late-stage hurdle.
Sources: spec/09-testing.md:1-8, spec/10-ai-coding-method.md:129-135