Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• package.json
• src/tests/e2e/_save-audio.ts
• src/tests/cartesia-speech-model.test.ts
• src/tests/google-speech-model.test.ts
• src/tests/e2e/elevenlabs.e2e.test.ts
• src/tests/save-result.test.ts

The Speech SDK employs a multi-tiered testing strategy designed to ensure reliability across a diverse range of AI speech providers. The strategy is built upon the Vitest framework and distinguishes between unit tests, which validate provider implementation logic using mocks, and end-to-end (E2E) tests, which verify integration with live provider APIs.

The testing environment is configured to run in Node.js, and the project enforces strict code standards through automated linting and formatting, ensuring that all contributions meet the quality requirements before they are merged.

Sources: AGENTS.md:16-24, package.json:69-72

Test Categorization

The project divides tests into two primary categories based on their scope and dependencies.

Unit Tests

Unit tests focus on the internal logic of individual provider implementations. These tests are located in src/__tests__/*.test.ts. They use vi.fn() to mock the global fetch function, allowing developers to verify:

• Correct construction of request URLs and HTTP methods.
• Proper mapping of SDK options to provider-specific API bodies.
• Correct handling of authentication headers (e.g., Bearer tokens, X-API-Keys).
• Accurate processing of binary audio responses and metadata.
• Error handling for non-2xx HTTP responses.

Sources: AGENTS.md:18-24, src/tests/cartesia-speech-model.test.ts:7-30, src/tests/google-speech-model.test.ts:36-55

End-to-End (E2E) Tests

E2E tests validate the full communication cycle with real AI providers. These are located in src/__tests__/e2e/*.e2e.test.ts. Unlike unit tests, E2E tests:

• Require valid API keys (e.g., OPENAI_API_KEY, ELEVENLABS_API_KEY) stored in a .env file or environment.
• Verify that the SDK can successfully generate audible audio data from a live service.
• Test features like streaming, word-level timestamps, and abort signals against production endpoints.

Sources: AGENTS.md:20-24, src/tests/e2e/elevenlabs.e2e.test.ts:13-35

Testing Architecture and Data Flow

The testing architecture relies on Vitest's global test runner. For unit tests, the SpeechProvider instance is injected with a mocked fetch implementation to intercept outgoing network requests.

Automated Commands and Workflow

The project uses pnpm to manage test execution and quality checks.

Command	Description
pnpm test	Runs unit tests using Vitest.
pnpm run test:e2e	Runs E2E tests against live provider APIs.
pnpm run typecheck	Validates TypeScript types without emitting code.
pnpm fix	Automatically formats and lints code via Biome.
pnpm check	Checks for lint issues without applying fixes.

Sources: AGENTS.md:7-14, package.json:69-74

Artifact Generation and Verification

A unique feature of the E2E suite is the ability to save generated audio and metadata for manual verification. This is controlled by the SPEECH_SDK_E2E_OUTPUT_DIR environment variable.

When this variable is set, the helper function maybeSaveResult performs the following:

1. Creates a provider-specific subdirectory (bucket).
2. Saves the raw audio file (e.g., .mp3, .wav).
3. Saves word-level timestamps as a .json file.
4. Generates and saves caption files in .srt and .vtt formats.

Test Conventions and Standards

To maintain consistency, the repository enforces several testing conventions:

• Assertions: All assertions must be inside it() or test() blocks.
• Asynchrony: Tests must use async/await rather than done callbacks.
• Restraints: Committing .only or .skip blocks is prohibited to ensure full suite coverage in CI.
• Slugification: Filenames for saved artifacts are "slugified" to remove special characters and ensure cross-platform compatibility.

Sources: AGENTS.md:55-58, src/tests/e2e/_save-audio.ts:41-44

Summary

The testing strategy for the Speech SDK provides a robust framework for validating high-stakes AI integrations. By combining mocked unit tests for implementation logic with real-world E2E tests for API compatibility, the project ensures that speech generation remains reliable across all 13 supported providers. The automated artifact saving system further assists developers in verifying the quality of the synthesized audio and the accuracy of word-level alignments.