Testing & Quality Assurance

Relevant source files

The following files were used as context for generating this wiki page:

• apps/desktop/test/integration/transcription.test.ts
• apps/desktop/test/evals/post-processing-eval.test.ts
• apps/desktop/package.json
• apps/desktop/README.md
• CLAUDE.md
• llms.txt
• apps/desktop/src/components/microphone/MicrophoneTester.tsx

Testing and Quality Assurance (QA) in mausVoice is a multi-tiered system designed to ensure the reliability of local and cloud-based transcription, the accuracy of AI-driven post-processing, and the stability of the cross-platform desktop application. The project employs a combination of unit tests, integration tests for hardware and API interactions, and specialized LLM evaluations ("evals") to measure the quality of cleaned-up text.

The QA infrastructure is integrated into the pnpm-based monorepo, utilizing Turborepo for efficient task execution. Key tools include Vitest for unit and integration testing, WebDriverIO for E2E smoke tests, and custom evaluation helpers for validating LLM outputs against specific writing styles.

Sources: llms.txt:75-81, apps/desktop/README.md:20-27, CLAUDE.md:6-10

Testing Infrastructure and Tooling

The project defines a standard set of quality commands accessible from the repository root. These commands enforce code style, type safety, and functional correctness across all workspaces, including shared packages and the desktop application.

Core Quality Commands

Command	Tool	Purpose
pnpm run build	Turborepo	Builds all workspaces and shared packages.
pnpm run lint	Prettier / oxlint	Enforces code formatting and identifies static errors.
pnpm run check-types	TypeScript (tsc)	Validates type safety across the monorepo.
pnpm run test	Vitest	Executes the full suite of unit, integration, and eval tests.

Sources: llms.txt:75-81, apps/desktop/package.json:11-30, apps/desktop/README.md:20-27

Test Categories in apps/desktop

Testing within the desktop application is categorized by scope and execution environment:

• Unit Tests (test:unit): Focused on individual React components and utility functions.
• Integration Tests (test:integration): Validates the interaction between TypeScript business logic, local repositories, and external APIs (e.g., Groq transcription).
• Evaluations (test:evals): Specialized tests that use LLMs to evaluate the quality and tone of processed transcriptions.
• E2E Smoke Tests (test:webdriver): Uses tauri-driver and WebDriverIO to perform high-level application testing.

Sources: apps/desktop/package.json:26-30, apps/desktop/README.md:20-27

LLM Evaluations (Evals)

Evals are a critical part of the mausVoice quality pipeline, specifically targeting the "AI cleanup" feature. These tests verify that the system correctly removes filler words, applies formatting, and maintains the requested writing style (e.g., Casual, Formal, Email).

Evaluation Architecture

The evaluation system uses a runPostProcessingEval helper to automate the flow from raw transcript to final output and then subjects the result to a list of criteria ("evals").

Key components include:

• Prompt Builders: Functions like buildSystemPostProcessingTonePrompt and buildPostProcessingPrompt create the instructions for the LLM.
• Schema Validation: Uses Zod (PROCESSED_TRANSCRIPTION_SCHEMA) to ensure the LLM returns structured JSON.
• Tone Configurations: Tests are run against various styles defined in ToneConfig.

Sources: apps/desktop/test/evals/post-processing-eval.test.ts:16-55

Evaluated Writing Styles

The system validates multiple writing styles to ensure consistency:

• Default: Removes filler words, handles emojis, and fixes self-corrections while preserving colloquialisms.
• Email: Ensures professional formatting with greetings, body, and sign-offs.
• Chat: Formats text to read like a casual message (concise, lowercase-friendly).
• Formal: Rewrites casual speech into a professional register without contractions.

Sources: apps/desktop/test/evals/post-processing-eval.test.ts:98-420

Integration Testing

Integration tests verify the application's core capabilities, such as audio transcription and hardware interaction.

Transcription Integration

Tests in test/integration/transcription.test.ts simulate the full transcription pipeline by feeding WAV files into the repository layer. This validates:

1. Audio Parsing: Correct conversion of 16-bit PCM WAV buffers to Float32 samples.
2. Segment Splitting: The ability to split long audio (e.g., 44s) into overlapping segments for processing.
3. Accuracy: Comparison of the result against a ground-truth text file using string similarity algorithms.

Hardware Verification (Microphone Tester)

The MicrophoneTester component provides a functional "test" mode within the UI, allowing users to verify their audio setup. It involves:

• Live Waveform Visualization: Visualizing audioLevels from the Zustand store during recording.
• Recording Capture: Invoking Tauri commands (start_recording, stop_recording) to capture samples from the hardware.
• Playback Validation: Building a temporary WAV file via buildWaveFile and generating a blob: URL for the user to hear the recorded audio.

Sources: apps/desktop/src/components/microphone/MicrophoneTester.tsx:103-185

Quality Assurance Best Practices

The project enforces specific development rules to maintain code quality:

• Root Cause Resolution: No "band-aid" fixes; architectural issues must be addressed directly.
• DRY Principles: Shared logic must be refactored into reusable modules (e.g., in packages/utilities).
• I18n Compliance: All user-facing strings must use <FormattedMessage /> to ensure they are extractable for the translation catalogs.
• Binding Generation: When Rust commands are modified, pnpm gen:bindings must be run to update the TypeScript wrappers.

Sources: CLAUDE.md:3-15, AGENTS.md:3-15, apps/desktop/package.json:20-22

Summary

Testing and Quality Assurance in mausVoice leverages a robust automated pipeline that spans from low-level unit tests to high-level AI output evaluations. By combining traditional testing methods with modern LLM evaluation techniques, the project ensures that its core value proposition—fast, accurate, and high-quality voice-to-text—is maintained across updates and supported platforms.