Live Transcription Engine
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Live Transcription Engine is the core functional module of mausVoice, responsible for converting spoken audio into real-time text. It leverages a hybrid architecture that supports both cloud-based streaming via providers like Deepgram (nova-3) and fully local transcription using Whisper models. The engine is designed to operate globally across the desktop environment, allowing users to dictate into any focused application via a hotkey-driven interface.
The engine orchestrates audio capture, segment processing, and AI-driven cleanup. It emphasizes privacy by encrypting API keys locally using XChaCha20-Poly1305 and providing an offline mode where no network calls are made. This system follows the "Rust is the API, TypeScript is the Brain" pattern, where high-performance native capabilities are handled in Rust while orchestration logic resides in TypeScript.
Sources: llms.txt:1-55, README.md:1-50, PRODUCT.md:1-25
Architecture and Data Flow
The transcription engine follows a strict data flow from native audio capture to finalized text injection. The TypeScript layer manages state using Zustand and dictates when transcription sessions begin or end based on user hotkey events.
High-Level Components
| Component | Description |
|---|---|
| Tauri Bridge (Rust) | Handles native audio capture and system-level keyboard/UI interactions. |
| Transcription Repos | Abstractions for different providers (Local, Deepgram, Groq, AssemblyAI). |
| Zustand Store | Maintains the single source of truth for transcription state and user settings. |
| Sidecar Crates | rust_transcription sidecars handle local Whisper (GGML/ONNX) execution. |
| Sources: llms.txt:20-40, apps/desktop/README.md:30-55 |
Transcription Flow Diagram
The following diagram illustrates the sequence of events from a user pressing a hotkey to the text being pasted into an external application.
The flow ensures that audio is either streamed for real-time results or batched for local processing, depending on user configuration.
Sources: llms.txt:25-35, README.md:30-45, apps/desktop/README.md:40-50
Transcription Implementation Details
Segment Splitting and Merging
For long-form audio, the engine utilizes a BaseTranscribeAudioRepo that manages audio segmentation. This prevents memory issues and allows for concurrent processing of audio chunks. The system uses specific durations for segments and overlaps to ensure high-quality merging without losing context at the boundaries.
- Segment Duration: Typically 20-60 seconds for API-based transcription.
- Overlap Duration: A buffer (e.g., 5-8 seconds) used to detect duplicate words at segment boundaries.
- Batching: Concurrent processing of segments to improve speed.
Sources: apps/desktop/test/integration/transcription.test.ts:130-155, apps/desktop/src/repos/transcribe-audio.repo.test.ts:10-80
Local vs. API Dispatch
The system employs a repository pattern to dispatch transcription tasks. Supported providers include:
- Deepgram: Used for fast streaming transcription (
detect_language=true). - Groq: Leverages
whisper-large-v3-turbofor high-speed cloud transcription. - Local Whisper: Executes models like
whisper-tinytowhisper-large-v3directly on the CPU or GPU viarust_transcription.
AI Post-Processing and Formatting
Once the raw transcription is generated, it is passed through a post-processing layer (primarily via Groq or OpenRouter) to clean up the text.
Prompt Engineering
The engine uses localized prompts to ensure the AI responds in the correct language and adheres to a "Glossary" provided by the user's personal dictionary.
- Dictionary Integration: Words from the personal dictionary are injected into the system prompt using a
<glossary/>tag. - Tone/Style Application: Users can select "Writing Styles" (e.g., Email, Chat, Formal) which apply specific system instructions to the LLM cleanup phase.
Sources: apps/desktop/src/utils/prompt.utils.ts:10-150, llms.txt:60-75
Post-Processing Input Structure
| Variable | Description |
|---|---|
transcript | The raw text output from the STT engine. |
userName | Injected into prompts for personalized signing (e.g., emails). |
dictationLanguage | Informs the LLM of the target output language. |
tone | The specific ToneConfig (Style or Template) selected by the user. |
| Sources: apps/desktop/src/utils/prompt.utils.ts:90-115 |
Conclusion
The mausVoice Live Transcription Engine provides a robust, cross-platform solution for high-performance dictation. By abstracting transcription providers behind a unified repository interface and utilizing LLM-based post-processing, it ensures high accuracy and readability while maintaining user privacy through local encryption and offline Whisper support.
Sources: llms.txt:100-110, PRODUCT.md:40-50