On-Device STT Models
Relevant source files
The following files were used as context for generating this wiki page:
MacParakeet utilizes a private, local-first architecture for Speech-to-Text (STT) processing on Apple Silicon Macs. The system is designed to run entirely on-device, primarily leveraging the Apple Neural Engine (ANE) via CoreML to ensure data privacy and high-performance execution.
The project supports a variety of local engines, with the Parakeet TDT 0.6B family as the primary default. This architecture allows for diverse recognition capabilities, including multilingual support through Parakeet v3, English-only optimization via Parakeet v2 or Unified models, and optional engines like Nemotron and WhisperKit for extended language coverage or experimental features.
Sources: AGENTS.md, spec/06-stt-engine.md:1-10
Core Engine Architecture
MacParakeet manages different STT models through a unified protocol layer and a centralized runtime scheduler. This ensures that only one high-intensity ML task occupies the hardware at a time, while providing specialized engines for different job types such as dictation, meeting recording, and file transcription.
Engine Comparison Table
| Engine | Model Type | Primary Use Case | Key Features |
|---|---|---|---|
| Parakeet v3 | TDT 0.6B | Default / Multilingual | 25 European languages, 155x realtime |
| Parakeet v2 | TDT 0.6B | English-only | Faster for English, no auto-detect issues |
| Parakeet Unified | FastConformer-RNNT | English-only | Native streaming, built-in punctuation |
| Nemotron | 3.5 ASR Streaming | Beta / Multilingual | Streaming dictation partials, local Beta |
| WhisperKit | Large-v3 Turbo | Extended Multilingual | Broad coverage (CJK, etc.), mature fallback |
Sources: spec/06-stt-engine.md:16-105, plans/completed/parakeet-unified-engine.md:46-55
Runtime Scheduling
The system employs a two-slot scheduler:
- Interactive Slot: Reserved for dictation to ensure immediate responsiveness.
- Background Slot: Shared by meeting finalization, live chunking, and file transcription.
This scheduling prevents background batch jobs from interfering with low-latency interactive dictation.
Sources: spec/06-stt-engine.md:231-255
Parakeet Engine Family
The Parakeet family is the project's primary STT stack, running on the ANE via the FluidAudio SDK.
Parakeet Unified Engine
The ParakeetUnifiedEngine is a specialized actor that wraps FluidAudio's UnifiedAsrManager for batch processing and StreamingUnifiedAsrManager for live dictation. Unlike standard Parakeet TDT models, the Unified model (FastConformer-RNNT) provides native streaming with punctuation and capitalization but does not emit word-level timestamps.
The ParakeetUnifiedEngine manages two lanes (interactive and background) to prevent internal backpressure during concurrent tasks.
Sources: Sources/MacParakeetCore/STT/ParakeetUnifiedEngine.swift:15-45, plans/completed/parakeet-unified-engine.md:73-85
Optional and Beta Engines
Nemotron Engine
Nemotron is surfaced as an opt-in Beta engine for both multilingual and English-only builds. It is a streaming engine, allowing MacParakeet to display live partials in the dictation overlay while the user is speaking.
- Multilingual Build: Approx. 1.5 GB download, supports ~40 languages.
- English Build: Approx. 600 MB download, optimized for English only.
The NemotronEngine actor coordinates between interactive and background managers, handling resampled 16kHz mono Float32 audio buffers.
Sources: Sources/MacParakeetCore/STT/NemotronEngine.swift:20-60, spec/06-stt-engine.md:65-85
WhisperKit Engine
WhisperKit serves as a mature fallback for languages not covered by Parakeet (such as CJK languages). It utilizes the WhisperEngine actor to wrap the WhisperKit runtime. It is capable of detecting languages and providing word timestamps when supported by the specific model variant.
Sources: Sources/MacParakeetCore/STT/WhisperEngine.swift:18-40, spec/06-stt-engine.md:95-105
Model Lifecycle and Management
All on-device models follow a strict lifecycle managed by the STTRuntime.
Initialization and Loading Sequence
Models are loaded lazily on first request or during an explicit warm-up.
Cache and Storage
Models are stored in ~/Library/Application Support/MacParakeet/models/ or under FluidAudio specific paths:
- Parakeet TDT: ~465 MB per build.
- Nemotron: 600 MB - 1.5 GB.
- Whisper: ~632 MB (Turbo variant).
Sources: spec/06-stt-engine.md:280-310
Performance and Resource Usage
On-device execution is optimized to minimize CPU and RAM impact while maximizing throughput on the Neural Engine.
| Metric | Parakeet TDT | Parakeet Unified | Nemotron |
|---|---|---|---|
| RAM usage | ~66 MB | ~565 MB (int8) | ~600 MB - 1.5 GB |
| Latency | < 100ms (dictation) | 2.08s (streaming) | 1.12s (streaming) |
| Throughput | 155x realtime | Competitive | 65x realtime (M5 Pro) |
Sources: spec/06-stt-engine.md:180-200, plans/completed/parakeet-unified-engine.md:28-40
The use of 4-bit and 8-bit quantization (such as DWQ or int8) allows large models to fit within the limited memory overhead of a background macOS application.