STT Engine Orchestration

Relevant source files

The following files were used as context for generating this wiki page:

• Sources/MacParakeetCore/STT/STTScheduler.swift
• Sources/MacParakeetCore/STT/STTRuntime.swift
• Sources/MacParakeetCore/STT/README.md
• spec/README.md
• spec/06-stt-engine.md
• plans/completed/parakeet-unified-engine.md
• plans/active/nemotron-english-streaming-variant.md

The Speech-to-Text (STT) Engine Orchestration system in MacParakeet is a centralized control plane designed to manage local inference on Apple Silicon. It provides a unified interface for multiple STT engines, including Parakeet (TDT and Unified), Nemotron (Multilingual and English Beta), and WhisperKit. The architecture ensures that high-priority interactive tasks like system-wide dictation remain responsive even during long-running background transcription or meeting recording sessions.

This orchestration layer is governed by a two-slot scheduler that manages admission control, priority queuing, and engine routing. By abstracting the complexities of model lifecycles, memory management on the Apple Neural Engine (ANE), and concurrent session handling, the system provides a robust foundation for all voice-driven features in the application.

Sources: Sources/MacParakeetCore/STT/README.md, spec/README.md

Core Architecture and Components

The orchestration logic is divided between two primary actors: the STTScheduler and the STTRuntime. This separation of concerns ensures that job scheduling is decoupled from the physical management of machine learning models.

STTScheduler

The STTScheduler acts as the public broker for all STT work. It handles job admission, manages two execution slots (Interactive and Background), and enforces priority rules. It also manages "Engine Leases" for active meetings to prevent configuration changes mid-session.
Sources: Sources/MacParakeetCore/STT/STTScheduler.swift:1-30, Sources/MacParakeetCore/STT/README.md

STTRuntime

The STTRuntime is the sole owner of the underlying STT engines. It manages the lifecycle of AsrManager instances (Parakeet), ParakeetUnifiedEngine, NemotronEngine, and WhisperEngine. It handles model initialization (warm-up), cache management, and build swaps (e.g., switching between Parakeet v2 and v3).
Sources: Sources/MacParakeetCore/STT/STTRuntime.swift:1-50, Sources/MacParakeetCore/STT/README.md

Scheduling and Priority Management

The system utilizes two distinct execution slots to maintain application responsiveness. The scheduling logic prevents resource contention on the Neural Engine.

Execution Slots

1. Interactive Slot: Reserved exclusively for dictation. This ensures low latency for real-time user input.
2. Background Slot: Shared by meetingFinalize, meetingLiveChunk, and fileTranscription.
   Sources: Sources/MacParakeetCore/STT/README.md, Sources/MacParakeetCore/STT/STTScheduler.swift

Job Priority Hierarchy

If the background slot is occupied, incoming jobs are queued or admitted based on the following priority:

1. Meeting Finalize: Highest background priority.
2. Meeting Live Chunk: Medium priority.
3. File Transcription: Lowest priority.
   Sources: Sources/MacParakeetCore/STT/README.md

Engine Leases

When a meeting recording begins, the STTScheduler issues an engine lease. This locks the active engine and language configuration to ensure a single meeting transcript is consistent. Attempts to switch engines via Settings or CLI are blocked while a lease is active.
Sources: Sources/MacParakeetCore/STT/STTScheduler.swift:450-480, Sources/MacParakeetCore/STT/README.md

Engine Routing Logic

The STTRuntime routes transcription requests based on the persisted SpeechEngineSelection and model variants.

Engine	Variants	Primary Use Case
Parakeet	TDT v3 (Default), TDT v2, Unified	System default; optimized for ANE
Nemotron	3.5 Multilingual, English Beta	Opt-in Beta; broad multilingual coverage
WhisperKit	Large v3 Turbo, etc.	Fallback for languages not covered by others

Sources: Sources/MacParakeetCore/STT/README.md, spec/06-stt-engine.md, plans/completed/parakeet-unified-engine.md

Native Live Dictation

Engines like Parakeet Unified and Nemotron support native streaming. For these engines, the STTRuntime routes live dictation through a NativeLiveDictating protocol. This allows for real-time partial transcripts while the audio is being captured, without waiting for the final batch transcription.
Sources: Sources/MacParakeetCore/STT/README.md, plans/active/nemotron-english-streaming-variant.md

Model Lifecycle and Management

The STTRuntime handles the heavy lifting of model management to ensure efficient use of system resources.

Warm-up and Initialization

The runtime supports a backgroundWarmUp() method that loads models into memory before they are needed. This reduces the "first-dictation" latency. The onboarding flow triggers this automatically.
Sources: Sources/MacParakeetCore/STT/STTRuntime.swift:238-250, Sources/MacParakeetCore/STT/README.md

Variant Swapping

Switching between variants (e.g., Parakeet v2 to v3) requires the runtime to:

1. Check for engine busy states.
2. Unload the current model.
3. Load/Download the new model variant.
4. Handle failures by restoring the previous stable state.
   Sources: Sources/MacParakeetCore/STT/STTRuntime.swift:600-650, plans/active/nemotron-english-streaming-variant.md

Data Structure: STTResult

Every transcription job returns a standardized STTResult object containing:

• text: The transcribed string.
• words: Token-level timestamps (if supported by the engine).
• language: The detected or forced language code.
• engine: The SpeechEnginePreference used.
• engineVariant: The specific model variant identifier.
  Sources: Sources/MacParakeetCore/STT/README.md, plans/completed/2026-04-engine-attribution-polish.md

Conclusion

The STT Engine Orchestration layer is a critical component of MacParakeet’s performance on Apple Silicon. By centralizing model management in the STTRuntime and task prioritization in the STTScheduler, the system provides a seamless experience for both real-time dictation and complex background transcription tasks. The architecture’s support for multiple local engines and native streaming variants ensures flexibility as new on-device speech models emerge.