Model Cache Management
Relevant source files
The following files were used as context for generating this wiki page:
Model Cache Management in MacParakeet oversees the lifecycle of local machine learning models used for Speech-to-Text (STT) and Large Language Model (LLM) tasks. The system ensures that multi-gigabyte model files—such as those for Parakeet, Nemotron, Whisper, and MLX-based LLMs—are downloaded on-demand, verified, and stored efficiently within the application's support directories. This local-first posture eliminates reliance on cloud APIs for core transcription while managing the significant disk footprint of these assets.
The management logic is distributed between STTClient, STTRuntime, and various service-specific managers like InProcessModelDownloader. It handles specific hardware requirements, such as the Apple Neural Engine (ANE) on Apple Silicon Macs, and provides user controls for clearing caches to reclaim disk space.
Sources: spec/README.md, plans/completed/2026-03-fluidaudio-0.12.3-upgrade.md, plans/active/2026-06-27-on-device-local-llm.md
Architecture and Storage
MacParakeet utilizes specific directory structures for different model families. Most STT models are managed via the FluidAudio framework, while LLM models use a dedicated MLX-based structure.
Cache Directory Structure
Models are stored in ~/Library/Application Support/MacParakeet/ subdirectories:
| Model Family | Storage Path |
|---|---|
| Parakeet / Nemotron | ~/Library/Application Support/FluidAudio/Models/ |
| Whisper | ~/Library/Application Support/MacParakeet/models/stt/whisper/ |
| Local LLM (MLX) | ~/Library/Application Support/MacParakeet/LLMModels/<id>/ |
Sources: plans/completed/2026-03-fluidaudio-0.12.3-upgrade.md, spec/README.md, plans/active/2026-06-27-on-device-local-llm.md, plans/active/nemotron-english-streaming-variant.md
Component Relationship Diagram
This diagram shows how the system orchestrates model loading and cache verification across different engines.
The STTRuntime acts as the central coordinator for engine switching and cache verification.
Sources: plans/active/nemotron-english-streaming-variant.md, plans/completed/parakeet-unified-engine.md
Model Lifecycle Operations
The system supports a standard set of operations for managing model assets: verification, download with progress tracking, and destructive cleanup.
Cache Verification
Before an engine is initialized, the system performs a cache check. For example, the NemotronEnglishEngine requires both a metadata.json and a specific encoder_int8.mlmodelc to be present to avoid re-downloading.
Sources: plans/active/nemotron-english-streaming-variant.md
Download and Progress Tracking
Download operations utilize asynchronous progress handlers. In version 0.12.3, the system migrated from polling the filesystem to using byte-level progress callbacks provided by the underlying framework.
Progress updates are throttled to approximately 4 updates per second (250ms interval) to prevent UI churn.
Sources: plans/completed/2026-03-fluidaudio-0.12.3-upgrade.md
Cache Cleanup
Users can reclaim space via the "Delete Models" button in Settings. This operation:
- Shuts down active STT engines to release file handles.
- Invokes
DownloadUtils.clearAllModelCaches()to remove ASR, VAD, and Diarization models. - Resets the
modelsReadystate in theSTTClient.
Sources: plans/completed/2026-03-fluidaudio-0.12.3-upgrade.md
Engine-Specific Implementations
Each engine manages its variants and cache logic slightly differently based on hardware requirements and model sizes.
Parakeet and Nemotron
The Parakeet engine supports multiple variants (v2, v3, and Unified). Nemotron variants include Multilingual and English-only builds.
- Parakeet Unified (English): A 600M parameter model (~565 MB int8) that runs on the CPU and ANE.
- Nemotron English Streaming: Specifically tuned for 1120ms chunk sizes, requiring approximately 600 MB of space.
Sources: plans/active/nemotron-english-streaming-variant.md, plans/completed/parakeet-unified-engine.md
Local LLM (MLX)
Model management for local LLMs involves "one-click" setup paths. The system targets specific Apple Silicon RAM tiers:
- 16 GB RAM: Recommends 4B parameter models (e.g., Qwen3-4B).
- 32 GB+ RAM: Supports MoE (Mixture of Experts) models (~18.6 GB).
Sources: plans/active/2026-06-27-on-device-local-llm.md
Summary of Configuration Options
| Option | Configuration Key | Logic |
|---|---|---|
| Engine Selection | speechEnginePreference | Switches between Parakeet, Nemotron, and Whisper. |
| Nemotron Variant | nemotronModelVariant | Toggles between Multilingual and English-only (~600MB) builds. |
| Parakeet Variant | parakeetModelVariant | Toggles between v2, v3, and Unified models. |
| Whisper Language | whisperDefaultLanguage | Determines which language-specific Whisper assets to cache. |
Sources: plans/active/2026-06-15-settings-engine-characterization-tests.md, plans/active/nemotron-english-streaming-variant.md, Sources/MacParakeetCore/STT/WhisperLanguageCatalog.swift
Technical Accuracy and Hardware Limits
On macOS, model cache management must respect the Metal wired-memory limits, which are approximately 67% for machines with ≤36 GB of RAM and 75% for those with >36 GB. The system performs "RAM-gated" recommendations to ensure that downloaded models can actually be loaded into memory without causing system instability.
Sources: plans/active/2026-06-27-on-device-local-llm.md
The Model Cache Management system is a critical infrastructure component that enables MacParakeet's private, local-first operation. By centralizing the orchestration of multi-gigabyte downloads, providing throttled progress feedback, and enforcing hardware-aware memory gates, it ensures high-performance transcription and AI capabilities on Apple Silicon while giving users control over their local storage.