Model Store & Provisioning
Relevant source files
The following files were used as context for generating this wiki page:
The Model Store & Provisioning system manages the lifecycle of speech and filler detection models within Crisp. Since these models (such as ggml-base.en.bin for Whisper) are large (~148 MB), they are not bundled with the application to avoid bloating build sizes and update downloads. Instead, the system handles on-demand fetching, integrity verification, and local storage management.
The system supports two primary model types: Speech Models (driven by whisper.cpp) and Filler Models (the custom-trained Wren models using Core ML). It provides a robust infrastructure for downloading these assets from remote repositories like Hugging Face, ensuring they are ready for use by the cleaning engine.
Sources: CLAUDE.md, research/NOTES.md
Architecture and Components
The provisioning logic is split into several layers, separating the model definitions (Catalog) from the storage and retrieval logic (Store).
Model Catalog
The ModelCatalog defines the available models and their metadata. Each model is represented by a ModelSpec containing the download URL, expected SHA-256 hash, and file size. This allows the app to verify the integrity of the downloaded file before marking it as "Ready."
Model Store
The ModelStore is an @Observable service that tracks the local state of a model. It determines if a model is currently missing, downloading, or ready for use. It handles the transition between these states and interacts with the file system to locate installed models.
Chunked Downloader
To handle large model files reliably, Crisp uses a ChunkedDownloader. This component supports:
- HTTP-Range Resume: If a download is interrupted, it can resume from the last successful byte.
- SHA-256 Verification: Validates the file after completion.
- Atomic Publish: Downloads to a temporary location and moves the file to the final destination only after successful verification.
Sources: CLAUDE.md, research/NOTES.md, research/filler_classifier/publish_hf.py
Model Provisioning Flow
The provisioning flow ensures that the cleaning engine is gated until necessary models are available on disk.
The diagram above shows the lifecycle of a model from discovery to readiness.
Sources: CLAUDE.md, research/NOTES.md
Custom Filler Models (Wren)
Crisp uses custom-trained models named Wren for fast filler detection. These models are managed through a specialized version of the Model Store that supports different release channels and developer sideloading.
Release Channels and Manifests
Filler models follow the app's release flow. Stable builds poll the main branch of the Hugging Face model repository, while Nightly and Dev builds poll the nightly branch for staging models.
| Component | Description | Source |
|---|---|---|
| Model Branch | main for Stable, nightly for Nightly/Dev | FillerModelDevFlowTests.swift |
| Wren v1 | Original per-chunk classifier (v0.0.x) | research/NOTES.md |
| Wren v2 | Context-aware sequence TCN model | research/NOTES.md |
| Sideloading | Developer ability to load .mlmodel from disk | research/NOTES.md |
Versioning Logic
Model versions are derived from the commit count of the Hugging Face repository (0.0.N). This ensures monotonic versioning that mirrors the app's own scheme.
Sources: research/filler_classifier/publish_hf.py, research/NOTES.md, apps/desktop/Tests/CrispTests/FillerModelDevFlowTests.swift
Technical Implementation Details
Model Verification and Storage
Models are stored in the channel-specific data directory (~/.crisp*/models/). The app derives the model state from disk at each launch to self-heal from corrupted or interrupted downloads.
// Example ModelSpec structure derived from ModelCatalog.swift and publish_hf.py
struct ModelSpec {
let id: String
let url: URL
let sha256: String
let approxBytes: Int64
}
Sources: CLAUDE.md, research/filler_classifier/publish_hf.py
Developer Dev-Flow Seams
For development, Crisp includes "seams" to override the standard provisioning logic.
- Dev Sideload: Allows running a local
.mlmodelfile directly from disk using a path picker in Settings. - Version History: A specialized
FillerModelVersionstool (Dev build only) can list Hugging Face tags and install specific historical versions for A/B testing.
Sources: apps/desktop/Tests/CrispTests/FillerModelDevFlowTests.swift, research/NOTES.md
Summary
Model Store & Provisioning is a critical infrastructure component that balances app size and functionality. By leveraging a robust chunked downloader with SHA-256 verification and atomic publishing, Crisp ensures that heavy ML assets are delivered reliably to the user's Mac. The system's extensibility allows for seamless updates of both standard speech models and experimental custom filler detection models.