Model Lifecycle & Registry
Relevant source files
The following files were used as context for generating this wiki page:
The Model Lifecycle & Registry system manages the discovery, registration, and state transitions of AI models within the RunAnywhere SDK. The native C++ commons core serves as the source of truth for model paths, registry state, and storage management across all platform SDKs (iOS, Android, Web, React Native, and Flutter). Sources: bindings/react-native/Docs/Documentation.md, core/README.md
This system ensures that models are correctly identified by their framework (e.g., llama.cpp, ONNX, MLX) and categorized by modality (e.g., Language, Vision, Speech Synthesis). It handles the transition of models from remote catalog entries to locally resident artifacts available for inference. Sources: idl/model_types.proto, core/AGENTS.md
Model Registry Architecture
The Model Registry maintains a durable catalog of model entries, persisted via .rac-manifest.binpb files. It tracks whether a model is registered, downloading, or loaded in memory. Sources: idl/model_types.proto:11-13, idl/model_types.proto:177-185
Registry State Machine
Models progress through defined statuses as they move from discovery to readiness for inference.
Sources: idl/model_types.proto:177-185
Key Data Structures
The registry uses Protobuf IDL schemas to define model metadata.
| Message | Description |
|---|---|
ModelInfo | The primary metadata container for a model, including ID, category, framework, and local paths. |
ModelArtifactType | Defines the bundle kind, such as SINGLE_FILE, ARCHIVE, or MULTI_FILE. |
InferenceFramework | Identifies the engine executing the model (e.g., LLAMA_CPP, ONNX, QHEXRT). |
ModelCategory | Classifies the model by task (e.g., LANGUAGE, VISION, SPEECH_RECOGNITION). |
Sources: idl/model_types.proto:44-59, idl/model_types.proto:65-88, idl/model_types.proto:265-288
Lifecycle Management
The Lifecycle Manager in the C++ core manages the resident models in memory. It tracks reference counts to prevent a model from being unloaded while an inference task is active. Sources: core/AGENTS.md:143-152
Model Loading Sequence
Initialization follows a two-phase process across all SDKs.
Loading Constraints
- Atomic Cancel: An
atomic<bool>flag allows for canceling generation without deadlocking the generating thread. Sources: core/AGENTS.md:146-147 - Refcount Pinning: The core uses
rac_lifecycle_acquire_serviceandrelease_serviceto manage memory residency. Sources: core/AGENTS.md:148-150 - Offline Operation: Phase 2 errors (like network failures) are non-fatal; the registry still operates using discovered local models. Sources: bindings/react-native/Docs/ARCHITECTURE.md:54-55
Model Artifacts and Storage
The SDK supports various artifact types, which determines how they are stored and verified on disk.
Artifact Types
- Single File: A direct model file download (e.g.,
.ggufor.onnx). - Archive: A compressed bundle (
ZIP,TAR_GZ) that requires extraction. - Multi-File: A collection of related files (e.g., a primary model plus a vision projector or tokenizer).
- Built-in: System-provided models (e.g., Apple Foundation Models) that do not require a download.
Sources: idl/model_types.proto:162-175, bindings/swift/Sources/RunAnywhere/CRACommons/include/rac_model_types.h:112-118
Storage Pathing
The C++ core resolves paths following a specific hierarchy:
{base_dir}/RunAnywhere/Models/{framework}/{modelId}/
Sources: core/AGENTS.md:155-156
Registry Operations API
The registry provides a set of proto-byte ABI calls for managing model state.
| Operation | Input Message | Result Message |
|---|---|---|
| Refresh | ModelRegistryRefreshRequest | ModelRegistryRefreshResult |
| List | ModelListRequest | ModelListResult |
| Import | ModelImportRequest | ModelImportResult |
| Discovery | ModelDiscoveryRequest | ModelDiscoveryResult |
| Compatibility | ModelCompatibilityRequest | ModelCompatibilityResult |
Sources: idl/model_types.proto:441-455, idl/model_types.proto:492-511, idl/model_types.proto:522-532, idl/model_types.proto:591-611
Compatibility Verification
The ModelCompatibilityRequest evaluates if a model can be run or stored based on device resources. It compares the model's memory_required_bytes and download_size_bytes against the device's available_ram_bytes and available_storage_bytes. Sources: idl/model_types.proto:591-611, bindings/swift/Sources/RunAnywhere/CRACommons/include/rac_model_types.h:594-604
Summary
The Model Lifecycle & Registry system centralizes model management in the C++ core to ensure consistency across platforms. By using a unified Protobuf-based ABI and a robust state machine, it manages the entire journey of a model from initial discovery to active inference. The system prioritizes local-first execution while providing hooks for remote catalog updates and hardware compatibility checks. Sources: core/README.md, bindings/react-native/Docs/ARCHITECTURE.md