ONNX Runtime Integration
Relevant source files
The following files were used as context for generating this wiki page:
The ONNX Runtime integration provides the core machine learning inference engine for the DJ Assistant audio analysis pipeline. This system executes locally on the user's machine to compute track metadata including BPM, musical key, beat grids, and phrase boundaries without transmitting raw audio data to the cloud.
The integration relies on the ort Rust crate to manage platform-aware dynamic library discovery and execution. It facilitates a privacy-first architecture where audio decoding via Symphonia and model inference occur entirely within the desktop application's Rust core.
Sources: README.md:167-178, crates/analysis/Cargo.toml:8-10
Architecture and Components
The ONNX integration is housed within the dj-assistant-analysis crate. It interfaces between decoded PCM audio data and specialized machine learning models. The system uses a versioned ModelRegistry to manage ModelSpec definitions, ensuring that the correct model weights are loaded and verified before execution.
The diagram shows the flow from raw audio decoding to ONNX inference within the analysis crate.
Sources: CHANGELOG.md:112-120, README.md:210-220
Dependency Configuration
The project utilizes specific features of the ort crate to support cross-platform compatibility and dynamic loading.
| Feature | Description |
|---|---|
load-dynamic | Enables platform-aware dynamic dylib discovery. |
ndarray | Provides the tensor data structures required for model input/output. |
std | Standard library support for the runtime. |
Sources: crates/analysis/Cargo.toml:9
Model Verification and Safety
To ensure the integrity of the analysis pipeline, the system implements a SHA-256 verification step for all model assets. Before a session initializes, the registry verifies the model file against a pinned hash. If the hash does not match, or if the path is invalid (e.g., attempting to escape the models directory using ..), the loader fails immediately.
A specialized "smoke test" model is included in the codebase to verify the runtime environment without requiring heavy production weights. This 61-byte model performs a Relu operation (y = Relu(x)). This specific operator was chosen because it is supported even by operator-trimmed runtimes, such as the one bundled with Microsoft Edge WebView2 on Windows.
Sources: CHANGELOG.md:87-95, CHANGELOG.md:121-128, scripts/make-smoke-model.py:15-28
Smoke Model Specification
The smoke test model is hand-encoded in Protobuf to avoid large dependencies like the onnx Python wheel.
| Attribute | Value |
|---|---|
| Operator | Relu |
| Input | x (float tensor, shape [1]) |
| Output | y (float tensor, shape [1]) |
| SHA-256 | 5bd717dd17f1a40364178ea8aff2017dabc1221b13b681cd8d1a6809af1b5196 |
| Size | 61 bytes |
Sources: scripts/make-smoke-model.py:44-50, CHANGELOG.md:121-128
Model Discovery Logic
The analysis engine resolves model paths using a prioritized discovery strategy. This allows developers and users to configure model locations via environment variables or standard directory structures.
The flow represents the precedence order for locating ONNX model assets.
Sources: CHANGELOG.md:87-94, README.md:253-256
Integration with Desktop Core
The ONNX Runtime is wired into the dj-assistant-desktop Tauri application. The Rust core manages the analysis_version string, which aggregates pipeline version stamps and model versions. This allows the system to determine if a track needs re-analysis when models are updated or improved.
The desktop application's Cargo.toml explicitly includes the analysis crate and configures the ort runtime for the platform's native TLS and dynamic loading requirements.
Sources: apps/desktop/src-tauri/Cargo.toml:19-22, CHANGELOG.md:107-111
Conclusion
The ONNX Runtime integration is a critical component that enables local, high-performance audio analysis in DJ Assistant. By utilizing a deterministic registry and strict checksum verification, the system maintains high confidence in analysis results while protecting user privacy through entirely offline inference.