Vocal Presence & Loudness
Relevant source files
The following files were used as context for generating this wiki page:
Vocal Presence & Loudness represents a core part of the local on-device analysis pipeline in DJ Assistant. This system computes a track's vocal density and EBU R128 loudness metrics to assist DJs in set sequencing and transition planning. The analysis runs entirely offline using Rust-based signal processing and ONNX Runtime inference.
Sources: README.md:95-99, CHANGELOG.md:104-106
Vocal Presence Estimation
The vocal presence system estimates the density of vocals throughout an audio track. It produces a time-series presence curve and a single track-level aggregate score between 0.0 and 1.0.
Analysis Logic
The system uses a deterministic spectral heuristic to detect vocals when an external ONNX model is not present. The process follows these steps:
- The analyzer converts decoded audio to a mono "mid" signal and a "side" signal
(L-R)/2. - It performs Short-Time Fourier Transform (STFT) passes using a Hann window.
- It calculates the vocal-band (300–4000 Hz) energy ratio of the mid signal relative to all energy above 150 Hz.
- It applies a center-dominance gate; since lead vocals usually sit in the center, wide-panned signals deflate the presence score.
- It applies a median-smoothing window to the resulting curve to prevent transient spikes from affecting the data.
Sources: crates/analysis/src/vocal.rs:1-24, crates/analysis/src/vocal.rs:103-125
Implementation Structures
| Structure | Description |
|---|---|
VocalPoint | Represents a single point on the curve with a timestamp (start_seconds) and a presence value (0.0–1.0). |
VocalProfile | Contains the complete VocalPoint vector, the aggregate presence mean, the VocalMethod, and a version stamp. |
VocalOptions | Configures sample_rate, frame_size (default 4096), hop_size (default 2048), and vocal_band_hz. |
Sources: crates/analysis/src/vocal.rs:40-90
Data Flow for Vocal Analysis
The following diagram illustrates how raw audio samples transform into a vocal profile.
The flow ensures that only centered, frequency-relevant signals contribute to the final vocal density score.
Sources: crates/analysis/src/vocal.rs:96-146
Loudness Measurement (EBU R128)
Loudness analysis follows the BS.1770-4 standard to provide consistent volume metrics across different tracks. This ensures that DJs can normalize their library and avoid unexpected volume jumps during a performance.
Measurement Components
The system measures three primary metrics:
- Integrated Loudness: The gated average loudness of the entire track in LUFS.
- True Peak (dBTP): The maximum absolute level of the signal waveform.
- Loudness Range (LRA): A measure of the variation in loudness over time, as defined in EBU Tech 3342.
The analysis is performed on the raw PCM data using the ebur128 crate. It handles multi-channel audio by performing a BS.1770 channel-power sum rather than a simple mono downmix to maintain accuracy.
Sources: CHANGELOG.md:104-113, README.md:120-123
Technical Specifications
Data Model Integration
Loudness and vocal presence data are stored in the local SQLite database and surfaced through the library interface.
| Field | Type | Description |
|---|---|---|
vocal_presence | REAL | Aggregate track-level vocal density (0.0–1.0). |
loudness_lufs | REAL | EBU R128 integrated loudness. |
loudness_range | REAL | The measured LRA of the track. |
analysis_version | TEXT | Stable version string (e.g., vocal.1.0.0) to trigger re-analysis if logic changes. |
Sources: crates/analysis/src/vocal.rs:31-33, CHANGELOG.md:115-119, crates/core/src/metadata.rs:37-43
Versioning and Re-analysis
The analysis_version string aggregates per-analyzer pipeline version stamps. If the VOCAL_PIPELINE_VERSION is bumped in the code, the system identifies that existing database records are stale and queues them for re-analysis.
Sources: crates/analysis/src/vocal.rs:31-33, CHANGELOG.md:115-119
User Interface Surface
In the desktop application, these metrics contribute to the "Library Health" and track details. While raw LUFS are used for normalization, vocal presence is utilized by the Beam Search optimizer to avoid vocal overlaps during automated sequencing.
The sequence shows the transition from raw track metadata to analyzed features visible in the user interface.
Sources: README.md:146-168, apps/desktop/src/lib/library.ts:258-264
The Vocal Presence and Loudness modules provide the quantitative data necessary for DJ Assistant to perform intelligent set building. By using deterministic signal processing alongside optional ONNX models, the system maintains a high standard of accuracy for track structure and volume consistency.
Sources: README.md:20-25, crates/analysis/src/vocal.rs:26-30