Audio Testing & Golden Fixtures
Relevant source files
The following files were used as context for generating this wiki page:
DJ Assistant utilizes a deterministic audio testing framework composed of synthetic fixtures and "golden" expected outputs. This system ensures that the library scanner, metadata extractors, and audio analysis engines remain accurate across different platforms and code changes without requiring copyrighted audio files.
The testing architecture is split into two primary corpora: the Tag Corpus, which validates metadata extraction and file deduplication, and the Analysis Corpus, which validates BPM, key detection, and track segmentation.
Sources: fixtures/README.md:1-12, crates/core/tests/fixture_manifest.rs:1-10
Corpus Architecture
The project maintains separate fixture sets generated by Python scripts to target specific system behaviors.
Tag Corpus (Metadata & Scanner)
The Tag Corpus consists of 14 synthetic files designed to test the scanner's ability to handle various formats, corrupt data, and duplicate files. A manifest.json file serves as the ground truth for these tests.
- Generation Script:
scripts/make-fixtures.py - Verification Harness:
crates/core/tests/fixture_manifest.rs - Key Capabilities: Validates ID3v2.4, Vorbis comments, and MP4 tags across WAV, FLAC, AIFF, MP3, and M4A formats.
Sources: scripts/make-fixtures.py:11-38, crates/core/tests/fixture_manifest.rs:13-18
Analysis Corpus (Golden Fixtures)
The Analysis Corpus contains longer synthetic clips (64–128 beats) that provide sufficient data for the audio analysis engine to establish tempo and key.
- Generation Script:
scripts/make-analysis-fixtures.py - Verification Harness:
crates/analysis/tests/golden.rs(TA.6) - Key Capabilities: Validates all 24 Camelot key slots and varied BPM ranges.
Sources: scripts/make-analysis-fixtures.py:3-20, crates/analysis/tests/golden.rs:1-15
Fixture Generation Flow
The generation process uses Python and numpy to synthesize audio to ensure bit-level determinism across different environments.
The generator uses a fixed seed to ensure every byte is a pure function of the script's constants. FFmpeg is invoked with -fflags +bitexact to prevent version strings from altering file hashes.
Sources: scripts/make-fixtures.py:72-87, scripts/make-fixtures.py:270-285
Analysis Verification (Golden Tests)
The Golden Fixture harness (TA.6) executes the full analysis pipeline against the Analysis Corpus. It compares results against analysis_expected.json.
| Verification Target | Acceptance Criteria | Logic / Notes |
|---|---|---|
| Analysis Status | complete or needs_review | Status derived from classify_analysis_status |
| BPM | ±2.0 BPM Delta | Tolerates slight environment variations |
| Camelot Key | Exact Match | Validates strings like "8A" or "12B" |
| Classical Key | Enharmonic Match | Folds Db minor and C# minor to the same pitch class |
| Performance (p50) | < 3s per track | Measured in release mode (P1.5) |
Sources: crates/analysis/tests/golden.rs:17-30, crates/analysis/tests/golden.rs:114-165, crates/analysis/src/pipeline.rs:43-58
Enharmonic Key Comparison
Because the manifest might use flats (e.g., Db) while the analyzer emits sharps (e.g., C#), the test harness converts keys to pitch classes (0-11) for comparison.
Scanner & Fingerprint Testing
The system validates that identical files result in the same fingerprint while distinct files remain unique. This prevents the scanner from skipping valid tracks or importing duplicates.
Duplicate Detection Rules
- Fingerprint Match: Files with identical content-addressed hashes (BLAKE3) are identified as duplicates.
- Corrupt File Resilience: Corrupt fixtures (e.g.,
truncated.wav) must not cause the scanner to error; they should fall back to filename-derived metadata. - Discovery Filtering: The scanner must skip unsupported extensions (e.g.,
.txt) and only discover files defined inSUPPORTED_EXTENSIONS.
Sources: crates/core/tests/fixture_manifest.rs:188-215, crates/core/tests/fixture_manifest.rs:222-249, scripts/make-fixtures.py:321-343
Summary of Fixture Files
| Category | File Count | Purpose |
|---|---|---|
| Tagged Audio | 7 | Validates ID3/Vorbis/MP4 tag extraction accuracy. |
| Corrupt Audio | 4 | Tests system resilience against truncated or empty files. |
| Edge Cases | 2 | Validates Unicode filenames and apostrophe handling. |
| Analysis (Full) | 50 | Provides standard data for BPM and Key detection. |
| Analysis (Short) | 7 | Used for regression testing of needs_review status. |
Sources: scripts/make-fixtures.py:11-25, scripts/make-analysis-fixtures.py:108-115
The Audio Testing and Golden Fixtures system provides a robust, non-copyrighted foundation for verifying the most critical paths of DJ Assistant. By using synthetic generation, the project ensures that any developer can reproduce a consistent testing environment without external dependencies.