Testing Strategies

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• Tests/AGENTS.md
• Scripts/check.sh
• Tests/SpottyDomainTests/PlaybackCommandPresentationChecks.swift
• Tests/SpottyBoundaryTests/PaginationWalkChecks.swift
• Scripts/test_playback_promotion.py
• Tests/SpottyBoundaryTests/FixtureContractChecks.swift
• Tests/SpottyBoundaryTests/VisualStyleContractChecks.swift

Testing in Spotty ensures account safety, playback correctness, and architectural integrity through a multi-layered verification gate. The project categorizes tests by their scope and proximity to the production environment, ranging from pure domain logic to concrete boundary adapters and CI-driven artifact promotion.

The primary entry point for local verification is the Scripts/check.sh script, which executes formatting checks, Rust backend tests, ABI signature validations, and Swift Testing suites.

Sources: AGENTS.md:88-92, Scripts/check.sh:1-10

Categorized Test Targets

Spotty separates tests into two distinct Swift modules to manage dependencies and link-time requirements.

SpottyDomainTests

This target covers pure business policy and domain logic. It is entirely Rust-free and does not require the playback engine at link time. It focuses on deterministic checks for parsing, state reduction, and presentation rules.

SpottyBoundaryTests

This target covers concrete SpottyCore adapters, data stores, and injected workflows. Because it validates the integration with the playback subsystem, it requires the playback archive at link time. These tests are marked with @MainActor and run without parallelism to ensure stable verification of asynchronous effects.

Sources: Tests/AGENTS.md:3-13, Scripts/check.sh:196-224

Automated Verification Pipeline

The check.sh script enforces a strict ordering of verification steps to catch failures early and preserve session safety.

Key Testing Mechanisms

C ABI and Signature Validation

The project maintains a strict contract between the Swift code and the Rust/librespot leaf. The verification process parses the C umbrella header and compares defined symbols against the static library exports using nm.

• ABI Fixture: Matches C declarations against an unchanged Rust ABI fixture using __builtin_types_compatible_p in Clang.
• Dead Export Detection: Ensures every symbol exported in the C header is actually consumed by PlaybackCore.swift.

Sources: Scripts/check.sh:82-166

Playback State Reduction

Tests for playback logic use deterministic state reduction. This validates how the system handles optimistic updates, command failures, and authoritative snapshots from the engine.

Test Case	Description	Source File
Optimistic Pause	Validates that a pause command applies paused transport and freezes timing atomically.	PlaybackCommandPresentationChecks.swift:50-70
Command Rollback	Ensures a rejected command restores the exact pre-command transport and timing.	PlaybackCommandPresentationChecks.swift:72-85
Lagging Snapshots	Verifies that engine snapshots with older timestamps cannot undo optimistic updates.	PlaybackCommandPresentationChecks.swift:230-245

Fixture and Protocol Contracts

Boundary tests use synthetic fixtures to validate JSON decoding and API shapes without live network access.

• Fixture Loading: Decodes responses for search results, albums, artists, and home sections to ensure the data model matches the backend schema.
• Pagination Walks: Uses a ScriptedOffsetTransport to simulate multi-page fetches, verifying that the system walk ends correctly on empty pages or when overshooting totals.

Sources: Tests/SpottyBoundaryTests/FixtureContractChecks.swift:9-45, Tests/SpottyBoundaryTests/PaginationWalkChecks.swift:10-45

CI Candidate Promotion

The project uses a Python-based promotion system to validate CI-built artifacts before they are published. This ensures that only reviewed, successful builds reach a release state.

Testing Principles

• Deterministic Execution: Tests must catch behavioral failures rather than implementation details or regex snapshots.
• Main Actor Isolation: Boundary tests must run on the @MainActor to prevent race conditions during async polling.
• Safety First: Verification must never silently fall back to self-signed identities or disturb authenticated sessions unless explicitly authorized.

Sources: Tests/AGENTS.md:6-13, AGENTS.md:38-44, Scripts/check.sh:230-245