Testing Framework Overview

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• test/test_api.py
• test/test_main.py
• test/test_database.py
• test/test_crawler_extended.py
• test/test_data_processor.py
• test/test_config.py
• test/test_logger.py

The testing framework for the Spotify API Data Collector is a robust suite designed to validate the entire data pipeline, from API integration and web services to database persistence and configuration. It utilizes pytest as the primary test runner and leverages extensive mocking to simulate external Spotify API responses and filesystem operations.

The suite ensures the reliability of the system's core features, including the FastAPI-based web interface, the asynchronous data crawler, and the SQLite storage layer. It includes unit tests for individual components and integration tests that verify the interaction between services using specialized fixtures and temporary environments.

Sources: README.md:144-156, test/test_api.py:11-30

Core Testing Components

The testing architecture is divided into several specialized modules that target specific layers of the application.

API and Web Service Testing

API testing is performed using the FastAPI TestClient. It verifies endpoint availability, request filtering, and response payloads. The framework includes fixtures to create temporary database states and CSV files to simulate different data scenarios, such as empty results or populated datasets.

This diagram illustrates the flow of an API test where the TestClient interacts with the FastAPI application, which in turn queries mocked data sources to return a response for assertion.

Sources: test/test_api.py:15-104, test/test_api_extended.py:15-40

Service and Logic Testing

The framework tests the business logic found in the crawler, data_processor, and spotify_client. These tests rely heavily on unittest.mock to bypass actual network calls to Spotify, allowing for deterministic testing of pagination logic, rate limiting, and data enrichment.

Component	Test Focus	Primary Tools
Crawler	Pagination, filtering top tracks, artist enrichment	Mock, patch
Data Processor	Batch insertions, CSV export logic, progress logging	MagicMock, tempfile
Main Orchestrator	Service initialization, configuration validation flow	patch

Sources: test/test_crawler_extended.py:28-100, test/test_data_processor.py:65-120, test/test_main.py:15-45

Test Environment & Fixtures

The framework uses Pytest fixtures to maintain clean and isolated test environments. This includes the automatic creation and teardown of temporary databases and configuration overrides.

Database Isolation

To prevent tests from affecting production data, the temp_db fixture creates a unique SQLite database file using tempfile, initializes the schema, and ensures the file is deleted after the test execution completes.

The sequence shows the lifecycle of a database test, ensuring environment isolation through temporary file management.

Sources: test/test_database.py:19-35, test/test_db.py:11-23

Mocking Strategy

The project employs a comprehensive mocking strategy to handle the Spotify API and configuration.

• API Mocks: spotipy.Spotify and SpotifyClientCredentials are patched to return predefined JSON items.
• Config Mocks: app.config.Config attributes are patched to test different environment scenarios without modifying actual .env files.
• Log Mocks: The logging system is verified to ensure it captures relevant events without cluttering the console during test runs.

Sources: test/test_spotify_client.py:17-25, test/test_config.py:26-45, test/test_logger.py:12-25

Test Coverage and Execution

The project supports coverage reporting to ensure all critical paths are exercised. Tests are organized by module within the test/ directory.

Execution Commands

Developers can execute specific test subsets or the full suite with coverage as follows:

# Run all tests with verbosity
python3.11 -m pytest test/ -v

# Run only API tests
python3.11 -m pytest test/test_api.py -v

# Generate coverage report for app and service modules
python3.11 -m pytest test/ --cov=app --cov=service --cov-report=term-missing

Sources: README.md:148-156

Summary of Validated Features

Feature	Validation Method	Key Test File
Database Schema	Table existence and index creation verification	test/test_database.py
Data Models	Track creation and dictionary serialization	test/test_track.py
Idempotence	Duplicate track insertion rejection	test/test_database.py
API Filters	Year, popularity, and artist name query parameters	test/test_api.py
Config Validation	Missing .env variables handling	test/test_config.py

The testing framework provides high confidence in the data collector's ability to process Spotify metadata accurately while maintaining a clean, isolated environment for continuous integration.

Sources: test/test_database.py:73-85, test/test_api.py:108-125, test/test_track.py:12-35