Installation and Setup Guide
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Spotify API Data Collector is a modular system designed to harvest track metadata from the Spotify Web API, store it in a local SQLite database, and expose it via a FastAPI web service. This guide provides the necessary steps to configure the environment, authenticate with Spotify, and deploy the services for data collection and analysis.
Sources: README.md:1-12
Environment Setup and Installation
Prerequisites
The system requires Python 3.11. Dependencies are managed via a requirements.txt file, which includes essential libraries such as fastapi, uvicorn, spotipy, and pandas.
Sources: README.md:46-51, requirements.txt
Quick Installation
To set up the local development environment, follow these steps:
- Navigate to the project directory.
- Create and activate a virtual environment.
- Install the required dependencies.
cd /Users/amadou/Desktop/test_technique_spotify_api
python3.11 -m venv .venv
source .venv/bin/activate
python3.11 -m pip install -r requirements.txt
Sources: README.md:46-51
Configuration and Authentication
Spotify API Credentials
The application uses the Client Credentials Flow for authentication. Users must obtain credentials from the Spotify Developer Dashboard.
Sources: README.md:65-68, service/spotify_client.py:27-31
Environment Variables
Configuration is centralized in the Config class, which loads values from a .env file located at the project root.
| Variable | Description | Default Value |
|---|---|---|
SPOTIPY_CLIENT_ID | Spotify API Client ID (Required) | None |
SPOTIPY_CLIENT_SECRET | Spotify API Client Secret (Required) | None |
DATABASE_URL | SQLAlchemy-style database URL | sqlite:///spotify_data.db |
TRACKS_PER_YEAR | Number of tracks to search per year | 1000 |
TOP_N_TRACKS | Number of top tracks to keep after filtering | 200 |
API_RATE_LIMIT_DELAY | Delay between general API calls (seconds) | 0.1 |
ARTIST_FETCH_DELAY | Delay between artist enrichment calls (seconds) | 0.05 |
Sources: app/config.py:10-25, README.md:55-63
Validation Logic
The system includes a validation method to ensure that critical credentials (SPOTIPY_CLIENT_ID and SPOTIPY_CLIENT_SECRET) are present before execution starts.
Sources: app/config.py:27-34, app/main.py:23-31
System Architecture and Execution
The project is composed of three main operational phases: Data Collection, Storage/Cleaning, and API Exposure.
The diagram shows the flow from configuration loading to the final API consumption by the user.
Sources: app/main.py:16-43, app/api.py:22-30
Launching Data Collection
The collection process iterates through the years 2020-2024, searching for tracks and enriching them with artist-specific data (followers, genres).
python3.11 -m app.main
Sources: app/main.py:1-55, README.md:71-78
Running the API
The API is built with FastAPI and can be served using Uvicorn. The database tables are automatically initialized during the API startup event.
python3.11 -m uvicorn app.api:app --host 0.0.0.0 --port 8000 --reload
Sources: app/api.py:32-38, README.md:80-84
Data Export and Maintenance
Exporting to CSV
For external analysis, a dedicated script allows exporting the SQLite database content into a CSV file.
python3.11 scripts/export_csv.py -o custom_filename.csv
Sources: scripts/export_csv.py:17-106, README.md:120-124
Data Cleaning
A cleaning script is provided to remove test data and limit entries to the top 200 tracks per year based on popularity.
Sources: scripts/Clean_finale.py:10-34
Conclusion
The installation and setup of the Spotify API Data Collector require a valid Python environment and Spotify API credentials. Once the .env file is configured, the system automates the orchestration of the crawler, database management, and the FastAPI server, providing a robust pipeline for musical data analysis.
Sources: README.md:14-25, app/main.py:33-43