PyPI Packaging & Deployment
Relevant source files
The following files were used as context for generating this wiki page:
The PyPI Packaging and Deployment system for the hyperdrive project automates the process of preparing, versioning, and distributing the algorithmic trading library to the Python Package Index. It integrates GitHub Releases for version management and provides utilities for maintaining up-to-date dependency requirements.
Sources: setup.py:1-50, README.md:15-30
Packaging Configuration
The project uses setuptools to define the package metadata and distribution parameters. The configuration is centralized in setup.py, which dynamically gathers information such as the version number and project requirements.
Metadata and Dependencies
The package is named hyperdrive and is described as "An algorithmic trading platform." It requires Python 3.7 or higher and relies on a list of external libraries parsed from a requirements.txt file.
| Parameter | Value / Source | Description |
|---|---|---|
name | hyperdrive | The distribution name on PyPI |
python_requires | >=3.7 | Minimum supported Python version |
install_requires | requirements.txt | List of package dependencies |
author | Krish Suchak | Project maintainer |
packages | find_packages() | Automatically discovers package modules |
Sources: setup.py:34-50, README.md:25-28
Versioning Logic
Versions are not hardcoded but are retrieved from the GitHub API. The get_version() function queries the latest release tag from the repository and strips the 'v' prefix to conform to PyPI versioning standards.
def get_version():
url = 'https://api.github.com/repos/suchak1/hyperdrive/releases/latest'
token = os.environ.get('GITHUB')
headers = {'Authorization': f'token {token}'}
response = requests.get(url, headers=headers if token else None)
data = response.json()
version = data['tag_name'].replace('v', '')
return version
Sources: setup.py:11-20
Dependency Management & Updates
The project includes a utility script, util/update.py, designed to keep requirements.txt synchronized with the latest versions of dependencies available on PyPI.
Automated Upgrade Flow
This script performs a look-up for every package defined in the requirements file against the PyPI JSON API. It identifies the latest stable release and, if a newer version exists, updates the local environment via pip and overwrites the requirements file with the new version string.
The following flowchart illustrates the dependency update logic:
The system specifically avoids updating certain packages (e.g., scikit-learn) because version changes might cause unpredictable behavior in trained models.
Sources: util/update.py:1-40
Deployment Environment
Deployment behavior is influenced by environment variables, primarily used to distinguish between local development and Continuous Integration (CI) environments.
| Variable | Source | Purpose |
|---|---|---|
CI | os.environ | Determines if the script is running in a GitHub Action; influences python command choice (python vs python3) |
GITHUB | os.environ | Personal Access Token for authenticating GitHub API requests to avoid rate limits |
DEV | Constants.py | Boolean flag for development mode |
Sources: hyperdrive/Constants.py:14-18, util/update.py:24-26, setup.py:13
Repository and Storage Maintenance
To ensure a clean deployment state, scripts/update_repo.py handles the cleanup of the S3 storage bucket. It preserves data and models while removing stale deployment artifacts before uploading the current project state.
Sources: scripts/update_repo.py:1-12
Summary
The PyPI packaging and deployment infrastructure for hyperdrive relies on dynamic version retrieval from GitHub and automated dependency auditing. By integrating metadata from setup.py with the cleanup routines in scripts/update_repo.py and the update logic in util/update.py, the project maintains a streamlined pipeline for distributing the library while ensuring environmental consistency.