Introduction to AlphaPy

Relevant source files

The following files were used as context for generating this wiki page:

• README.rst
• setup.py
• alphapy/__main__.py
• alphapy/model.py
• alphapy/globals.py
• alphapy/market_flow.py
• alphapy/sport_flow.py
• alphapy/estimators.py

AlphaPy is a machine learning framework designed for speculators and data scientists to build predictive models and trading systems. Written in Python, it leverages core libraries such as scikit-learn, pandas, Keras, XGBoost, LightGBM, and CatBoost to facilitate feature engineering, model selection, and visualization. The framework provides specialized pipelines for different domains, notably MarketFlow for financial markets and SportFlow for sporting events.

The system is structured as a pipeline that manages the entire machine learning lifecycle, from data ingestion and transformation to model training, ensemble generation, and prediction. It includes built-in support for portfolio management through integration with Quantopian's pyfolio.

Sources: README.rst:1-25, setup.py:9-12

Core Framework Architecture

AlphaPy operates through a central execution engine that coordinates data processing, feature engineering, and model estimation. The architecture is modular, allowing users to define model specifications via YAML configuration files (model.yml) which govern the behavior of the training and prediction pipelines.

The Model Object

The Model class serves as the primary data structure, unifying algorithms, features, and results. It stores training and testing matrices, feature importances, coefficients, and evaluation metrics.

Execution Pipelines

AlphaPy features three primary entry points accessible via console scripts:

• alphapy: The general-purpose machine learning pipeline.
• mflow: The MarketFlow pipeline specialized for financial market analysis.
• sflow: The SportFlow pipeline specialized for predicting sports outcomes.

Sources: setup.py:62-66

Machine Learning Pipelines

Training Pipeline

The training pipeline handles data splitting, feature extraction, model selection (including grid search and RFE), and ensembling. It supports both classification and regression tasks.

Prediction Pipeline

The prediction pipeline is used for making inferences on new data. It loads a previously saved model and feature map to ensure that the same transformations applied during training are applied to the prediction set.

Key Components and Features

AlphaPy provides a wide array of technical features summarized in the table below:

Component	Description	Reference File
Estimators	Supports scikit-learn, XGBoost, LightGBM, CatBoost, and Keras.	alphapy/estimators.py
Encoders	15 encoding types including OneHot, Target, and CatBoost encoding.	alphapy/globals.py
Sampling	Handling imbalanced data via SMOTE, Random Under-sampling, etc.	alphapy/globals.py
Metrics	Automated calculation of AUC, Log Loss, F1, MSE, and R-squared.	alphapy/model.py
Visualization	Automated ROC curves, Confusion Matrices, and Learning Curves.	alphapy/plots.py

Sources: alphapy/globals.py:84-118, alphapy/estimators.py:150-170, alphapy/model.py:650-750

Specialized Flow Modules

MarketFlow (mflow)

MarketFlow is designed for financial speculators. It manages market-specific data fractals (e.g., 5m, 1h, 1d), handles technical indicator generation, and integrates with trading systems. It allows for "Long Entry", "Short Entry", and "Hold Period" specifications.
Sources: alphapy/market_flow.py:118-215

SportFlow (sflow)

SportFlow automates the creation of "Team Frames" and "Game Frames". It calculates streaks, cover margins against the spread, and rolling averages for points scored/allowed to predict game outcomes.
Sources: alphapy/sport_flow.py:214-315

Configuration and Global Constants

The framework is highly configurable via model.yml. Users can toggle features like recursive feature elimination (rfe), grid search, and various data scaling options (minmax, standard).

# Example model.yml snippet
model:
    algorithms: ['RF', 'XGB', 'LOGR']
    scoring_function: 'roc_auc'
    type: classification
    grid_search:
        option: True
        iterations: 100

Sources: alphapy/model.py:155-250, alphapy/examples/Trading Model/config/model.yml:34-55

Model and Partition Types

AlphaPy defines strict enumerations for its operating modes:

• ModelTypes: classification, clustering, multiclass, oneclass, regression.
• Partitions: train, test, predict.

Sources: alphapy/globals.py:126-140, alphapy/globals.py:186-193

Conclusion

AlphaPy provides a structured, automated environment for rapid experimentation with machine learning models. By abstracting the boilerplate code for data processing and model evaluation into standardized pipelines, it enables users to focus on feature engineering and strategy development in the financial and sports domains.