Analysis & Visualizations
Relevant source files
The following files were used as context for generating this wiki page:
The Analysis and Visualization modules in AlphaPy provide a structured framework for preparing financial or sports data for machine learning and generating comprehensive diagnostic plots. The system is designed to handle temporal data splitting, feature sequencing, and model performance evaluation through diverse graphical representations.
Analysis acts as the connective tissue between raw data and the model pipeline, managing the transition from raw frames to training and testing sets. Visualizations leverage libraries like matplotlib, seaborn, and bokeh to produce insights into model behavior, such as feature importance and classification reliability.
Analysis Framework
The analysis framework is centered around the Analysis class and the run_analysis function. It manages how data for specific groups (e.g., market symbols or sports teams) are processed, lagged, and partitioned.
Analysis Lifecycle
The analysis process involves loading data frames for a group, applying temporal lags, and splitting the data based on a predict_date. This ensures that models are trained on historical data and tested on subsequent "unseen" data.
Key Components
- Analysis Class: Stores the relationship between a
Modeland aGroup. Sources: alphapy/analysis.py:65-103 - run_analysis: The primary execution function that performs feature sequencing (lagging laggards and leading variables) and data partitioning. Sources: alphapy/analysis.py:114-118
- sequence_frame: A utility used within analysis to shift data for forecasting. Sources: alphapy/analysis.py:192
Visualizations and Plotting
AlphaPy includes an extensive suite of plotting functions to evaluate model performance and perform Exploratory Data Analysis (EDA).
Model Performance Plots
These plots are typically generated during the training pipeline to diagnose algorithm behavior.
| Plot Type | Description | Source |
|---|---|---|
| Calibration | Shows the reliability of predicted probabilities for classification models. | alphapy/plots.py:212-263 |
| Feature Importance | Ranks predictors by their contribution to the model's decisions. | alphapy/plots.py:270-323 |
| Learning Curve | Visualizes the effect of training sample size on model score. | alphapy/plots.py:330-403 |
| ROC Curve | Displays the True Positive vs False Positive trade-off. | alphapy/plots.py:410-459 |
| Confusion Matrix | A matrix showing true vs predicted class labels. | alphapy/plots.py:466-541 |
Technical Architecture for Plots
The write_plot function serves as a unified interface for multiple visualization backends, allowing the system to save PNGs via Matplotlib/Seaborn or HTML via Bokeh.
Sources: alphapy/plots.py:111-205
Visualization Configuration
Plot generation is controlled by the model.yml configuration file, which defines which charts to generate during the pipeline.
plots:
calibration : True
confusion_matrix : True
importances : True
learning_curve : True
roc_curve : True
Sources: alphapy/examples/Trading Model/config/model.yml:105-110, alphapy/model.py:421-426
Data Flow in Analysis
In specialized pipelines like MarketFlow and SportFlow, the analysis is customized for the domain (financial markets vs. sports stats).
SportFlow Analysis
In sport_flow.py, the analysis focuses on team-based statistics and seasonal data, merging "home" and "away" statistics into a single model frame before the final analysis run.
Sources: alphapy/sport_flow.py:421-550
MarketFlow Analysis
In market_flow.py, the analysis utilizes technical indicators (variables) and sequences them using the forecast_period and lag_period parameters.
Sources: alphapy/market_flow.py:214-224
Implementation Details
Automated Plot Generation
The training pipeline automatically triggers plot generation if enabled in the specifications.
# From alphapy/__main__.py
# Generate plots
generate_plots(model, Partition.train)
if model.test_labels:
generate_plots(model, Partition.test)
Sources: alphapy/main.py:228-231
Visualization Libraries
AlphaPy supports several libraries, handled dynamically based on the plot type:
- Matplotlib: Used for most model diagnostic plots (ROC, Calibration, Importances).
- Seaborn: Used for EDA (Distribution, Box, Swarm, Scatter plots).
- Bokeh: Used for interactive time-series visualizations, specifically Candlestick charts.
Sources: alphapy/plots.py:73-88, alphapy/plots.py:151-205
The Analysis and Visualization systems ensure that every model built with AlphaPy is not only mathematically validated through metrics but also visually interpreted, allowing developers to identify issues like overfitting (via Learning Curves) or poor probability scaling (via Calibration Plots).