Feature Engineering & Transforms
Relevant source files
The following files were used as context for generating this wiki page:
AlphaPy provides a robust pipeline for transforming raw data into predictive features through automated engineering, scaling, and selection. This module handles everything from simple numerical imputation to complex dimensionality reduction and categorical encoding, ensuring that data is optimally prepared for machine learning estimators.
The feature engineering process is integrated directly into the training_pipeline and prediction_pipeline, allowing for consistent application of transformations across training and inference stages. It supports a wide variety of data types, including numerical, categorical (factors), and text features.
Sources: alphapy/main.py:80-160, alphapy/features.py:612-700
Pipeline Architecture
The feature engineering workflow follows a sequential logic where transformations are applied, features are generated, and then the feature space is pruned via selection or variance thresholds.
The diagram shows the sequential flow of feature processing within the AlphaPy pipeline.
Sources: alphapy/main.py:126-160, alphapy/features.py:612-710
Core Components and Logic
The system distinguishes between several types of feature operations:
- Transforms: External or custom functions applied to specific columns before general feature creation.
- Base Features: The primary processing of numerical, factor, and text data, including imputation and encoding.
- Dimensionality Reduction: Techniques like PCA, Isomap, and T-SNE to condense the feature space.
- Interactions: Generation of polynomial features based on existing high-importance variables.
- Pruning: Removal of features based on low variance or univariate statistical significance.
Sources: alphapy/features.py:108-140, alphapy/features.py:612-660, alphapy/features.py:840-860
Transformation and Imputation
AlphaPy uses specific strategies to handle missing values and non-normal distributions. Numerical features are often imputed using the median for floats and most frequent for integers.
Imputation Strategies
| Data Type | Strategy | Description |
|---|---|---|
float64 | Median | Replaces NaNs and Infs with the median of the column. |
int64 | Most Frequent | Replaces NaNs with the most common value. |
bool | None | No imputation applied. |
| Others | Constant | Replaces NaNs with a configured sentinel value. |
Sources: alphapy/features.py:182-214, alphapy/model.py:180-190
Transform Application
The apply_transforms function allows users to specify custom modules and functions to be applied to features. This is often used for time-series operations like "runs tests" or financial technical indicators.
def apply_transform(fname, df, fparams):
module = fparams[0]
func_name = fparams[1]
plist = fparams[2:]
ext_module = import_module(module)
func = getattr(ext_module, func_name)
plist.insert(0, fname)
plist.insert(0, df)
return func(*plist)
Sources: alphapy/features.py:88-106
Categorical Encoding
AlphaPy supports a wide array of encoding methods for categorical "factor" variables, leveraging the category_encoders library.
| Encoder Type | Global Constant | Implementation |
|---|---|---|
| Target | Encoders.target | ce.TargetEncoder |
| One-Hot | Encoders.onehot | ce.OneHotEncoder |
| Binary | Encoders.binary | ce.BinaryEncoder |
| CatBoost | Encoders.catboost | ce.CatBoostEncoder |
| Ordinal | Encoders.ordinal | ce.OrdinalEncoder |
Factors are processed through get_factors, which handles the fitting on training data and transformation on testing/prediction data to prevent data leakage.
Sources: alphapy/globals.py:84-106, alphapy/features.py:73-86, alphapy/features.py:488-535
Advanced Feature Generation
Beyond standard data cleaning, AlphaPy can generate complex statistical and structural features.
Dimensionality Reduction and Clustering
If enabled in the model.yml configuration, the pipeline generates additional columns using:
- Clustering:
MiniBatchKMeansis used to create cluster assignment features (e.g.,cluster_3). - PCA: Principal Component Analysis for linear dimensionality reduction.
- Manifold Learning: T-SNE and Isomap for non-linear dimensionality reduction.
Sources: alphapy/features.py:598-610, alphapy/features.py:726-836
Statistical Features
AlphaPy can compute row-wise statistics across all features using NumPy and SciPy, creating features like sp_skew, sp_kurtosis, and sp_standard_error_of_mean.
Sources: alphapy/features.py:560-596
Feature Selection and Pruning
To manage the "curse of dimensionality," AlphaPy provides multiple layers of feature pruning.
Low Variance Removal
Features with variance below a user-defined threshold (defaulting to 0.1 in some examples) are removed using VarianceThreshold.
Sources: alphapy/features.py:910-940, alphapy/examples/Trading Model/config/model.yml:95
Univariate Selection
The select_features function uses SelectPercentile to keep only the most statistically significant features based on scorers like f_classif, chi2, or f_regression.
The sequence shows how the support mask is generated and applied to reduce the feature space.
Sources: alphapy/features.py:840-880
Specialized Flow Transformations
AlphaPy includes domain-specific feature engineering in its sub-flows.
MarketFlow Features
In market_flow.py, the system applies financial-specific variables like phigh (probability >= 0.7) and plow (probability <= 0.3), and applies user-defined functions to market data frames.
Sources: alphapy/market_flow.py:126-140
SportFlow Features
In sport_flow.py, complex "delta" features are generated by calculating the difference between home and away team statistics (e.g., delta.wins, delta.cover_win_streak). It also calculates streaks and rolling averages for point margins and cover margins.
Sources: alphapy/sport_flow.py:382-402, alphapy/sport_flow.py:58-96
Summary
The Feature Engineering & Transforms module in AlphaPy is a comprehensive suite designed to automate the preparation of data for predictive modeling. By combining standard preprocessing (imputation, scaling) with advanced techniques (clustering, PCA, categorical encoding) and domain-specific logic (MarketFlow/SportFlow), it allows users to move from raw data to a refined feature set with minimal manual intervention. The integration of feature selection and variance thresholds further ensures that the resulting models remain performant and generalize well.
Sources: alphapy/features.py, alphapy/main.py