Autogluon Forecasting
Relevant source files
The following files were used as context for generating this wiki page:
AutoGluon Forecasting is a high-performance time-series modeling system within the Rabid Raccoon project designed to predict price movements for Micro E-mini S&P 500 (MES) futures. It leverages the AutoGluon 1.5 "Model Zoo," which includes over 35 distinct models ranging from classical statistical methods to state-of-the-art deep learning transformers and Chronos foundation models. The system integrates market data, economic indicators from FRED, and sentiment-based news headlines to generate multi-horizon forecasts.
The forecasting engine operates through a two-step pipeline: extensive feature engineering via TypeScript-based dataset builders and model training/inference via a Python-based TimeSeries engine. It supports varying "quality" profiles, with the "extreme" profile utilizing a 4-hour training window to evaluate the full model suite.
Sources: TRAINING-COMPLETE.md:1-5, mes_hft_halsey/mes_autogluon_timeseries.py:270-340
Architecture and Data Flow
The forecasting architecture is structured as a pipeline that transitions from raw data ingestion to feature-rich CSV datasets, and finally into trained model artifacts and JSON summaries.
The diagram above illustrates the transition from raw market and macro data through the TypeScript-based feature engineering layer into the Python-based AutoGluon training engine.
Sources: TRAINING-COMPLETE.md:58-85, train-complete.sh:35-65
Feature Engineering and Datasets
The project uses specialized TypeScript scripts to assemble high-dimensional feature matrices. These scripts join OHLCV market data with lagged economic and news data.
Data Categories
- Market Data: OHLCV for MES and 60+ other futures symbols.
- Economic Data (FRED): 23 categories including interest rates, yields, VIX, inflation, and money supply.
- Sentiment/News: 7-day rolling counts of economic/policy news and raw text headlines embedded via transformers.
- Technical Indicators: RSI (2 and 14 period), rolling Moving Averages (MA8, MA24, MA120), and volatility ratios.
Sources: TRAINING-COMPLETE.md:7-45, scripts/build-complete-dataset.ts:100-150
Dataset Variants
| Script | Timeframe | Columns | Focus |
|---|---|---|---|
build-complete-dataset.ts | 1h | ~85 | Full production with all FRED and news sources. |
build-15m-dataset.ts | 15m | ~80+ | High-granularity intraday features. |
build-lean-dataset.ts | 1h/15m | ~66 | Optimized for speed; excludes stale monthly/quarterly data. |
Sources: scripts/build-complete-dataset.ts:16-30, scripts/build-15m-dataset.ts:12-25, scripts/build-lean-dataset.ts:10-25
Model Training and Selection
The Python module mes_autogluon_timeseries.py serves as the primary training driver. It utilizes autogluon.timeseries.TimeSeriesPredictor to automate model selection and ensembling.
The AutoGluon 1.5 Model Zoo
The system evaluates several classes of models:
- Statistical: AutoARIMA, AutoETS, DynamicOptimizedTheta.
- Deep Learning: TemporalFusionTransformer (TFT), PatchTST, DeepAR.
- Foundation Models: Chronos (tiny, mini, small, base, large).
- Tabular: RecursiveTabular, DirectTabular.
Sources: TRAINING-COMPLETE.md:47-65, mes_hft_halsey/mes_autogluon_timeseries.py:270-320
Training Quality Profiles
Users can specify the training intensity, which dictates the time limit and the breadth of models tested.
| Quality | Time Limit | Presets |
|---|---|---|
| Fast | 5 min | fast_training |
| Medium | 30 min | medium_quality |
| Best | 2 hours | best_quality |
| Extreme | 4 hours | best_quality (Full Zoo) |
Sources: mes_hft_halsey/mes_autogluon_timeseries.py:118-132
Multi-Symbol and Database Integration
The system supports "Global Training," where a single model is trained across 60+ different futures symbols (e.g., MES, ES, NQ, CL, GC). This allows the model to learn cross-asset correlations and improves generalization through transfer learning.
Database Source Logic
When the --use-database flag is provided to mes_autogluon_timeseries.py, the script bypasses the Databento API and pulls historical data directly from local PostgreSQL tables like futures_ex_mes_1h.
# From mes_hft_halsey/mes_autogluon_timeseries.py
def _fetch_from_database(timeframe: Timeframe, days_back: int):
# Map timeframe to table
table_map = {
"1h": "futures_ex_mes_1h",
"1d": "futures_ex_mes_1d",
}
# ... executes SQL to fetch all symbol_code groups
Sources: mes_hft_halsey/mes_autogluon_timeseries.py:214-230, TRAINING-COMPLETE.md:105-115
Forecast Output and Summarization
Upon completion, the system produces a standardized summary and forecast file.
Summary Schema
The MesRealModelSummary dataclass captures the results of the training run:
- Leaderboard Top Model: The best performing model (often
WeightedEnsemble). - Leaderboard Top Score: Validation metric (MASE).
- Bias: Categorical prediction (
LONG,SHORT, orFLAT) based on the threshold-adjusted forecast delta. - Horizon Forecasts: Specific price targets for horizons like
15m,1h,4h, and24h.
Sources: mes_hft_halsey/mes_autogluon_timeseries.py:137-164, mes_hft_halsey/mes_autogluon_timeseries.py:350-380
Performance Optimization
The pipeline enforces strict anti-leakage policies during dataset construction to ensure technical accuracy:
- Daily Data: 1-day lag.
- Weekly Data: 8-day lag.
- Monthly Data: 35-day lag.
- Headlines: 1-day lag for 7-day rolling windows.
Sources: TRAINING-COMPLETE.md:117-125, scripts/build-complete-dataset.ts:245-260
AutoGluon Forecasting provides the project with a robust, automated pipeline for transitioning from raw multi-asset market data to actionable trading forecasts. By combining classical statistical rigor with deep learning transformers and foundation models, the system attempts to capture complex macro and technical patterns in the MES futures market.