Algorithmic Trading Engine
Relevant source files
The following files were used as context for generating this wiki page:
The Algorithmic Trading Engine is a comprehensive framework designed for quantitative research, data collection, strategy backtesting, and live order execution. It serves as the core infrastructure for power quant research, prioritizing robust data ingestion from multiple providers and seamless transition from historical analysis to live market interaction. Sources: README.md:16-24
System Architecture
The engine is structured into modular components that handle distinct stages of the trading lifecycle: data acquisition, historical simulation, and execution.
The diagram above illustrates the high-level data flow from external providers through the internal processing modules to final execution. Sources: hyperdrive/DataSource.py, hyperdrive/History.py, hyperdrive/Exchange.py, hyperdrive/Workflow.py
Key Components
| Component | Description | Primary Classes |
|---|---|---|
| Data Ingestion | Fetches and standardizes market data, dividends, and social sentiment. | MarketData, IEXCloud, Polygon, Glassnode |
| Research & Backtesting | Provides tools for signal generation, portfolio simulation, and ML classification. | Historian |
| Execution | Interface for interacting with cryptocurrency exchanges and managing orders. | Binance |
| Orchestration | Manages scheduled jobs and ensures data consistency across workflows. | Flow |
Sources: hyperdrive/DataSource.py:12, hyperdrive/History.py:27, hyperdrive/Exchange.py:9, hyperdrive/Workflow.py:7
Data Acquisition and Standardization
The MarketData class and its specialized subclasses (e.g., IEXCloud, Polygon) manage the ingestion of diverse data types. The engine enforces standardization of column names and data types (e.g., converting timestamps to UTC and scaling values). Sources: hyperdrive/DataSource.py:12-45
Data Providers and Types
The engine supports multiple third-party APIs:
- Equities/Crypto: IEXCloud and Polygon for OHLCV, Intraday, Dividends, and Splits. Sources: hyperdrive/DataSource.py:315, hyperdrive/DataSource.py:461
- Social: StockTwits for sentiment and volume scores. Sources: hyperdrive/DataSource.py:545
- On-chain: Glassnode for Bitcoin-specific metrics like Stock-to-Flow (S2F) and SOPR. Sources: hyperdrive/DataSource.py:656
- Macro: Bureau of Labor Statistics (BLS) for unemployment rates. Sources: hyperdrive/DataSource.py:616
Quantitative Research and Backtesting
The Historian class leverages vectorbt for performance analysis. It supports creating portfolios from signals and calculating advanced metrics like Sharpe, Sortino, and Calmar ratios. Sources: hyperdrive/History.py:27-50
Signal Generation and Optimization
The engine can identify optimal historical signals using local extrema or generate random strategies for benchmarking. Sources: hyperdrive/History.py:73-88
- Optimal Signals: Uses
scipy.signal.argrelextremato find ideal buy/sell points based on a lookback periodn. Sources: hyperdrive/History.py:75-80 - Machine Learning: Includes utilities for undersampling imbalanced datasets, PCA for dimensionality reduction, and a suite of classifiers (Random Forest, SVM, MLP) to predict future signals. Sources: hyperdrive/History.py:123-150, hyperdrive/History.py:195-210
Order Execution
Live execution is primarily handled through the Binance class, which interacts with the Binance US API.
Execution Logic
The execution script evaluates the most recent signals produced by the research pipeline. If a signal change is detected (e.g., switching from a 0 to a 1), it triggers a market order. Sources: scripts/execute_order.py:15-30
# Simplified order logic from scripts/execute_order.py
if should_order:
order = bn.order(
base, quote, 'BUY' if signal else 'SELL', spend_ratio, C.TEST
)
Sources: scripts/execute_order.py:27-30
Order Parameters
The Binance.order method calculates quantities based on the account's "free" balance and applies necessary asset-specific filters like LOT_SIZE and quoteAssetPrecision. Sources: hyperdrive/Exchange.py:31-60
Workflow Management
The Flow class in Workflow.py monitors the status of automated data updates. It calculates the expected duration of GitHub Action workflows (like ohlc or dividends) to prevent resource contention or redundant API calls. Sources: hyperdrive/Workflow.py:7-25
Sources: hyperdrive/Workflow.py:26-45
Summary
The Algorithmic Trading Engine provides a modular pipeline for financial engineering. By abstracting data provider complexities through the DataSource module and providing a robust backtesting suite via Historian, the system enables rapid strategy development. Integration with the Binance exchange and automated Workflow monitoring ensures that researched strategies can be deployed into live environments with consistent data and execution logic. Sources: README.md:200-215, hyperdrive/History.py:27, hyperdrive/Exchange.py:9