Daily Cron Scripts
Relevant source files
The following files were used as context for generating this wiki page:
Daily Cron Scripts in the Hyperdrive library serve as the production backbone for automated data collection, quantitative analysis, and order execution. These scripts leverage GitHub Actions workflows to perform periodic tasks such as updating market data (OHLC, dividends, splits), calculating social sentiment, and executing trades based on generated signals.
The system is designed to interface with multiple data providers including IEXCloud, Polygon, and Binance while managing API rate limits and data persistence in S3-compatible storage.
Sources: README.md:214-230, hyperdrive/Workflow.py:1-15
Workflow Management and Scheduling
Automated tasks are governed by the Flow class, which parses GitHub Actions YAML configurations to determine execution timing and prevent overlapping executions that might exceed API rate limits.
Workflow Monitoring Logic
The Flow class identifies scheduled start times by extracting cron expressions from workflow files. It specifically monitors workflows for ohlc, intraday, dividends, and splits to calculate if a process is currently active based on the expected duration of data retrieval for the number of symbols tracked.
The diagram shows the logic used by the system to prevent concurrent API-heavy tasks.
Sources: hyperdrive/Workflow.py:7-45, test/test_Workflow.py:11-18
Order Execution Logic
The execute_order.py script facilitates daily trade execution by comparing the most recent signals generated by research models. It utilizes the Binance exchange interface to place market orders.
Execution Flow
- Signal Loading: Loads historical signal data from a CSV path provided by
PathFinder. - Change Detection: Analyzes the last two signal entries. An order is only triggered if the signal has flipped (e.g., from 0 to 1).
- Market Action: Executes a
BUYorder if the current signal is truthy, or aSELLorder if it is falsy. - Persistence: Logs the resulting order details (price, quantity, status) back to a central orders registry.
| Component | Responsibility | Source |
|---|---|---|
Binance.order | Interface for placing BUY/SELL orders on exchange | scripts/execute_order.py:27 |
signals_df | Dataframe containing model-generated trading signals | scripts/execute_order.py:14 |
should_order | Logic gate checking if len(set(last_two_signals)) > 1 | scripts/execute_order.py:18 |
Sources: scripts/execute_order.py:1-42
API and Research Updates
The update_api.py script generates "previews" used by the frontend or external APIs. It calculates performance metrics for a "Buy and Hold" strategy versus the "Hyperdrive" algorithmic strategy.
Portfolio Preview Generation
The script performs backtesting on the fly to generate performance stats including Total Return, Max Drawdown, and Sharpe Ratio. It supports both standard USD-denominated results and inverted BTC-denominated results for crypto-focused analysis.
The sequence demonstrates the data flow from raw storage to processed performance JSON.
Sources: scripts/update_api.py:24-106
Performance Metrics Tracked
The cron scripts calculate and export the following metrics for research validation:
- Total Return [%]: Aggregate profit/loss.
- Max Drawdown [%]: Largest peak-to-trough decline.
- Win Rate [%]: Percentage of profitable trades.
- Sharpe/Sortino Ratio: Risk-adjusted return measurements.
- Profitable Time [%]: Specifically for inverted BTC previews, measuring time outperforming the benchmark.
Sources: scripts/update_api.py:34-60
Data Collection Maintenance
Daily scripts rely on the MarketData class to maintain local and cloud caches of financial data. The system automatically standardizes incoming data from various providers to a unified format.
Supported Data Providers
The scripts interact with the following providers via DataSource.py:
| Provider | Data Types | File Path Reference |
|---|---|---|
| IEXCloud | OHLC, Dividends, Splits, Intraday | hyperdrive/DataSource.py:446 |
| Polygon | OHLC, Aggregates, Crypto | hyperdrive/DataSource.py:590 |
| StockTwits | Social Sentiment, Social Volume | hyperdrive/DataSource.py:730 |
| Glassnode | BTC S2F Ratio, Difficulty Ribbon, SOPR | hyperdrive/DataSource.py:840 |
| Labor Stats | Unemployment Rate | hyperdrive/DataSource.py:800 |
Sources: hyperdrive/DataSource.py:12-25, README.md:220-230
Conclusion
The Daily Cron Scripts form the operational layer of Hyperdrive, transforming it from a library into a live trading platform. By coordinating Workflow timing, DataSource collection, and Exchange execution, these scripts ensure that the quantitative models operate on the latest market data and that resulting insights are executed in real-time.