Alternative Data Streams
Relevant source files
The following files were used as context for generating this wiki page:
Alternative data refers to non-traditional financial data used to derive insights and signals for algorithmic trading. Within the hyperdrive project, alternative data streams include social sentiment analysis, macroeconomic indicators like unemployment rates, and blockchain-specific metrics such as Stock-to-Flow (S2F) ratios and SOPR (Spent Output Profit Ratio).
These streams are integrated through specific provider classes that inherit from a base MarketData architecture. This design allows the project to supplement standard OHLC market data with qualitative and structural data to enhance quantitative research and model training.
Sources: README.md:200-210, hyperdrive/DataSource.py:10-20
Architecture and Data Flow
The architecture for alternative data follows a provider-consumer model where specialized classes interface with external APIs (StockTwits, Bureau of Labor Statistics, Glassnode), standardize the resulting DataFrames, and persist them to local or S3 storage.
Data Collection Workflow
The MarketData base class provides a standardize method that ensures consistent column naming and data types across different alternative sources.
Sources: hyperdrive/DataSource.py:38-55, hyperdrive/DataSource.py:440-450, test/test_DataSource.py:200-210
Social Sentiment and Volume
The project utilizes StockTwits to monitor investor sentiment. This data is split into two primary components: social volume (the intensity of discussion) and social sentiment (the bullish vs. bearish bias).
Key Components
- Provider Class:
StockTwits - Endpoints:
volume.jsonandsentiment.json - Persistence: Data is saved to a unified sentiment path via
save_social_sentiment.
| Field | Description | Source Mapping |
|---|---|---|
timestamp | Time of the observation | C.TIME |
bullish | Number of bullish messages | C.POS |
bearish | Number of bearish messages | C.NEG |
volume_score | Relative frequency of mentions | C.VOL |
volume_change | Delta in mentions over time | C.DELTA |
Sources: hyperdrive/DataSource.py:440-485, test/test_Constants.py:48-52
Macroeconomic Indicators
The LaborStats class interfaces with the Bureau of Labor Statistics (BLS) to fetch national unemployment rates. This data provides a broader economic context for long-term trading strategies.
The system maps the BLS response fields year and period (e.g., "M01") into a standard YYYY-MM format to align with other time-series data in the library.
Sources: hyperdrive/DataSource.py:530-565, test/test_DataSource.py:215-225
On-Chain Blockchain Metrics
The Glassnode class provides access to specialized crypto-asset metrics, primarily focused on Bitcoin (BTC). These metrics are derived from blockchain analysis rather than exchange order books.
Supported Metrics
- Stock-to-Flow (S2F) Ratio: Tracks the relationship between the existing supply and the annual production rate.
- Difficulty Ribbon: Measures the impact of mining difficulty on price trends using multiple moving averages (MA9 to MA200).
- SOPR (Spent Output Profit Ratio): Indicates whether the market is in a state of profit or loss by analyzing on-chain transaction outputs.
Glassnode Data Structures
| Metric | Mapping Constant | Description |
|---|---|---|
| S2F Ratio | C.RATIO | On-chain supply scarcity model |
| Halving | C.HALVING | Days remaining until next BTC halving |
| Ribbon MAs | C.MAs | List of difficulty moving averages |
| SOPR | C.SOPR | Realized profit/loss ratio |
Sources: hyperdrive/DataSource.py:570-650, test/test_DataSource.py:245-265
Implementation Details
Standardization Logic
The MarketData.standardize method is the core utility for alternative data. It filters input DataFrames based on a mapping of provider-specific keys to internal constants (defined in Constants.py).
# hyperdrive/DataSource.py:38-42
def standardize(self, df, full_mapping, filename, columns, default):
mapping = {k: v for k, v in full_mapping.items() if k in df}
df = df[list(mapping)].rename(columns=mapping)
# ... further processing for time and value types
Sources: hyperdrive/DataSource.py:38-55
Redundancy and Error Handling
The try_again decorator pattern is used across all alternative data requests to handle transient API failures with configurable retries and delays.
Sources: hyperdrive/DataSource.py:22-35
Summary
Alternative Data Streams in hyperdrive provide a multi-faceted view of the market by combining social signals (StockTwits), economic health indicators (BLS), and structural asset metrics (Glassnode). This data is standardized through a unified pipeline, enabling developers to incorporate non-price signals into quantitative models and backtesting workflows.