Data Resampling & Filters
Relevant source files
The following files were used as context for generating this wiki page:
Data Resampling and Filters constitute a core architectural component of the backtrader framework, enabling the transformation, augmentation, and temporal adjustment of market data feeds. Resampling allows a user to take a data feed at a specific resolution (e.g., 1-minute bars) and transform it into a higher timeframe (e.g., 5-minute or daily bars), while Replaying simulates the tick-by-tick development of those bars for more realistic backtesting.
Filters are modular components applied to data feeds to modify their behavior, such as filling missing time gaps in a session or filtering specific time periods. These tools allow strategies to operate across multiple timeframes simultaneously and ensure data integrity during simulation.
Sources: README.md:27-32, backtrader/resamplerfilter.py:126-146
Data Resampling and Replaying
The resampling system is primarily implemented through the Resampler and Replayer classes, which inherit from a common _BaseResampler logic. Resampling involves aggregating multiple smaller bars into a single larger bar, effectively increasing the timeframe and compression of the data feed.
Resampling Architecture
Resampling works by accumulating data points until a "bar over" condition is met. This condition is determined by the target timeframe and compression parameters. The system can handle various timeframes including Ticks, Seconds, Minutes, Days, Weeks, Months, and Years.
The diagram above shows the logic flow for aggregating source data into resampled bars.
Sources: backtrader/resamplerfilter.py:148-185, backtrader/resamplerfilter.py:469-526
Replaying Logic
Unlike resampling, which delivers a closed bar only when the period is finished, replaying delivers "in-progress" bars. It simulates the market by updating a single bar multiple times as new data arrives, which is critical for strategies that need to react to intraday movements within a larger timeframe.
Sources: backtrader/resamplerfilter.py:539-623, samples/data-multitimeframe/data-multitimeframe.py:108-124
Core Resampling Components
| Class | Description |
|---|---|
Resampler | Aggregates bars and delivers them only when the timeframe boundary is crossed. |
Replayer | Updates the current bar dynamically with each new data point from the source. |
DTFaker | Simulates datetime objects for data sources that require wall-clock checks in live environments. |
_Bar | A internal data holder for OHLCV values during the aggregation process. |
Sources: backtrader/resamplerfilter.py:35-117, backtrader/resamplerfilter.py:462-467
Data Filters
Filters are used to clean or modify the incoming stream of data. A prominent example is the DataFiller, which ensures that a strategy receives a continuous stream of data even if the original source has gaps (e.g., periods of no trading volume).
Session Filling and Filtering
The DataFiller (and specialized variants like SessionFiller) uses the data feed's timeframe, compression, sessionstart, and sessionend to identify missing bars. If a gap is detected, it generates synthetic bars using the closing price of the last known bar.
The sequence illustrates how the DataFiller intercepts gaps and provides synthetic bars to the strategy.
Sources: backtrader/filters/datafiller.py:33-55, samples/data-filler/data-filler.py:65-72
Filter Configuration Options
Filters are added to data feeds using the addfilter() method. The behavior of a filler can be customized via parameters.
| Parameter | Type | Default | Description |
|---|---|---|---|
fill_price | Float | None | The price to use for fill bars; defaults to the last close. |
fill_vol | Float | NaN | The volume to assign to synthetic bars. |
fill_oi | Float | NaN | The open interest to assign to synthetic bars. |
Sources: backtrader/filters/datafiller.py:57-61, samples/data-filler/data-filler.py:126-133
Multi-Timeframe Integration
The system allows a single Cerebro instance to handle multiple data feeds at different timeframes. This is often achieved by cloning a base data feed and applying a resampler or replayer to the clone.
Multi-Timeframe Data Flow
When multiple timeframes are used, the strategy's next() method is called when all data feeds have at least one bar. The Strategy class manages the "minimum period" required for indicators across all timeframes.
Sources: backtrader/strategy.py:175-225, samples/data-multitimeframe/data-multitimeframe.py:126-140
Architecture of a multi-timeframe system where Data1 is a resampled version of Data0.
Sources: samples/data-multitimeframe/data-multitimeframe.py:125-145
Technical Implementation Details
Time Alignment (adjbartime)
In _BaseResampler, the adjbartime and bar2edge parameters control how the timestamp of the resulting bar is determined. If adjbartime is true, the bar timestamp is adjusted to the boundary of the timeframe (e.g., a 5-minute bar starting at 10:01 will be timestamped at 10:05 if rightedge is active).
Sources: backtrader/resamplerfilter.py:148-166, backtrader/resamplerfilter.py:421-459
Buffer Management
Resampling and filtering interact with the memory management system of the strategy. The qbuffer method in the Strategy class enables memory-saving schemes by strictly keeping only the minimum needed bars for resampled feeds when savemem is enabled.
Sources: backtrader/strategy.py:148-173
Summary
Data Resampling and Filters provide the necessary abstraction to handle complex temporal data transformations in Backtrader. By separating the logic of bar aggregation (Resampling), intraday simulation (Replaying), and gap management (Filtering), the framework allows for robust backtesting across varied market conditions and timeframes.