Market Data Fetching
Relevant source files
The following files were used as context for generating this wiki page:
Market Data Fetching in mt5linux facilitates the retrieval of real-time and historical financial information from the MetaTrader 5 (MT5) terminal. The system enables Linux-based clients to access symbol specifications, price ticks, OHLCV (Open, High, Low, Close, Volume) rates, and Market Depth (Depth of Market) through a gRPC bridge.
The architecture transitions from older RPyC-based implementations to a high-performance gRPC framework, allowing for both synchronous and asynchronous data fetching. Data is typically returned as JSON-serialized dictionaries or high-performance NumPy arrays, ensuring compatibility with data analysis and machine learning workflows. Sources: [README.md:3-10], [mt5.proto:1-5], [mt5linux/mt5_pb2_grpc.py:1-40].
Data Architecture and Flow
The market data fetching process involves a gRPC client (Sync or Async) sending requests to a bridge server, which interacts with the MT5 terminal running under Wine or Windows. The response is then deserialized into Pydantic models or NumPy arrays for the end user.
Visual representation of the data flow from the MT5 terminal to the end user via the gRPC bridge.
Sources: [mt5.proto:265-275], [mt5linux/mt5_pb2_grpc.py:100-150].
Symbol Information and Selection
Before fetching market data, symbols must typically be selected in the MT5 "Market Watch" window. The system provides methods to query total symbol counts, retrieve detailed symbol specifications, and enable/disable symbol tracking.
Key Components
- SymbolInfo Model: A comprehensive Pydantic model containing 96 fields related to symbol identification, trading modes, digits, spreads, and session data.
- SymbolRequest: A protobuf message used to query data for a specific symbol string.
| Method | Request Message | Response | Description |
|---|---|---|---|
SymbolsTotal | Empty | IntResponse | Returns the total number of available symbols. |
SymbolsGet | SymbolsRequest | SymbolsResponse | Returns a chunked list of available symbols, optionally filtered by group. |
SymbolInfo | SymbolRequest | DictData | Returns detailed metadata for a specific symbol. |
SymbolSelect | SymbolSelectRequest | BoolResponse | Enables or disables a symbol in the Market Watch window. |
Sources: [mt5.proto:277-282], [mt5linux/models.py:276-350], [tests/integration/test_parametrized.py:146-160].
Rates and Ticks Retrieval
Historical and real-time price data are retrieved via CopyRates and CopyTicks methods. These operations return NumpyArray protobuf messages, which are reconstructed into structured NumPy arrays on the client side for efficient processing.
Rates (OHLCV) Fetching
The system supports three primary ways to fetch OHLCV bars:
- From Date: Fetching a specific number of bars starting from a provided timestamp.
- From Position: Fetching a specific number of bars starting from an index (e.g., 0 for the most recent bar).
- Range: Fetching all bars within a specific start and end time boundary.
Tick Fetching
Ticks represent individual price changes. Clients can fetch a specific number of ticks or ticks within a time range, using flags to filter for all ticks, info ticks (price changes), or trade ticks (last trade changes).
Sequence diagram for fetching OHLCV rates via positional index.
Sources: [mt5.proto:118-160], [mt5linux/constants.py:530-555], [tests/integration/test_async_client.py:165-190].
Market Depth (Depth of Market)
Market Depth (Order Book) data provides insights into the liquidity of a symbol at different price levels. This requires an explicit subscription lifecycle.
Lifecycle Management
- Subscription:
MarketBookAdd(symbol)initiates the subscription. If the broker does not support DOM for that symbol, it returnsFalse. - Retrieval:
MarketBookGet(symbol)fetches the current state of the book. - Release:
MarketBookRelease(symbol)stops the subscription to free resources.
Book Entry Data
Data is returned as a list of BookEntry objects, which include:
- Type: The type of order (Buy, Sell, Buy Market, Sell Market).
- Price: The price level of the entry.
- Volume: The liquidity available at that price.
Sources: [mt5.proto:336-340], [mt5linux/models.py:465-475], [tests/integration/test_market_depth.py:20-60].
Timeframes and Constants
Market data retrieval relies on standardized integer constants for timeframes and data flags.
| Constant Name | Value (Minutes/Hex) | Source Description |
|---|---|---|
TIMEFRAME_M1 | 1 | 1-minute chart |
TIMEFRAME_H1 | 16385 | 1-hour chart |
TIMEFRAME_D1 | 16408 | Daily chart |
COPY_TICKS_ALL | -1 | All tick types |
COPY_TICKS_INFO | 1 | Price change ticks only |
COPY_TICKS_TRADE | 2 | Last trade ticks only |
Sources: [mt5linux/constants.py:506-538], [tests/integration/test_parametrized.py:535-555].
Resilience and Error Handling
Data fetching operations are classified by criticality to determine retry strategies. Most market data operations (e.g., copy_rates, symbol_info) are classified as NORMAL or LOW criticality, meaning they are safe to retry upon transient network errors like DEADLINE_EXCEEDED or UNAVAILABLE.
Sources: [mt5linux/constants.py:75-120], [mt5linux/constants.py:310-335].
The Market Data Fetching module provides the foundational data necessary for automated trading strategies, offering efficient binary transfers for large datasets and a robust subscription model for real-time market depth.