gRPC and Protobuf Integration

Relevant source files

The following files were used as context for generating this wiki page:

• docker/container/metatrader/mt5.proto
• docker/container/metatrader/mt5_pb2.py
• docker/container/metatrader/mt5_pb2_grpc.py
• docker/container/metatrader/_generated_protocols.py
• docker/container/metatrader/bridge.py
• docker/container/metatrader/setup.sh

The gRPC and Protobuf integration in the mt5-docker project provides a high-performance, asynchronous communication layer that replaces the legacy RPyC bridge. This system allows remote Python applications (via mt5linux) to control a MetaTrader 5 (MT5) terminal running inside a Linux container using Wine. By utilizing Protocol Buffers (Protobuf) for serialization and gRPC for the transport layer, the project achieves better performance, concurrent request handling, and a strictly defined API surface.

The core of this integration is the MT5Service, which exposes the full MetaTrader 5 API—including market data, trading operations, and account management—over a network socket. This architecture ensures that complex data structures like NumPy arrays and MT5 namedtuples are efficiently materialized and transmitted between the isolated Wine environment and the host system or other network clients.

Sources: docker/container/metatrader/mt5.proto:1-5, docker/container/metatrader/bridge.py:1-25

Service Architecture and Components

The integration follows a standard gRPC client-server model where the server runs inside the Wine environment. The server (bridge.py) implements the logic defined in the .proto file and interacts directly with the MetaTrader 5 Python module installed under Wine.

System Overview Diagram

This diagram illustrates the relationship between the Protobuf definition, the generated stubs, and the runtime implementation within the container.

Core Components

Component	File Path	Description
Protocol Definition	mt5.proto	Defines the MT5Service RPC methods and the structured message types for requests and responses.
Generated Data	mt5_pb2.py	Contains the Python classes generated from Protobuf messages.
Generated Service	mt5_pb2_grpc.py	Contains the service stub for clients and the servicer base class for the server.
Implementation	bridge.py	Implements the MT5GRPCServicer class, which contains the actual logic for calling the MT5 API.
Protocol Layer	_generated_protocols.py	Provides a typed runtime boundary for the generated modules, ensuring type safety during development.

Sources: docker/container/metatrader/mt5.proto:1-10, docker/container/metatrader/bridge.py:53-60, docker/container/metatrader/_generated_protocols.py:1-15

API Definition and Service Methods

The MT5Service is categorized into several functional areas that mirror the MetaTrader 5 Python API capabilities.

Functional Categories

1. Terminal Operations: Controls the lifecycle of the connection, including Initialize, Login, and Shutdown.
2. Market Data: High-bandwidth operations for fetching rates and ticks, utilizing NumpyArray messages for efficient transport.
3. Trading Operations: Methods like OrderSend and OrderCheck that handle JSON-serialized order dictionaries.
4. Introspection: Dynamic discovery of MT5 constants, methods, and models via GetConstants, GetMethods, and GetModels.
5. Provisioning: Specialized methods for container management, such as CreateDemoAccount.

Sources: docker/container/metatrader/mt5.proto:195-265, docker/container/metatrader/bridge.py:355-385

Data Flow for Market Data

The following sequence diagram shows how market data is fetched and transformed into a Protobuf-compatible format.

Data Serialization and Protobuf Messages

Because MetaTrader 5 returns complex objects (NamedTuples and structured NumPy arrays), the bridge implements custom serialization logic to convert these to Protobuf messages.

Specialized Message Types

Message Type	Fields	Serialization Logic
NumpyArray	data (bytes), dtype (str), shape (int32 list)	Uses numpy.tobytes() for the raw data buffer.
DictData	json_data (string)	Uses orjson to serialize dictionaries into JSON strings for transport.
DictList	json_items (string list)	A list of JSON-serialized dictionary strings.
SymbolsResponse	total (int32), chunks (string list)	Handles large symbol lists by chunking them into smaller JSON arrays (500 items per chunk).

Sources: docker/container/metatrader/mt5.proto:80-100, docker/container/metatrader/bridge.py:270-300, docker/container/metatrader/bridge.py:610-630

Example: Order Serialization

For trading operations, order requests are transmitted as JSON strings within an OrderRequest message. The server deserializes this using orjson.loads() before passing it to the MT5 terminal.
Sources: docker/container/metatrader/bridge.py:288-295, docker/container/metatrader/mt5.proto:139-141

Runtime Implementation in bridge.py

The bridge.py script serves as the gRPC server entry point inside Wine. It uses a ThreadPoolExecutor to handle concurrent requests and includes signal handling for graceful shutdowns.

Server Lifecycle

The server is initialized by the serve() function, which binds the MT5GRPCServicer to a specific host and port (defaulting to 0.0.0.0:50051).
Sources: docker/container/metatrader/bridge.py:1150-1175

def serve(host: str = "0.0.0.0", port: int = 50051, max_workers: int = 10) -> None:
    server = _grpc_runtime.server(futures.ThreadPoolExecutor(max_workers=max_workers))
    add_MT5ServiceServicer_to_server(MT5GRPCServicer(), server)
    server_address = f"{host}:{port}"
    server.add_insecure_port(server_address)
    server.start()
    server.wait_for_termination()

Sources: docker/container/metatrader/bridge.py:1155-1170

Timeout Protection

To prevent the gRPC server from hanging on blocking MT5 API calls, the bridge implements a _call_mt5_with_timeout helper. This wraps synchronous MT5 calls in a future and enforces a configurable timeout (defaulting to 30 seconds).
Sources: docker/container/metatrader/bridge.py:220-255

Build and Deployment Integration

The gRPC bridge files are integrated into the container during the build and setup phases.

Setup and Verification

The setup.sh script is responsible for:

1. Creating the mt5linux package directory in the Wine Python site-packages.
2. Copying bridge.py, mt5_pb2.py, and mt5_pb2_grpc.py to the target.
3. Verifying the Protobuf runtime version by attempting to import the stubs immediately after copying.

Sources: docker/container/metatrader/setup.sh:255-285

# From setup.sh: verifying the gRPC bridge files
if ! "$wine_executable" "$WINE_PYTHON_PATH" -c "from mt5linux import mt5_pb2, mt5_pb2_grpc" 2>&1; then
    log ERROR "[setup] FATAL: bridge stubs fail to import -- protobuf runtime/gencode mismatch?"
    exit 1
fi

Sources: docker/container/metatrader/setup.sh:280-285

Summary

The gRPC and Protobuf integration provides a robust, typed interface for MetaTrader 5 remote control. By defining the API in mt5.proto and implementing the logic in a specialized Wine-based bridge, the project enables high-performance financial data streaming and trading while maintaining strict separation between the Linux host and the Windows-based terminal.