Health Monitoring & Auto-Recovery
Relevant source files
The following files were used as context for generating this wiki page:
Health Monitoring and Auto-Recovery in the MT5-Docker project is a multi-layered system designed to ensure the continuous operation of the MetaTrader 5 terminal and its associated gRPC/RPyC bridge. The system proactively monitors process status, network connectivity, and service responsiveness, triggering automated recovery actions when failures exceed defined thresholds.
The architecture relies on a decoupled design where a background monitoring script (health_monitor.sh) identifies issues and signals a dedicated service supervisor (svc-mt5server under s6-overlay) to perform restarts. This ensures that transient failures in the terminal or bridge do not result in permanent service downtime, which is critical for automated trading environments.
Sources: README.md:27-28, docker/container/metatrader/health_monitor.sh:5-9
Monitoring Architecture
The system utilizes a token-based communication mechanism between the health monitor and the service manager to coordinate restarts. This prevents race conditions during the initial container startup and ensures that recovery actions are only taken when the system is in a stable state.
Coordination Workflow
The monitoring lifecycle is managed through specific marker files located in /tmp:
- Startup Marker (
/tmp/.mt5-startup-complete): Created bystart.shafter successful environment setup. The monitor waits for this file before beginning checks. - Restart Token (
/tmp/.mt5-restart-requested): Created by the monitor when a service fails health checks. The service manager watches for this file to trigger a restart.
Sources: docker/container/metatrader/health_monitor.sh:11-15, docker/container/metatrader/start.sh:82-88
Health Check Logic Flow
The following diagram illustrates the sequential checks performed by the monitoring daemon during each interval.
The health monitor validates three critical states: the existence of the MT5 process, the availability of the listener port, and the ability of the bridge to respond to new TCP connections.
Sources: docker/container/metatrader/health_monitor.sh:78-135
Auto-Recovery Mechanisms
When a failure is detected, the system initiates a "full restart" rather than a container exit, allowing for faster recovery without losing the entire container context.
Recovery Actions
The svc-mt5server service (managed by s6-overlay) performs the following recovery sequence:
- Kill MT5: Terminating
terminal64.exeand any hanging Wine processes. - Kill Bridge: Terminating the Python bridge process.
- Start MT5: Relaunching the terminal within a Wine virtual desktop (
explorer /desktop=). - Start Bridge: Relaunching the gRPC/RPyC server.
- Clear Token: Removing the
/tmp/.mt5-restart-requestedfile to resume monitoring.
Sources: tests/test_static.py:535-546, docker/container/metatrader/health_monitor.sh:5-6
Configuration Parameters
| Variable | Default | Purpose |
|---|---|---|
AUTO_RECOVERY_ENABLED | 1 | Global toggle for the recovery system. |
HEALTH_CHECK_INTERVAL | 30 | Seconds between consecutive health checks. |
FAILURE_THRESHOLD | 3 | Number of consecutive failures before triggering a restart. |
STARTUP_WAIT_MAX | 300 | Maximum seconds to wait for initial setup before timing out. |
Sources: docker/container/metatrader/health_monitor.sh:22-30, docker/compose.yaml:57-58
System Health Integration
Beyond the internal monitor, the project integrates with Docker's native healthcheck system and provides gRPC endpoints for external monitoring tools.
Docker Healthcheck
The compose.yaml defines a container-level healthcheck that validates the gRPC bridge accessibility from the host perspective.
healthcheck:
test: ["CMD", "bash", "-c", "echo >/dev/tcp/localhost/8001"]
interval: 30s
timeout: 10s
retries: 3
start_period: 900s
Sources: docker/compose.yaml:93-102
gRPC HealthCheck Endpoint
The gRPC bridge implements a HealthCheck RPC that returns detailed status regarding the MetaTrader 5 connection and trade permissions.
message HealthStatus {
bool healthy = 1;
bool mt5_available = 2;
bool connected = 3;
bool trade_allowed = 4;
int32 build = 5;
string reason = 6;
}
Sources: docker/container/metatrader/mt5.proto:98-105
Implementation Details
Process and Network Verification
The monitor uses lightweight Linux utilities to verify health without requiring heavy dependencies like netstat.
- Process Check: Uses
pgrep -f "terminal64.exe"to verify the terminal is active. - Port Check: Uses
ss -tulnto check if the bridge port (default 8001) is in aLISTENstate. - Connection Check: Uses
nc -z localhost(netcat) to verify the port accepts connections.
Sources: docker/container/metatrader/health_monitor.sh:42-55
Monitoring Service Sequence
The following sequence diagram shows the interaction between the startup process, the monitor, and the service supervisor.
Conclusion
The Health Monitoring & Auto-Recovery system provides a robust fail-safe for running MetaTrader 5 in a containerized environment. By combining process-level checks, network validation, and coordinated service restarts via s6-overlay, the system maintains high availability for automated trading operations while requiring minimal manual intervention.