Troubleshooting Connection Problems
Relevant source files
The following files were used as context for generating this wiki page:
This document outlines the connection, monitoring, and automated recovery mechanisms within the Pulsar Dome Driver. Understanding these systems is crucial for diagnosing and resolving communication issues with the dome controller. The driver employs a multi-layered approach, including serial port management, continuous status polling, and several watchdog systems designed to detect and recover from stalls, timeouts, or hardware errors.
The primary connection is established over a serial (COM) port. The driver's stability relies on a polling loop to get status updates and a system watchdog to ensure the polling loop itself is running. For long-running actions like slewing or opening the shutter, a separate ActionWatchdog is used. If any of these mechanisms detect a problem, an automated reset routine can be triggered to attempt recovery, ranging from a soft (command-based) reset to a hard (power-cycling) reset.
Initial Connection
The connection process is initiated when the Connected property in DomeDriver is set to true. It involves selecting the correct serial port, opening it, and verifying communication with the controller.
Sources: Driver/Pulsar_DomeDriver.cs:220-394, UI/SettingsForm.cs:25-45, Helper/SerialPortGuard.cs
Connection Flow
The ConnectController method orchestrates the connection sequence. It ensures a serial port is configured, opens the port, creates a SerialPortGuard to manage access, and then pings the controller to confirm it's responsive.
The following diagram illustrates the initial connection sequence.
Key Components
| Component | Description | Source File |
|---|---|---|
DomeDriver.ConnectController() | The main method that handles opening the serial port and verifying the connection. It performs up to two ping attempts before failing. | Driver/Pulsar_DomeDriver.cs:273-352 |
DomeDriver.PingController() | Sends a "PULSAR" command and expects a specific response ("Y519") to verify the controller is alive and ready. | Driver/Pulsar_DomeDriver.cs:354-365 |
SerialPortGuard | A helper class that wraps the SerialPort object. It uses a SemaphoreSlim to ensure that only one command is sent at a time, preventing race conditions and garbled communication. | Helper/SerialPortGuard.cs |
SettingsForm | The UI where the user selects the COM port. If the configured port is unavailable, it is noted in the dropdown. | UI/SettingsForm.cs:25-45 |
Communication and Monitoring
Once connected, the driver relies on a polling loop and a system watchdog to continuously monitor the dome's status and the health of the driver's own processes.
Sources: Driver/Pulsar_DomeDriver.cs:466-773
Serial Communication via SerialPortGuard
All commands to the dome controller are funneled through the SerialPortGuard.Send() method. This is a critical component for stability.
- It uses a
SemaphoreSlimto act as a gate, allowing only one thread to access the serial port at a time. - Before sending a command, it can flush the input buffer to discard stale data.
- It writes the command followed by a carriage return (
\r). - It waits for a response with a 750ms timeout. If no response is received, an
IOExceptionis thrown.
// Simplified logic from SerialPortGuard.cs
public string? Send(string command, bool expectResponse = true)
{
_gate.Wait(); // Acquire lock
_setBusy?.Invoke();
try
{
// ... flush buffers ...
_port.Write($"{command}\r");
// ... read response with timeout ...
return raw;
}
finally
{
_clearBusy?.Invoke();
_gate.Release(); // Release lock
}
}
Sources: Helper/SerialPortGuard.cs:33-87
Polling Loop and System Watchdog
The driver maintains connection health using two cooperating background tasks: a polling loop and a system watchdog.
- Polling Loop (
PollLoopAsync): This loop runs continuously, callingDomeStatus()at a configurable interval (_pollingIntervalMs).DomeStatus()sends commands to the controller to get the current azimuth, shutter state, etc. The timestamp_lastPollTimestampis updated on every successful poll. - System Watchdog (
StartWatchdog): This separate task checks periodically (every 5 seconds) to see if the polling loop has stalled. It does this by comparing the current time to_lastPollTimestamp. If the time elapsed exceeds a threshold (10 seconds), it assumes the polling loop is stuck and restarts it by callingStopPolling()followed byStartPolling().
This relationship ensures that if the communication thread hangs, the driver will automatically attempt to recover it.
The following diagram shows the monitoring and recovery flow between the watchdog and the polling loop.
Automated Reset Routines
If communication cannot be re-established or a command fails to complete, the driver can trigger an automated reset routine. This is the most powerful recovery mechanism and can be configured to perform a soft or hard reset.
Sources: Driver/Pulsar_DomeDriver.cs:1003-1349, Helper/ActionWatchdog.cs:100-120
Reset Triggers
A reset is typically triggered under these conditions:
- An
ActionWatchdogtimes out waiting for a command (e.g.,OpenShutter) to complete. - A command sent to the controller is not acknowledged (e.g.,
AbortSlewfails). - A user manually triggers it via an ASCOM
Action.
Sources: Helper/ActionWatchdog.cs:108-112, Driver/Pulsar_DomeDriver.cs:1757-1763, Driver/Pulsar_DomeDriver.cs:1865-1875
Reset Logic Flow
The ResetRoutineAsync method manages the entire process. It first attempts a soft reset if enabled. If that fails or is disabled, it proceeds to a hard reset if enabled.
Soft Reset
A soft reset is attempted by sending the RESTART command to the controller. The driver then monitors the shutter status; a status of 6 ("Not Fitted") is interpreted as a sign that the controller is rebooting. It then waits for the shutter status to return to a normal value (0-3) before considering the reset successful.
Sources: Driver/Pulsar_DomeDriver.cs:1169-1196
Hard Reset
A hard reset is a more forceful recovery method that power-cycles the controller using an external tool.
- Disconnect: The driver closes the serial port.
- Power Off: It executes an external program (
ResetExe) with specified "Off" parameters (ResetOffParameters). - Wait: It pauses for a
CycleDelayperiod. - Power On: It executes the same program with "On" parameters (
ResetOnParameters). - Wait: It pauses for a
ResetDelayto allow the controller to boot. - Reconnect: It attempts to reconnect and ping the controller.
This entire process is configurable through the driver's settings dialog.
Sources: Driver/Pulsar_DomeDriver.cs:1198-1250, UI/SettingsForm.cs:81-96
Configuration for Connection Troubleshooting
Many connection issues can be traced back to incorrect settings. These are configured in the ASCOM setup dialog for the driver.
Sources: Config/ConfigManager.cs, UI/SettingsForm.cs, UI/SettingsForm.Designer.cs
| Setting | UI Control | Description | Source File |
|---|---|---|---|
SerialPort | comboBoxSerialPort | The COM port used to communicate with the dome controller. | Config/ConfigManager.cs:91-95 |
DebugLog / TraceLog | chkBoxDebugLog / chkBoxTraceLog | Enables verbose logging, which is essential for diagnosing issues. Logs are written to the LogLocation. | Config/ConfigManager.cs:219-243 |
SoftReset | checkBoxInternalReset | Enables the software-based RESTART command as part of the recovery routine. | Config/ConfigManager.cs:97-108 |
HardReset | checkBoxExternalReset | Enables the hardware-based power-cycle as part of the recovery routine. | Config/ConfigManager.cs:110-121 |
ResetExe | txtBoxExternalExe | The full path to the executable used for the hard reset (e.g., a command-line utility for a USB relay). | Config/ConfigManager.cs:135-139 |
ResetOnParameters | txtBoxOnParameters | Command-line arguments to pass to ResetExe to turn the controller ON. | Config/ConfigManager.cs:141-145 |
ResetOffParameters | txtBoxOffParameters | Command-line arguments to pass to ResetExe to turn the controller OFF. | Config/ConfigManager.cs:147-151 |
ResetDelay | txtBoxResetDelay | Time (in seconds) to wait after turning the controller ON before attempting to reconnect. | Config/ConfigManager.cs:153-164 |
CycleDelay | txtBoxCycleDelay | Time (in seconds) to wait between turning the controller OFF and turning it back ON. | Config/ConfigManager.cs:166-177 |
Logging and Diagnostics
The driver provides several mechanisms for diagnostics.
Sources: Diagnostics/FileLogger.cs, MQTT/MqttPublisher.cs, [Driver/Pulsar_DomeDriver.cs:86-93]
- File Logging: The
FileLoggerclass writes timestamped messages to a log file specified in the settings. Enabling "Debug Log" and "Trace Log" provides the most detailed information, including raw serial commands and responses. - MQTT: If enabled, the driver publishes status updates and alarms to an MQTT broker. This allows for remote monitoring of the driver's state. Key topics include:
Dome/DriverStatus: "Driver connected", "Driver disconnected"Dome/Watchdog: Heartbeat messages from the system watchdog.Dome/Alarm: Published when an unrecoverable error occurs or a reset is triggered.
Summary
To troubleshoot connection problems with the Pulsar Dome Driver, follow these steps:
- Check Basic Configuration: Ensure the correct COM Port is selected in the settings.
- Enable Logging: Turn on Debug Log and Trace Log in the settings and check the specified log file for errors. Look for messages like "Failed to open serial port", "No response received", or "Ping failed".
- Verify Physical Connection: Ensure the USB/serial cable is securely connected and that the dome controller has power.
- Review Reset Configuration: If the logs indicate repeated reset attempts, verify the Hard Reset settings. Use the "Test" buttons in the settings form to confirm that the external
ResetExecan successfully power-cycle the controller. - Monitor MQTT: If enabled, check the
Dome/Alarmtopic for any critical failure messages.
By understanding the interplay between the serial guard, polling loop, watchdogs, and reset routines, developers can effectively diagnose and resolve nearly any connection-related issue.