Debugging & Troubleshooting

Relevant source files

The following files were used as context for generating this wiki page:

• src/relp.h
• src/librelp.h
• src/tcp.c
• tests/receive.c
• tests/send.c
• tests/test-framework.sh
• devtools/gather-check-logs.sh
• tests/basic-sessionbreak-vg.sh

Debugging and troubleshooting in librelp is centered around an engine-driven event and error handling architecture. The library provides hooks for applications to capture internal debug messages, authentication errors, and generic protocol failures. This system is designed to provide visibility into the RELP (Reliable Event Logging Protocol) state machine, TLS handshake processes, and socket-level interactions.

The infrastructure supports various diagnostic layers, including application-level callbacks, detailed internal debug printing, and integration with external tools like Valgrind for memory analysis and automated test log gathering for Continuous Integration (CI) environments.

Error Handling Architecture

The librelp engine utilizes a callback-based system to report errors to the calling application. Developers can register specific functions to handle different categories of issues, ensuring that protocol-level errors or authentication failures are captured and logged according to the application's needs.

Callback Registration

Applications register debugging and error handlers during the construction of the RELP engine. The following table describes the primary registration functions available in the API:

Function	Purpose	Description
relpEngineSetDbgprint	Debug Output	Sets a printf-style function for internal library trace messages.
relpEngineSetOnErr	Object Errors	Callback for errors related to specific RELP objects (e.g., sessions, servers).
relpEngineSetOnGenericErr	Engine Errors	Callback for general library errors not tied to a specific object.
relpEngineSetOnAuthErr	Auth Failures	Specifically handles TLS authentication and certificate validation errors.

Sources: src/librelp.h:138-150, tests/receive.c:266-269

Internal Error Propagation

Internal functions use the relpRetVal type to propagate status codes. Macros like CHKRet and ABORT_FINALIZE are used throughout the codebase to ensure consistent error checking and resource cleanup.

The diagram shows how internal failures trigger specific application callbacks based on the context of the error.
Sources: src/relp.h:176-189, src/tcp.c:64-84

Protocol & TLS Troubleshooting

Troubleshooting connectivity often involves diagnosing TLS handshake failures or protocol framing issues. Librelp supports both GnuTLS and OpenSSL, each providing specific error reporting mechanisms.

TLS Diagnostics

When TLS is enabled, the library captures library-specific error codes and translates them into human-readable strings via internal debug printing. For OpenSSL, the library implements a relpTcpLastSSLErrorMsg helper to dump the OpenSSL error stack to the debug output.

Key diagnostic points during TLS operations include:

• Handshake Retries: Internal logging indicates when gnutls_handshake or SSL_do_handshake requires a retry (e.g., GNUTLS_E_AGAIN).
• Certificate Validation: Errors during certificate path validation or fingerprint mismatches trigger the onAuthErr callback with specific details like "non-permitted fingerprint" or "certificate expired".

Sources: src/tcp.c:284-315, src/tcp.c:1168-1175, src/tcp.c:1340-1355

Connection State Monitoring

The library tracks retry operations using the rtryOp state in the TCP object. This allows developers to see if the library is currently waiting on a handshake, a receive operation, or a send operation to complete after a blocking-like condition.

State Constant	Meaning
relpTCP_RETRY_none	No retry pending.
relpTCP_RETRY_handshake	Waiting for TLS handshake completion.
relpTCP_RETRY_recv	Waiting for data to become available for reading.
relpTCP_RETRY_send	Waiting for the socket to become writable.

Sources: src/tcp.c:750-760, src/tcp.c:1186-1191

Automated Diagnostic Tools

Librelp includes a robust testing framework designed to surface bugs and memory issues automatically.

Memory & Leak Detection

The test suite utilizes Valgrind to identify memory leaks and invalid memory accesses. Tests are configured to run under Valgrind with specific flags to catch hard-to-find issues, such as uninitialized memory use or use-after-free.

# Example Valgrind configuration used in tests
valgrind --malloc-fill=ff --free-fill=fe --leak-check=full --error-exitcode=10

Sources: tests/test-framework.sh:8-10, tests/basic-sessionbreak-vg.sh:20-20

Log Gathering & CI Diagnostics

In Continuous Integration environments, the gather-check-logs.sh script is used to collect and summarize test failures. It searches for .trs (test result) and .log files, extracting relevant failure snippets for quick analysis.

• Incomplete Logs: The system detects tests that aborted without producing a result file, indicating a crash or hang.
• Watchdog Timers: Test binaries like receive include watchdog timers that force termination if a process hangs for longer than the expected timeout, preventing CI pipelines from stalling.

Sources: devtools/gather-check-logs.sh:7-25, tests/receive.c:85-92

Sequence Checking

For verifying protocol reliability, the chkseq tool is used to ensure that messages are received in a strictly monotonically increasing order without gaps or duplicates (unless specifically permitted).

Sources: tests/chkseq.c:11-30

Common Error Codes

The library defines a set of standard return values used for troubleshooting API and protocol issues.

Constant	Value	Description
RELP_RET_OK	0	Success.
RELP_RET_SESSION_BROKEN	10007	The session was terminated unexpectedly.
RELP_RET_IO_ERR	10014	A low-level socket or system I/O error occurred.
RELP_RET_TIMED_OUT	10015	An operation (like connection) timed out.
RELP_RET_AUTH_CERT_INVL	10036	Peer certificate failed validation.

Sources: src/librelp.h:92-130

Summary

Debugging librelp requires leveraging the engine's callback system (onErr, onAuthErr, onGenericErr) and enabling the debug printer via relpEngineSetDbgprint. These mechanisms provide a detailed trace of the protocol's internal state, TLS handshakes, and I/O operations. Combined with the automated sequence checking and memory analysis tools provided in the test suite, developers can effectively isolate issues ranging from network timeouts to complex certificate authentication failures.