Reliability & Acknowledgements
Relevant source files
The following files were used as context for generating this wiki page:
The Reliable Event Logging Protocol (RELP) is designed to ensure that no messages are lost during transmission, even in the event of connection interruptions or server restarts. The core mechanism achieving this reliability is the application-level acknowledgement system, where every command issued by a client must be explicitly acknowledged by the server before it is considered successful.
Reliability in librelp is centered around the concept of a "RELP Transaction," which consists of a command and its corresponding response (rsp). Each transaction is tracked using a unique, monotonically increasing Transaction Number (TXNR).
Sources: doc/relp.html:10-25, doc/relp.html:64-67
Core Transaction Logic
RELP employs a command-response model. While the protocol supports full-duplex communication where multiple commands can be issued simultaneously, the server is required to respond to these commands in the exact order they were received to maintain synchronization.
The Transaction Number (TXNR)
The TXNR is the primary identifier for reliability. It starts at 1 and increases monotonically up to 999,999,999, after which it wraps back to 1. TXNR 0 is strictly reserved for "hint" commands (out-of-band messages) that do not require an acknowledgement.
Sources: doc/relp.html:20-25, doc/relp.html:104-108, src/relp.h:207-208
Message Acknowledgement Flow
When a client sends a syslog command, the message is not considered "delivered" until the server returns a rsp command containing the matching TXNR and a success code (200 OK). If the TCP connection is terminated before this response is received, the client considers the command unsuccessful and will attempt to re-send it upon session recovery.
The diagram above illustrates the synchronous nature of acknowledgements and how a connection drop affects unacknowledged messages.
Sources: doc/relp.html:70-75, doc/relp.html:208-220
Implementation of Acknowledgements
The library implements acknowledgements through specific command handlers and data structures that track unacknowledged frames.
Response Command Handler (src/scrsp.c)
The rsp command handler is responsible for processing server replies. It parses the response header, which includes a 3-digit status code and a human-readable message.
- Success (200 OK): The library locates the unacknowledged buffer (
relpSendbuf_t) associated with the TXNR. If a response handler callback is defined, it is executed, and the buffer is destructed, effectively clearing the message from the client's retry queue. - Error (e.g., 500): If the server returns an error code, the library triggers the
onErrcallback to notify the application.
Sources: src/scrsp.c:71-118, src/librelp.h:134-140
Server-Side Processing (src/scsyslog.c)
On the server side, the syslog command handler receives the message and invokes the registered application callback (onSyslogRcv). Only after the callback has been executed does the server send the 200 OK response. This ensures that the client only receives an acknowledgement after the server-side application has taken responsibility for the data.
Sources: src/scsyslog.c:56-83
Reliability Parameters & Windowing
To prevent resource exhaustion while maintaining high throughput, librelp utilizes a sliding window mechanism.
| Feature | Description | Default Value |
|---|---|---|
| Window Size | Limits the number of outstanding (unacknowledged) commands. | 128 frames |
| Max Data Size | The maximum size permitted for a single RELP frame. | 128 KB |
| TXNR Range | The range of transaction IDs before wrapping. | 1 - 999,999,999 |
| Oversize Mode | Action taken when a frame exceeds Max Data Size (Abort, Truncate, or Accept). | Abort |
Sources: src/relp.h:185-195, doc/relp.html:101-103, src/librelp.h:166-170
Error Handling and Recovery
Reliability is maintained during failures through strict connection management:
- Connection Closure: If a peer detects a protocol error (e.g., invalid framing), it must close the TCP connection immediately.
- Session Recovery: Upon reconnecting, the client re-sends all messages that were in its "unacknowledged" window at the time of the crash. This can lead to message duplication if the server processed the message but the
rspwas lost, but it prevents message loss. - Timeouts: Clients can set both command timeouts and connection timeouts to detect unresponsive servers.
Sources: doc/relp.html:56-61, doc/relp.html:238-250, tests/send.c:132-150, src/librelp.h:207-209
Conclusion
The reliability of librelp is built upon application-level acknowledgements and unique transaction numbering. By ensuring that the server only acknowledges a message after successful processing and requiring the client to track these acknowledgements within a defined window, the protocol provides a robust mechanism for event logging that survives network and application-level failures.
Sources: doc/relp.html:10-30, src/scrsp.c:100-110