TCP Transport Layer

Relevant source files

The following files were used as context for generating this wiki page:

• src/tcp.c
• src/tcp.h
• src/relp.h
• src/librelp.h
• src/relpsess.h
• doc/relp.html

The TCP Transport Layer in librelp provides the fundamental communication mechanism for the Reliable Event Logging Protocol (RELP). It handles the mapping of RELP frames onto TCP streams, managing socket lifecycles, connection establishment for both clients and servers, and reliable data transmission. This layer abstracts the underlying operating system socket API to provide a consistent interface for higher-level RELP session management.

Within the project architecture, the TCP layer is responsible for low-level byte stream handling, including non-blocking I/O operations and integration with TLS libraries (GnuTLS or OpenSSL) to provide secure transport. It ensures that the "Reliable" aspect of RELP is maintained at the network level by handling connection resets, timeouts, and persistent retry logic for cryptographic handshakes and data transfers.

Sources: doc/relp.html:13-35, src/tcp.c:38-60

Architecture and Components

The TCP transport is centered around the relpTcp_t object, which encapsulates all state necessary for a single TCP connection or a set of listening sockets.

The relpTcp_t Object

This structure is the core data container for the transport layer. It maintains references to the parent relpEngine_t, the associated relpSrv_t (for server connections) or relpClt_t (for client connections), and the raw file descriptors.

Field	Description
sock	The primary socket used for single-socket operations.
socks	An array of sockets used by listeners; index 0 contains the count.
pRemHostIP	The IP address of the remote peer.
bTLSActive	Boolean flag indicating if TLS is currently engaged.
rtryOp	State variable for managing non-blocking operation retries.

Sources: src/tcp.h:87-133, src/tcp.c:485-525

Component Relationship

The following diagram illustrates how the TCP transport layer sits between the high-level RELP engine and the system's network stack.

The transport layer can operate either in plain TCP mode or wrapped with a TLS provider.
Sources: src/relp.h:116-160, src/tcp.c:1660-1685

Connection Lifecycle

Server Listener Initialization

Servers initialize listeners using relpTcpLstnInit. This function performs DNS resolution for the binding address, creates sockets for each address family (IPv4/IPv6), sets socket options like SO_REUSEADDR and O_NONBLOCK, and begins the listen() call.

Client Connection Process

Clients establish connections through relpTcpConnect. It uses a non-blocking connect() call followed by a poll() to wait for the socket to become writable within a specified timeout.

1. Resolution: Calls getaddrinfo for the target host and port.
2. Socket Creation: Initializes the socket with the appropriate family.
3. Non-blocking Connect: Initiates connection and handles EINPROGRESS.
4. Poll: Waits for the socket to be ready or timeout.
5. TLS Upgrade: If enabled, initiates the TLS handshake immediately after the TCP connection is successful.

Sources: src/tcp.c:1688-1773, src/tcp.h:154-156

Secure Transport (TLS)

The TCP layer provides an abstraction for multiple TLS backends. The implementation supports both GnuTLS and OpenSSL.

TLS Abstraction Layer

The library uses a set of generic functions that dispatch to library-specific implementations based on pEngine->tls_lib.

• Handshake Retries: Because sockets are non-blocking, functions like relpTcpRtryHandshake manage the WANT_READ or WANT_WRITE states returned by the underlying crypto libraries.
• Authentication Modes: Supports None (Anonymous), Fingerprint, Name (Subject Alt Name/CN validation), and CertValid.

Function	GnuTLS Implementation	OpenSSL Implementation
relpTcpSend	gnutls_record_send	SSL_write
relpTcpRcv	gnutls_record_recv	SSL_read
relpTcpHandshake	gnutls_handshake	SSL_do_handshake / SSL_accept

Sources: src/tcp.c:537-550, src/tcp.c:1010-1055, src/tcp.h:73-85

Certificate Verification Flow

Verification occurs during the handshake. For GnuTLS, a callback relpTcpVerifyCertificateCallback is used. For OpenSSL, the verify_callback function handles result checking.

Data Transmission

The transport layer exposes relpTcpSend and relpTcpRcv for frame exchange.

Socket Optimization (CORK/NOPUSH)

To optimize network performance during large data bursts, the transport layer provides hints to the kernel using relpTcpHintBurstBegin and relpTcpHintBurstEnd. These manipulate the TCP_CORK (Linux) or TCP_NOPUSH (BSD/AIX) socket options to aggregate small packets into larger segments.
Sources: src/tcp.c:1571-1594

Keep-Alive Management

The layer implements TCP keep-alives to detect "half-open" connections where the peer has crashed or the network path is broken. This is configured via EnableKeepAlive, which sets SO_KEEPALIVE, TCP_KEEPCNT, TCP_KEEPIDLE, and TCP_KEEPINTVL.
Sources: src/tcp.c:1058-1123

Error Handling and Retries

The TCP layer utilizes a specific retry state machine to handle the complexities of non-blocking I/O, especially when combined with TLS.

State	Constant	Description
No Retry	relpTCP_RETRY_none	Normal operation.
Handshake	relpTCP_RETRY_handshake	Waiting for TLS handshake completion.
Receive	relpTCP_RETRY_recv	Waiting for data to be available to read.
Send	relpTCP_RETRY_send	Waiting for the socket buffer to accept more data.

When an I/O operation returns EAGAIN or EWOULDBLOCK, the rtryOp is set. The RELP engine's event loop (typically using epoll or select) monitors the socket until the required condition (read/write) is met, at which point the operation is resumed.

Sources: src/tcp.h:43-46, src/tcp.c:1520-1560

Conclusion

The TCP Transport Layer is the backbone of librelp, providing the necessary reliability and security abstractions over standard internet protocols. By handling the intricacies of non-blocking sockets and providing a unified interface for multiple TLS libraries, it allows the RELP protocol to remain focus on high-level message delivery guarantees while ensuring efficient network utilization.