Certificate Validation

Relevant source files

The following files were used as context for generating this wiki page:

• src/tcp.c
• src/relp.h
• src/relpsrv.c
• tests/receive.c
• tests/send.c
• tests/tls-basic-vg.sh
• tests/test-framework.sh

Certificate validation in librelp is a critical security component that ensures the authenticity and integrity of peers during a TLS-encrypted RELP session. The library supports multiple authentication modes and abstracts the underlying TLS implementation, allowing the use of either GnuTLS or OpenSSL for certificate verification and handshake processing.

The validation process involves verifying the certificate chain against trusted Certificate Authorities (CAs), checking expiration dates, and optionally matching the peer's identity against configured permitted peers using names or fingerprints.

Authentication Modes

Librelp defines several authentication modes that determine how strictly a peer's certificate is validated. These modes are configured through the relpAuthMode_t enumeration.

Mode	Description
None	Anonymous authentication; no certificate is required or validated.
CertValid	Validates that the certificate is signed by a trusted CA and is not expired.
Name	Performs standard validation and ensures the Subject Alternative Name (SAN) or Common Name (CN) matches a permitted peer list.
Fingerprint	Validates the certificate and ensures its SHA-1 hash matches a specific authorized fingerprint.

Sources: src/tcp.c:308-316, src/relpsrv.c:191-201, tests/receive.c:459-470

Validation Logic and Data Flow

The validation logic is primarily implemented in src/tcp.c. It follows a structured path starting from the TLS handshake to specific peer identity checks.

This diagram illustrates the decision flow during the peer authentication phase following a successful TLS handshake.
Sources: src/tcp.c:1015-1045, src/tcp.c:1571-1588

GnuTLS Implementation

When using GnuTLS, the library uses gnutls_certificate_set_verify_function to register relpTcpVerifyCertificateCallback. This callback performs the following:

1. Imports the peer certificate using gnutls_x509_crt_import.
2. Checks expiration and activation dates via standard GnuTLS verification.
3. If mode is Fingerprint, it calls relpTcpChkPeerFingerprint.
4. If mode is Name, it calls relpTcpChkPeerName_gtls.

Sources: src/tcp.c:1568-1616, src/tcp.c:1669-1681

OpenSSL Implementation

In OpenSSL, the library utilizes SSL_CTX_set_verify with a verify_callback. Post-handshake checks are handled in relpTcpChkPeerAuth:

1. Retrieves the peer certificate with SSL_get_peer_certificate.
2. Verifies the result using SSL_get_verify_result.
3. Handles X509_V_ERR_CERT_HAS_EXPIRED as a specific authentication error.
4. Performs name or fingerprint matching using relpTcpChkPeerName_ossl or relpTcpChkPeerFingerprint_ossl.

Sources: src/tcp.c:980-1011, src/tcp.c:1015-1055

Peer Identity Verification

Identity verification ensures that even if a certificate is validly signed, it belongs to an authorized entity.

Name Validation (SAN and CN)

The library extracts identity information from the certificate in a specific order:

1. Subject Alternative Name (SAN): The library iterates through DNS entries in the SAN extension.
2. Common Name (CN): If no match is found in the SAN, it extracts the CN from the Distinguished Name (DN).

Wildcard matching is supported for name-based authentication, allowing patterns like *.example.com or host*.example.com.
Sources: src/tcp.c:1473-1533, src/tcp.c:1775-1851

Fingerprint Validation

For the Fingerprint mode, the library generates a printable SHA-1 string of the certificate. It then compares this string against the list of permitted peers provided in the configuration.
Sources: src/tcp.c:860-900, src/tcp.c:1400-1450

Configuration and API

Developers interact with the validation system through several API functions to set paths for CA certificates and define authorized peers.

Function	Purpose
relpSrvSetCACert	Sets the path to the CA certificate file for the server.
relpSrvAddPermittedPeer	Adds a hostname or fingerprint to the authorized list.
relpSrvSetAuthMode	Configures the verification level (None, Name, Fingerprint, etc.).
relpCltSetOwnCert	Sets the client's own certificate for mutual authentication.

Sources: src/relpsrv.c:138-158, src/relpsrv.c:235-245, tests/receive.c:560-575

Sequence of Initialization

The following sequence diagram shows how a RELP server initializes its TLS and validation context.

Error Handling

Authentication failures are reported via a dedicated callback onAuthErr. Common errors include:

• RELP_RET_AUTH_NO_CERT: Peer did not provide a certificate.
• RELP_RET_AUTH_CERT_INVL: Certificate is expired or signature is invalid.
• RELP_RET_AUTH_ERR_NAME: Certificate is valid but the name does not match any permitted peer.
• RELP_RET_AUTH_ERR_FP: Certificate is valid but the fingerprint does not match.

Sources: src/tcp.c:102-110, src/tcp.c:1610-1615, tests/receive.c:163-171

Conclusion

Certificate validation in librelp provides a robust layer of security by integrating deep certificate inspection into the RELP protocol. By supporting both GnuTLS and OpenSSL and offering flexible authentication modes like fingerprint and wildcard name matching, it allows for secure log transmission across untrusted networks while maintaining compatibility with various infrastructure requirements.