Test Suite
Relevant source files
The following files were used as context for generating this wiki page:
The librelp test suite is a comprehensive validation framework designed to ensure the reliability and security of the Reliable Event Logging Protocol (RELP) implementation. It encompasses automated shell-based test drivers, compiled C utilities for traffic generation/reception, and continuous integration (CI) scripts that execute tests across multiple compilers and TLS libraries.
The suite focuses on functional correctness, protocol compliance, memory safety via Valgrind, and security through various TLS configuration scenarios. It utilizes a modular "test-framework" to provide common environment setup, port management, and process control for all individual test cases.
Sources: tests/test-framework.sh:1-5, devtools/run-ci.sh:1-10, tests/tls-basic-vg.sh:1-10
Core Architecture and Components
The test suite architecture relies on a client-server model where specialized C programs simulate RELP entities, managed by bash scripts that verify outcomes.
Testing Utilities
The suite employs three primary compiled C programs to facilitate testing:
- receive.c: A RELP receiver that listens on a specified port, writes received messages to an output file, and supports TLS configurations, PID file management, and watchdog timers.
- send.c: A RELP sender capable of transmitting single messages or large bursts of numbered messages to verify sequence and throughput.
- chkseq.c: A sequence verification tool that checks if an output file contains strictly monotonically increasing numbers, identifying duplicates or missing messages.
Sources: tests/receive.c:29-50, tests/send.c:29-50, tests/chkseq.c:1-25
Test Framework (test-framework.sh)
This shell script acts as the middleware for the test suite, providing functions to manage the test lifecycle:
The diagram above shows the standard lifecycle of a librelp test case.
Sources: tests/test-framework.sh:22-192
Process Management Functions
| Function | Description |
|---|---|
get_free_port() | Uses Python to bind and release a socket to find an available TCP port for testing. |
startup_receiver() | Launches the receive process in the background and waits for its PID file. |
stop_receiver() | Sends a termination signal to the receiver and waits for exit code. |
check_output() | Greps the output log for expected strings to verify functional success. |
check_msg_count() | Uses chkseq to verify the integrity and count of received messages. |
Sources: tests/test-framework.sh:24-180
Execution and CI Integration
The test suite is designed for integration into CI pipelines (such as Travis CI or custom buildbots) to validate the library against different environments.
CI Workflow
The run-ci.sh and travis-run-compile-tests.sh scripts automate the build and test process across different configurations:
- Environment Setup: Configures compilers (gcc-9, clang-9) and TLS libraries (GnuTLS, OpenSSL).
- Static Analysis: Runs static analyzers to catch potential bugs before execution.
- Compilation: Executes
make checkto run the test suite. - Log Aggregation: Uses
gather-check-logs.shto extract and truncate logs from failed.trsfiles for reporting.
The sequence diagram illustrates how CI scripts orchestrate the build and log gathering process.
Sources: devtools/run-ci.sh:23-50, devtools/gather-check-logs.sh:45-60, devtools/travis-run-compile-tests.sh:20-50
Protocol and Security Testing
TLS Verification
The suite includes specific tests for TLS functionality, testing different authentication modes and cipher suites.
- Authentication Modes: Tests
none,fingerprint,name, andcertvalid. - Library Switching: The
do_tls_subtestsfunction iterates tests over bothgnutlsandopensslif available. - OpenSSL Command Strings: Tests like
tls-basic-tlscommand.shverify advanced configuration via the-cflag for min/max protocol versions and specific cipher strings.
Sources: tests/test-framework.sh:196-208, tests/tls-basic-vg.sh:18-28, tests/tls-basic-tlscommand.sh:9-30
Memory Safety (Valgrind)
Tests often run under Valgrind to detect memory leaks or illegal accesses. The framework defines a valgrind variable with specific flags (--malloc-fill=ff --free-fill=fe) to catch uninitialized memory usage.
Sources: tests/test-framework.sh:9-11, tests/tls-basic-vg.sh:3-10
Data Verification Logic
The chkseq.c utility is critical for verifying reliability over unstable sessions. It parses log files where each line is expected to be a number.
| Option | Type | Description |
|---|---|---|
-s | Integer | Starting sequence number (default 0). |
-e | Integer | Ending sequence number. |
-d | Flag | Permits duplicate messages (useful for tests simulating retries). |
-m | Integer | Number of permitted missing messages for failover testing. |
-i | Integer | Expected increment between messages (default 1). |
Sources: tests/chkseq.c:5-25, tests/chkseq.c:45-75
Conclusion
The librelp Test Suite provides a robust framework for validating the library's primary goal: reliable message delivery. By combining low-level C utilities for precise protocol interaction with high-level shell scripts for environment orchestration, it ensures that changes to the core engine or TLS implementations do not introduce regressions in protocol state or memory safety.