HTTP Client & Connection Pooling

Relevant source files

The following files were used as context for generating this wiki page:

• include/HXLibs/net/client/HttpClient.hpp
• include/HXLibs/net/client/HttpClientPool.hpp
• include/HXLibs/net/protocol/http/Http.hpp
• tests/client/03_brokenWireAndReconnected.cpp
• tests/client/05_http_cli_pool.cpp
• tests/server/04_http_file_server.cpp

The HXLibs networking module provides a coroutine-based HTTP client architecture supporting standard requests, WebSocket upgrades, and automated connection management. It features an integrated connection pooling system to manage multiple concurrent client instances and optimize throughput in high-load scenarios.

HTTP Client Architecture

The HttpBaseClient class manages the lifecycle of HTTP connections using a coroutine-driven event loop. It handles socket creation, asynchronous I/O, and protocol-specific logic for HTTP, HTTPS, and WebSocket communications.

Core Components

• Event Loop Management: Each client uses either a unique or shared coroutine::EventLoop to schedule asynchronous tasks.
• Thread Safety: The client maintains an internal container::ThreadPool with a fixed thread count to manage task execution without blocking the main event loop.
• IO Abstraction: Supports both plain TCP (HttpIO) and TLS-encrypted (HttpsIO) connections via template specialization.

Sources: include/HXLibs/net/client/HttpClient.hpp:46-77, include/HXLibs/net/client/HttpClient.hpp:607-650

Connection Lifecycle

The client implements automatic reconnection logic. If a request fails due to a broken pipe or timeout, the client attempts to re-establish the socket before retrying the operation once.

The client manages socket state using a SocketFdType and performs asynchronous handshakes for Https and Proxies during the makeSocket phase.

Sources: include/HXLibs/net/client/HttpClient.hpp:89-91, include/HXLibs/net/client/HttpClient.hpp:426-476, tests/client/03_brokenWireAndReconnected.cpp:73-82

Request Types and Methods

The client supports various interaction models, ranging from standard GET/POST requests to streaming via WebSocket and SSE.

Standard and Chunked Requests

Clients can perform standard requests using the get, post, and request methods. For large file transfers, the uploadChunked method utilizes HTTP Chunked Transfer Encoding to stream data from disk.

Method	Type	Description
get / coGet	GET	Retrieves data from a specified URL.
post / coPost	POST	Sends data to a server, supporting various content types.
uploadChunked	Stream	Uploads a file using chunked encoding to minimize memory usage.
wsLoop	WS	Initiates a persistent WebSocket connection loop.
coSseLoop	SSE	Handles Server-Sent Events (SSE) streaming.

Sources: include/HXLibs/net/client/HttpClient.hpp:106-258, include/HXLibs/net/client/HttpClient.hpp:334-405, tests/server/04_http_file_server.cpp:39-44

WebSocket Integration

The wsLoop method provides a structured way to handle persistent bidirectional communication. It accepts a callback function that executes within the WebSocket context.

Connection Pooling

The HttpBaseClientPool class manages a collection of HttpClient instances to facilitate concurrent operations and load balancing across multiple connections.

Pooling Logic

• Initialization: The pool creates a fixed number of client instances during construction.
• Load Balancing: Uses a Round-Robin algorithm to select the next available client index for a request.
• Dynamic Resizing: Supports resizing the pool at runtime via the resize method.

Sources: include/HXLibs/net/client/HttpClientPool.hpp:38-51, include/HXLibs/net/client/HttpClientPool.hpp:64-77

Pool Operations

The pool wraps standard client methods (GET, POST, Request, WebSocket) and delegates them to a selected client instance.

// Example of using a pool for concurrent requests
HttpClientPool cliPool{2}; 
cliPool.get("http://127.0.0.1:28205/get").thenTry([](https://github.com/HengXin666/HXLibs/blob/23b9882b257fe12f3fc2e260ce7b65d11bc51e3c/auto%20t) {
    // Selection and request handled by pool index
});

Sources: include/HXLibs/net/client/HttpClientPool.hpp:86-103, tests/client/05_http_cli_pool.cpp:20-25

Technical Specifications

Header Management

The client automatically pre-processes headers for every request to ensure protocol compliance.

• Default Headers: Host, Accept (*/*), Connection (keep-alive), User-Agent (HXLibs/1.0), Content-Type, and Date.
• Case-Insensitivity: Headers are stored in a HeaderHashMap using TransparentCaseInsensitiveHash and Equal to handle case-insensitive HTTP header keys.

Sources: include/HXLibs/net/protocol/http/Http.hpp:52-94, include/HXLibs/net/client/HttpClient.hpp:579-595

Error Handling

The client uses the container::Try wrapper to propagate exceptions and results across coroutine boundaries.

• Timeout: Methods accept a Timeout parameter (defaulting to 30 seconds).
• Retry: The client performs a second attempt at connection and request if the initial pipe is found to be broken.

Sources: include/HXLibs/net/client/HttpClient.hpp:145-165, tests/client/03_brokenWireAndReconnected.cpp:75-80

HXLibs HTTP Client and Pooling systems provide a robust framework for asynchronous network communication, prioritizing resource efficiency through coroutines and optimized connection reuse.