Range Transfer & Resumable Downloads

Relevant source files

The following files were used as context for generating this wiki page:

• documents/UseRangeTransferFile.md
• include/HXLibs/net/protocol/http/Response.hpp
• include/HXLibs/net/protocol/http/Request.hpp
• include/HXLibs/net/protocol/http/Http.hpp
• tests/server/04_http_file_server.cpp
• examples/benchmarks/01_server.cpp

The HXLibs HTTP framework implements Range Transfer to enable resumable downloads and multi-threaded file fetching. The system detects Range headers in client requests to serve specific byte segments of a file rather than the entire content. This feature reduces bandwidth consumption for interrupted transfers and allows download managers to accelerate downloads by requesting multiple file parts simultaneously.

Sources: documents/UseRangeTransferFile.md:3-5, include/HXLibs/net/protocol/http/Response.hpp:426-429

Core Architecture

The Range Transfer logic resides within the HttpResponse class, specifically through the useRangeTransferFile method. This method acts as a high-level handler that coordinates file I/O and HTTP protocol state based on the request metadata provided by the HttpRequest object.

Request Data Structures

The framework uses specific structures to pass range information between the request parser and the response generator.

Structure	Field	Description
RangeRequestView	reqType	The HTTP method (e.g., GET, HEAD).
RangeRequestView	reqHead	A reference to the internal HeaderHashMap containing request headers.
SseEvent (Partial)	id	Used for reconnection recovery in streaming contexts.

Sources: include/HXLibs/net/protocol/http/Http.hpp:123-127, include/HXLibs/net/protocol/http/Request.hpp:513-516

Transmission Logic Flow

When a file request arrives, the server evaluates whether the client requested a specific range, a header-only response, or a standard full-file transfer.

The diagram shows the decision logic used by the server to determine the HTTP status code and transmission method.
Sources: include/HXLibs/net/protocol/http/Response.hpp:435-510

Implementation Details

Single Range Requests

If the client provides a standard bytes=X-Y range, the server calculates the remaining bytes using the formula Y - X + 1. It sets the Content-Range header to bytes X-Y/Z (where Z is total file size) and updates the Content-Length to match the segment size. The AsyncFile utility then moves the file pointer to the offset X before streaming.

Sources: include/HXLibs/net/protocol/http/Response.hpp:458-485

Multi-part Range Requests

The framework supports multiple ranges in a single request (e.g., Range: bytes=0-10, 20-30). In this mode, the server:

1. Sets Content-Type to multipart/byteranges; boundary=BOUNDARY_STRING.
2. Iterates through each requested range.
3. Wraps each binary segment with a boundary delimiter and segment-specific metadata (Content-Range and Content-Length).
4. Appends a final boundary --BOUNDARY_STRING-- to terminate the response.

Sources: include/HXLibs/net/protocol/http/Response.hpp:511-558

Resumable Download Workflow

The system facilitates resumable downloads by maintaining compatibility with standard tools like wget, aria2, and IDM.

1. Interruption: The client terminates the connection after receiving N bytes.
2. Truncation: Local files may be truncated or stored as partial parts (e.g., .part).
3. Resumption: The client sends a new request with Range: bytes=N-.
4. Partial Content: The server responds with 206 Partial Content and starts transmission from offset N.

Sources: documents/UseRangeTransferFile.md:8-60

Key API Methods

Server-Side

The primary interface for developers to enable this feature is the useRangeTransferFile method within the endpoint handler.

// Example: Basic File Server Endpoint
server.addEndpoint<GET>("/getFile", [] ENDPOINT {
    co_await res.useRangeTransferFile(req.getRangeRequestView(), "path/to/file.ext");
});

Sources: tests/server/04_http_file_server.cpp:55-58, examples/benchmarks/01_server.cpp:41-43

Related Internal Functions

Function	File	Description
getRangeRequestView()	Request.hpp	Extracts the HTTP method and headers into a view for the response handler.
useChunkedEncodingTransferFile()	Response.hpp	Alternative transmission using Transfer-Encoding: chunked for dynamic files.
setOffset(uint64_t)	Response.hpp	Moves the AsyncFile cursor to the start of the requested range.
fullySend(std::string_view)	Response.hpp	Ensures the entire buffer or file segment is transmitted over the socket.

Sources: include/HXLibs/net/protocol/http/Response.hpp:400-410, include/HXLibs/net/protocol/http/Request.hpp:513-516

Performance Considerations

For static assets, the project supports a "Memory Variant" where files are pre-loaded into memory to bypass Disk I/O during benchmarks. However, for large files or standard file server usage, the "Disk Variant" uses the asynchronous useRangeTransferFile to prevent blocking the coroutine executor while waiting for disk operations.

Sources: benchmarks/http/servers/hxlibs_server.cpp:78-83, examples/benchmarks/01_server.cpp:35-55

Summary

The Range Transfer system in HXLibs provides a robust mechanism for handling large file downloads. By integrating AsyncFile with standard HTTP range headers, the framework supports partial content delivery, multi-part requests, and seamless integration with third-party download accelerators. Developers can activate these features by passing the RangeRequestView from the request to the file transfer method in the response.