Parsing Compliance Tests
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Parsing Compliance Tests in libfastjson serve as the primary mechanism for ensuring the library correctly handles valid JSON data, rejects malformed input, and maintains consistency across different platforms. The test suite is designed to validate the core tokenizer logic, handle edge cases such as large integers and special floating-point values (NaN, Infinity), and enforce strict compliance modes.
These tests are integrated into the project's Continuous Integration (CI) pipeline. All pull requests must pass these tests to be considered for merging, and developers are encouraged to add new tests when fixing bugs or adding functionality.
Sources: CONTRIBUTING.md:1-15, tests/test_parse.c:20-40
Core Testing Architecture
The testing framework utilizes a combination of C-based test programs and shell scripts to execute and verify parsing behavior. The primary goal is to ensure that the fjson_tokener_parse and fjson_tokener_parse_ex functions behave predictably under various conditions.
Test Execution Flow
The following diagram illustrates the flow from a test invocation to the final verification of results.
The test suite utilizes a common function run_output_test within test-defs.sh to run binaries and compare their stdout against expected output files.
Sources: tests/test-defs.sh:56-120, tests/test_parse.c:20-30
Parsing Modes and Specific Tests
Strict vs. Liberal Parsing
libfastjson supports a strict mode via the FJSON_TOKENER_STRICT flag. Compliance tests verify that in strict mode, certain non-standard behaviors (like comments or specific case variations) are rejected.
| Feature | Strict Mode (FJSON_TOKENER_STRICT) | Liberal Mode (Default) |
|---|---|---|
| Comments | Rejected | Supported (/* */ or //) |
| Boolean Case | Only true/false allowed | True/False might be accepted |
| Trailing Chars | Error after JSON object end | Stops at end of object |
| Leading Zeros | Numbers starting with 0 (except 0 itself) rejected | Accepted |
Sources: tests/test_charcase.c:29-45, json_tokener.c:300-350, tests/test_parse.c:160-200
Integer Parsing and Overflow
Tests in test_parse_int64.c specifically target the robustness of fjson_parse_int64. This is critical because sscanf implementations vary across platforms, and libfastjson includes a fallback mechanism for broken sscanf environments.
The library tests values ranging from INT32_MAX up to and exceeding INT64_MAX to ensure proper saturation or error handling.
Sources: json_util.c:158-245, tests/test_parse_int64.c:25-80
Incremental Parsing Tests
Incremental parsing is tested by feeding the tokenizer fragments of a JSON string. The state machine must maintain the parsing state (e.g., fjson_tokener_state_eatws, fjson_tokener_state_object_field) between calls.
struct incremental_step {
const char *string_to_parse;
int length;
int char_offset;
enum fjson_tokener_error expected_error;
int reset_tokener;
};
These steps simulate network buffers where a single JSON message arrives in multiple chunks.
Sources: tests/test_parse.c:116-155, json_tokener.c:240-280
Error Handling and Reporting
Compliance tests verify that the fjson_tokener_error enum returns the correct diagnostic for specific malformed inputs.
| Error Enum | Triggering Condition |
|---|---|
fjson_tokener_error_parse_unexpected | Unexpected character or trailing data in strict mode |
fjson_tokener_error_parse_boolean | Invalid characters in a boolean token |
fjson_tokener_error_parse_number | Malformed numeric structure (e.g., 12.3.4) |
fjson_tokener_error_parse_object_key_name | Missing quotes on object keys |
Sources: json_tokener.c:56-72, tests/test_parse.c:100-114
Summary
The Parsing Compliance Tests ensure that the library's tokenizer remains performant yet reliable. By validating integer boundaries, incremental state transitions, and strict mode adherence, the test suite guarantees that libfastjson handles complex real-world data without regressions.
Sources: CONTRIBUTING.md:1-10, json_tokener.c:10-40