Testing Strategy and Fixtures
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The testing strategy for htmlparser2 is designed to ensure high performance and technical accuracy across a variety of parsing modes, including HTML, XML, and specialized feed formats (RSS, Atom, RDF). The suite employs a combination of unit tests, event-driven integration tests, and snapshot testing to validate the correctness of the Parser and Tokenizer components. By utilizing a robust set of document fixtures, the project verifies its ability to handle complex real-world scenarios, such as fragmented streaming data and malformed HTML.
The testing architecture is built upon the vitest framework and leverages specialized helpers to collect and compare parsing events. This ensures that every stage of the data flow—from raw buffer input to the emission of DOM events—is rigorously monitored for regressions.
Sources: package.json:58, src/Parser.events.spec.ts:1-20, README.md:148-160
Core Testing Components
Event-Driven Validation
The primary testing mechanism involves writing data to the parser and capturing the resulting events via an event collector. Tests verify that the parser emits the correct sequence of events, such as onopentagname, onattribute, and ontext, and that these results are consistent whether data is provided as a single blob or in multiple chunks.
The diagram above shows the dual-path verification where input is processed both incrementally and all at once to ensure consistency.
Sources: src/Parser.events.spec.ts:12-35, src/Parser.spec.ts:7-40
Stream and Buffer Testing
Tests for WebWritableStream and WritableStream focus on the parser's ability to handle streaming data, particularly fragmented Unicode characters. For instance, multi-byte characters like the Euro symbol (€) are split across separate write() calls to ensure the underlying decoder correctly reassembles them before passing them to the handler.
| Feature Tested | Source File | Logic Description |
|---|---|---|
| Fragmented Unicode | WebWritableStream.spec.ts | Splits UTF-8 bytes (e.g., 0xe2 0x82 and 0xac) across writes. |
| ReadableStream Pipe | WebWritableStream.spec.ts | Uses readable.pipeTo(stream) to simulate modern web streams. |
| Node.js Streams | WritableStream.spec.ts | Uses fs.createReadStream().pipe() for legacy stream validation. |
Sources: src/WebWritableStream.spec.ts:8-20, src/WritableStream.spec.ts:10-21
Fixtures and Document Snapshots
Document Fixtures
The project maintains a directory of XML and HTML fixtures used to validate complex document structures. These files represent standard use cases and edge cases for the parser.
| Fixture File | Format | Purpose |
|---|---|---|
Basic.html | HTML | Standard document structure with title and body. |
Attributes.html | HTML | Tests quoted, unquoted, and boolean attributes. |
Svg.html | HTML/SVG | Validates foreign context and tag case adjustments. |
RSS_Example.xml | XML | Validates RSS feed parsing and item extraction. |
Atom_Example.xml | XML | Validates Atom feed metadata and content parsing. |
Sources: src/WebWritableStream.spec.ts:54-60, src/FeedHandler.spec.ts:7-35
Snapshot Testing
htmlparser2 makes extensive use of snapshots to ensure that the output of the Tokenizer and FeedHandler does not change unexpectedly. This is particularly critical for the Tokenizer, where the exact sequence of low-level callbacks (e.g., onattribname, onattribdata) is verified against known-good states.
Sources: src/Tokenizer.spec.ts:38-70, src/FeedHandler.spec.ts:11-35
Specialized Test Scenarios
State Machine and Tokenizer Logic
The Tokenizer.spec.ts file isolates the tokenizer's internal state machine. It uses a Proxy based callback collector to log every interaction, ensuring that states like InSpecialTag (for <script> or <style>) or CDATASequence are entered and exited correctly.
This sequence illustrates how the tokenizer handles special tags that treat content as raw text rather than nested HTML.
Sources: src/Tokenizer.ts:153-176, src/Tokenizer.spec.ts:7-30
HTML Specification Compliance
Tests in Parser.spec.ts ensure the parser follows specific HTML behaviors, such as implicit tag closing for table elements (<tbody> closing <thead>, etc.) and the "open implies close" logic for <p> and <li> tags.
Key compliance tests include:
- Implied Closures: Ensuring
<td>is closed when<th>opens. - Position Tracking: Verifying
startIndexandendIndexare accurate even when tags are spread across chunks. - Error Handling: Validating that calling
.write()or.end()after the parser has finished throws appropriate errors.
Sources: src/Parser.spec.ts:98-175, src/Parser.events.spec.ts:246-260
Conclusion
The testing strategy for htmlparser2 balances unit-level precision with broad integration coverage. By simulating diverse input methods—streaming, chunked, and full-document—and validating against a rich set of fixtures and snapshots, the project ensures that its high-performance parsing remains reliable across all supported web and XML formats.