Parsing Lifecycle
Relevant source files
The following files were used as context for generating this wiki page:
The parsing lifecycle in htmlparser2 represents the end-to-end process of transforming a stream of raw HTML or XML data into structured events. This process is handled by two primary components: the Tokenizer, which breaks the input into low-level lexical units, and the Parser, which applies higher-level document logic, handles tag stacks, and manages namespace contexts like SVG and MathML.
The lifecycle supports both incremental streaming via .write() and full document parsing via .parseComplete(). It is designed to be "forgiving," meaning it can handle malformed HTML by implying missing tags or auto-closing elements based on defined relationship rules.
Sources: src/Parser.ts:241-267, README.md:16-20
Initialization and Configuration
The lifecycle begins with the instantiation of the Parser class. During construction, the parser initializes its internal state, including the tag stack and foreign context trackers. It also instantiates the Tokenizer, which acts as the data producer for the parser's logic.
The diagram shows the sequence of internal steps taken when a new Parser is created.
Sources: src/Parser.ts:272-291, src/Tokenizer.ts:167-175
Parser Options
The behavior of the lifecycle is heavily influenced by the ParserOptions provided during initialization:
| Option | Type | Default | Description |
|---|---|---|---|
xmlMode | boolean | false | Enables XML-specific behaviors like self-closing tags and CDATA. |
lowerCaseTags | boolean | !xmlMode | Automatically lowercases all tag names. |
decodeEntities | boolean | true | Decodes HTML entities within text and attributes. |
recognizeCDATA | boolean | xmlMode | Recognizes <![CDATA[ sections as text. |
recognizeSelfClosing | boolean | xmlMode | Triggers close events for tags ending in />. |
Sources: src/Parser.ts:200-239, README.md:112-120
Data Ingestion and Tokenization
Data enters the lifecycle through the .write(chunk) method. Chunks are buffered and passed to the Tokenizer. The Tokenizer maintains a state machine to identify characters belonging to text, tags, attributes, comments, or declarations.
The sequence diagram illustrates how data flows from the application through the Parser to the Tokenizer, and back up as events.
Sources: src/Parser.ts:586-607, src/Tokenizer.ts:211-236, src/Tokenizer.ts:717-733
Buffer Management
The Parser manages multiple string buffers to allow for efficient slicing of data across chunk boundaries. The getSlice method handles retrieving content even if a token spans across two different write calls.
Sources: src/Parser.ts:553-575, src/Parser.spec.ts:128-142
Tree Construction and Implied Logic
Unlike a simple tokenizer, the Parser maintains a stack of open elements to enforce document structure. In HTML mode, it uses an openImpliesClose map to determine if a new opening tag should automatically close currently open tags (e.g., a <li> closing a previous <li>).
Implied Closures
The parser handles various "forgiving" behaviors:
- Auto-close: Certain tags like
<p>,<li>, and<td>close their predecessors if they belong to specific sets. - Void Elements: Tags like
<img>or<br>are emitted as both open and closed immediately without being pushed to the stack. - Foreign Context: The parser tracks if it is inside
<svg>or<math>to adjust tag name casing and handle CDATA correctly.
Sources: src/Parser.ts:10-91, src/Parser.ts:392-414, src/Parser.spec.ts:168-212
Namespace Adjustments
When in an SVG context, the parser automatically adjusts tag names to their correct camelCase versions.
This flowchart describes the logic used in readTagName to handle casing and aliasing.
Sources: src/Parser.ts:335-373, src/Parser.ts:133-176
Termination and Cleanup
The lifecycle ends when .end() is called. The Tokenizer processes any remaining trailing data (such as unclosed comments or text) and signals the Parser. The Parser then pops all remaining elements from its stack, emitting onclosetag events for each, ensuring the consumer receives a balanced set of tags even for truncated input.
Trailing Data Handling
The Tokenizer includes specific logic for "Comment-like" data and markup declarations that haven't been formally closed at the end of the file.
- Unclosed CDATA in HTML mode is treated as a bogus comment.
- Unclosed tags in the buffer are ignored if they don't reach a valid boundary.
Sources: src/Parser.ts:540-551, src/Tokenizer.ts:796-848, src/Parser.spec.ts:16-30
Lifecycle Events Summary
The following table summarizes the key events emitted during the parsing lifecycle:
| Event | Trigger |
|---|---|
onopentagname | As soon as the tag name is identified. |
onattribute | When an attribute name and value pair is fully parsed. |
onopentag | Once the opening tag and all its attributes are finished. |
ontext | When text content is encountered (may fire multiple times). |
onclosetag | When a closing tag is found or implied by the parser. |
onend | When the input stream is fully processed and the stack is cleared. |
Sources: src/Parser.ts:241-267, README.md:70-85, src/Parser.events.spec.ts:39-45
The parsing lifecycle ensures that regardless of how data is delivered (single string or multiple chunks), the resulting event stream remains consistent and structurally valid according to the configured mode.
Sources: src/index.ts:34-45, src/Parser.events.spec.ts:16-35