System Architecture Overview

Relevant source files

The following files were used as context for generating this wiki page:

• package.json
• README.md
• src/Tokenizer.ts
• src/Parser.ts
• src/WritableStream.ts
• src/index.spec.ts
• src/Parser.spec.ts

The htmlparser2 library is a high-performance, forgiving HTML and XML parser for Node.js and the web. Its architecture is built around a streaming, event-driven model that minimizes memory allocations by processing data in chunks rather than loading entire documents into memory. The system is primarily composed of two layered components: a low-level Tokenizer that identifies lexical units (tokens) and a high-level Parser that manages the state machine for tag relationships, namespaces, and document structure.

The architecture supports multiple modes of operation, including standard HTML5 parsing and a strict xmlMode. It integrates with the broader ecosystem via domhandler for document object model construction and provides specialized interfaces like WritableStream for integration with Node.js streams.

Sources: package.json:5-13, README.md:9-15, src/Parser.ts:251-260

Core Components

The system architecture is divided into distinct layers that handle byte-level streaming, lexical analysis, and semantic parsing.

Component Relationship

The following diagram illustrates the relationship between the streaming interfaces, the Parser, and the Tokenizer.

The Parser acts as the central orchestrator, initializing the Tokenizer and receiving low-level events which it then translates into higher-level semantic events for the registered Handler.
Sources: src/Parser.ts:314-332, src/Tokenizer.ts:182-194, src/WritableStream.ts:22-38

Key Classes and Interfaces

Component	Responsibility	Source
Tokenizer	Lexical analysis; converts raw strings into raw tokens (tag names, attribute data).	src/Tokenizer.ts:161
Parser	State management; handles tag nesting, implicit closing, and foreign contexts (SVG/MathML).	src/Parser.ts:251
Handler	Interface for receiving high-level events (onopentag, ontext, onclosetag).	src/Parser.ts:203
WritableStream	Node.js Writable stream wrapper for the Parser.	src/WritableStream.ts:19
EntityDecoder	Decodes HTML/XML entities (e.g., &) during tokenization.	src/Tokenizer.ts:190-194

Data Flow and Tokenization

Data flows into the system through the write method. The Parser buffers incoming chunks and passes them to the Tokenizer. The Tokenizer maintains a state machine (e.g., State.Text, State.InTagName, State.InAttributeName) to identify boundaries within the text.

Tokenization State Machine

The Tokenizer uses a character-by-character scan to transition between internal states.

The Tokenizer uses fastForwardTo for performance optimization, skipping characters until a specific delimiter (like < or >) is found.
Sources: src/Tokenizer.ts:43-85, src/Tokenizer.ts:456-465, src/Tokenizer.ts:850-960

Semantic Parsing Logic

The Parser implements the higher-level logic required to handle the complexities of HTML, such as "open implies close" rules and namespace switching.

Tag Management

The Parser maintains a stack of open tags. When a new tag is encountered that implicitly closes an existing one (e.g., a <li> inside an unclosed <li>), the Parser pops the stack and emits onclosetag events before starting the new tag.

Foreign Contexts

The parser detects svg and math tags to switch into a "foreign context." In this context, certain HTML-specific rules (like self-closing tag restrictions) are relaxed, and tag name casing is adjusted (e.g., viewbox to viewBox).
Sources: src/Parser.ts:114-160, src/Parser.ts:376-382

Configuration Options

The behavior of the architecture is highly configurable through the ParserOptions interface.

Option	Effect	Default
xmlMode	Disables HTML-specific logic; enables strict XML parsing.	false
decodeEntities	Determines if entities like © are decoded.	true
lowerCaseTags	Forces tag names to lowercase.	!xmlMode
recognizeSelfClosing	Allows <tag /> to close itself in HTML mode.	xmlMode

Sources: src/Parser.ts:166-201, README.md:120-130

Stream Integration

The architecture provides a WritableStream that inherits from Node's Writable. It uses a StringDecoder to handle multi-byte characters that may be split across chunk boundaries.

// src/WritableStream.ts:22-38
export class WritableStream extends Writable {
    private readonly _parser: Parser;
    private readonly _decoder = new StringDecoder();

    constructor(cbs: Partial<Handler>, options?: ParserOptions) {
        super({ decodeStrings: false });
        this._parser = new Parser(cbs, options);
    }

    override _write(chunk: string | Buffer, encoding: string, callback: () => void): void {
        this._parser.write(
            isBuffer(chunk, encoding) ? this._decoder.write(chunk) : chunk
        );
        callback();
    }
}

Sources: src/WritableStream.ts:22-38, src/WebWritableStream.spec.ts:11-23

Summary

The htmlparser2 architecture is a tiered system designed for speed and flexibility. By separating lexical tokenization from semantic tree logic, it allows for efficient streaming processing of both malformed HTML and strict XML. The system relies on a complex state machine in the Tokenizer to handle raw data, while the Parser ensures that higher-level semantic requirements and namespace adjustments are met.