Parser Events: Comments and Processing Instructions

Relevant source files

The following files were used as context for generating this wiki page:

• src/Parser.ts
• src/Tokenizer.ts
• src/Parser.events.spec.ts
• src/Tokenizer.spec.ts
• README.md

Introduction

The htmlparser2 library is a high-performance, streaming HTML and XML parser that utilizes a callback-based architecture to process document nodes. A significant portion of this architecture is dedicated to handling non-element markup, specifically Comments, CDATA sections, and Processing Instructions (PIs). These events allow developers to capture metadata, scripts, or structural information that is not part of the standard DOM tree text or element nodes.

The parser distinguishes between these events based on the xmlMode configuration and the specific character sequences encountered during tokenization. While HTML mode often treats certain instructions as "bogus comments" for compatibility with browser behavior, xmlMode enables strict handling for XML-specific declarations and processing instructions.

Sources: README.md:1-10, src/Parser.ts:315-325

Architectural Overview

The handling of comments and instructions involves a two-layer system: the Tokenizer identifies character sequences and maintains state, while the Parser consumes these tokens to emit higher-level events to the user-provided Handler.

Event Flow

The following diagram illustrates how raw character data is transformed into specific handler events for comments and instructions.

The Tokenizer transitions through states like State.BeforeDeclaration and State.InCommentLike to determine the exact nature of the markup before notifying the Parser.
Sources: src/Tokenizer.ts:60-95, src/Parser.ts:578-620

Comment Handling

Comments are primarily defined as content between <!-- and -->. However, htmlparser2 handles several edge cases including abruptly closed comments and "bogus" comments.

Parser Callbacks

The parser provides two distinct callbacks for comments:

• oncomment(data): Emits the string content found within the comment tags.
• oncommentend(): Signals that the comment block has fully closed.

HTML Bogus Comments

In HTML mode (when xmlMode is false), markup starting with <? or <! that does not match a standard comment or DOCTYPE is treated as a "bogus comment". This matches browser behavior where such tags are consumed until the next > character.

Feature	HTML Mode Behavior	XML Mode Behavior
<!-- content -->	Emits oncomment	Emits oncomment
<?target data?>	Treated as Bogus Comment	Emitted as Processing Instruction
<!abruptly closed	Emitted as Bogus Comment	Treated as Text/Error
--!>	Recognized as comment end	Only --> recognized

Sources: src/Parser.ts:599-606, src/Tokenizer.ts:503-518, src/Tokenizer.spec.ts:167-170

Processing Instructions and Declarations

Processing Instructions (PIs) are used in XML to provide instructions to the application. In htmlparser2, these are handled via the onprocessinginstruction event.

Implementation Logic

When the tokenizer encounters <?, it enters the InProcessingInstruction state. In XML mode, it scans for the closing ?> sequence. The Parser then extracts the "name" (the first word after the prefix) and the "data" (the full content).

DOCTYPE and Declarations

Declarations (starting with <!) are routed differently based on the mode. In HTML mode, <!DOCTYPE is specifically recognized, while other <! sequences fall back to bogus comments. In XML mode, ondeclaration is used for generic XML declarations.

Sources: src/Parser.ts:559-597, src/Tokenizer.ts:654-666

CDATA Sections

CDATA sections (<![CDATA[ ... ]]>) are used to escape blocks of text containing characters that would otherwise be considered markup.

Configuration Impact

The handling of CDATA is heavily dependent on two options: xmlMode and recognizeCDATA.

1. XML Mode / recognizeCDATA enabled: The parser emits oncdatastart(), followed by ontext(), and finally oncdataend().
2. Foreign Context (SVG/MathML): CDATA is treated as raw text content.
3. HTML Mode: If not explicitly enabled, CDATA sections are treated as comments, emitting oncomment("[CDATA[...]]").

Summary of Events

Event Name	Parameter	File Reference
oncomment	data: string	src/Parser.ts:187
oncommentend	none	src/Parser.ts:190
oncdatastart	none	src/Parser.ts:188
oncdataend	none	src/Parser.ts:189
onprocessinginstruction	name: string, data: string	src/Parser.ts:191

Sources: src/Parser.ts:187-191

Conclusion

The handling of comments and processing instructions in htmlparser2 provides a robust mechanism for parsing both standard HTML and complex XML documents. By delegating low-level state management to the Tokenizer and high-level event emission to the Parser, the library ensures that metadata and instructions are captured accurately while maintaining the performance required for large-scale streaming applications.