Parsing Engines

Relevant source files

The following files were used as context for generating this wiki page:

• src/parse.spec.ts
• package.json
• Readme.md
• website/src/content/blog/2024-08-07-version-1.md
• src/cheerio.spec.ts
• THREAT_MODEL.md
• src/static.spec.ts

Cheerio serves as a fast and flexible wrapper around underlying parsing engines to transform HTML and XML strings into a traversable DOM tree. By default, it utilizes parse5 for robust, spec-compliant HTML parsing and offers htmlparser2 as a high-performance, forgiving alternative, particularly suited for XML and non-standard HTML.

The parsing architecture is designed to be environment-agnostic, supporting both server-side Node.js and browser environments. It handles various input formats, including strings, buffers, and streams, while ensuring that the resulting DOM structures adhere to a consistent model compatible with domhandler.

Sources: Readme.md:46-51, package.json:109-112, THREAT_MODEL.md:9-11

Parser Architecture

Cheerio manages two primary internal parsing paths. The choice between these engines is typically determined by user options, such as the xml flag or the internal _useHtmlParser2 property.

Supported Engines

Engine	Description	Use Case
parse5	Standard-compliant HTML5 parser.	Default HTML parsing, strict spec adherence.
htmlparser2	Fast, forgiving parser with XML support.	XML parsing, performance-critical tasks, malformed HTML.

Sources: Readme.md:46-51, src/parse.spec.ts:7-11, package.json:109-112

Logic Flow for Parsing Input

When a user calls cheerio.load() or internal parsing functions, the system evaluates the provided options to delegate to the appropriate adapter.

The diagram above illustrates the decision logic for selecting a parsing engine based on configuration options.
Sources: src/parse.spec.ts:7-11, src/cheerio.spec.ts:251-255, website/src/content/blog/2024-08-07-version-1.md:83-88

Core Parsing Features

HTML5 Compliance with parse5

parse5 provides a high degree of fidelity to the HTML5 specification. It handles implicit element creation (such as adding <tbody> to tables or <head> to documents) and resolves misnested tags according to standard adoption agency rules.

• Implicit Tags: Automatically adds <html>, <head>, and <body> tags to documents if they are missing.
• Table Correction: Implicitly creates <tr> wrappers around <td> tags and <tbody> within <table> elements.
• Source Location: Can include source code location information (offsets) if the sourceCodeLocationInfo option is enabled.

Sources: src/parse.spec.ts:279-284, src/parse.spec.ts:286-302, src/cheerio.spec.ts:446-460

XML and Forgiving Parsing with htmlparser2

htmlparser2 is used when the xml option is set to true. Unlike parse5, it does not perform spec-compliant tree correction (like adding <tbody>), which is often required for RSS feeds or generic XML documents.

• Case Sensitivity: Preserves camelCase tag and attribute names when XML mode is active.
• Self-Closing Tags: Correctly handles self-closing syntax (e.g., <media:thumbnail />).
• Performance: Generally prioritized for speed and lower memory overhead during large-scale scraping.

Sources: src/cheerio.spec.ts:227-234, src/tests/xml.spec.ts:20-25, Readme.md:46-51

Parsing Configuration Options

Users can fine-tune the behavior of the parsing engines through the CheerioOptions object passed to load().

Option	Type	Default	Description
xml	boolean | object	false	Enables XML mode. If an object, specifies htmlparser2 options.
scriptingEnabled	boolean	true	If false, content inside <noscript> tags is parsed as tags rather than text.
sourceCodeLocationInfo	boolean	false	Enables tracking of line/column offsets for DOM nodes (parse5 only).
baseURI	string	undefined	The base URL used to resolve relative links in the prop('href') API.

Sources: src/cheerio.spec.ts:427-444, src/cheerio.spec.ts:446-460, src/api/attributes.spec.ts:275-285, website/src/content/blog/2024-08-07-version-1.md:83-88

Example: Custom Parser Options

// Using htmlparser2 for XML with specific options
const $ = cheerio.load('<XML>content</XML>', {
  xml: {
    withStartIndices: true,
    decodeEntities: false
  }
});

// Enabling location info in parse5
const $html = cheerio.load('<p>Hello</p>', {
  sourceCodeLocationInfo: true
});

Sources: website/src/content/blog/2024-08-07-version-1.md:83-88, src/parse.spec.ts:360-367

Data Flow and DOM Model

Regardless of the engine used, Cheerio converts input into a consistent DOM model defined by domhandler. This model includes standard properties like tagName, childNodes, parentNode, nextSibling, and previousSibling.

The sequence diagram demonstrates the transformation from a raw string to a Cheerio instance through the parser and domhandler layers.
Sources: src/parse.spec.ts:7-11, Readme.md:129-138, package.json:109-112

Security and Parsing

Parsing is a critical boundary in the Cheerio threat model. The engines are responsible for preventing "XSS enablement" where the serializer might produce dangerous output from malicious input. However, Cheerio does not execute scripts found during parsing; it treats them strictly as data within the DOM tree.

• Resource Consumption: While legitimate large inputs consume memory, the parsers aim to avoid asymmetric resource consumption (e.g., ReDoS or exponential entity expansion).
• Trust Boundary: Input markup is considered untrusted, and the parsers are expected to handle malformed content without crashing or modifying the Object.prototype.

Sources: THREAT_MODEL.md:14-16, THREAT_MODEL.md:27-38

Summary

Parsing engines in Cheerio provide the foundation for all subsequent manipulation and selection. By abstraction through a common interface, Cheerio allows developers to choose between the spec-compliant behavior of parse5 for standard web pages and the flexibility and speed of htmlparser2 for XML and data extraction tasks. All parsed output is unified under the domhandler schema, enabling a consistent jQuery-like developer experience.