Feed Parsing: RSS
Relevant source files
The following files were used as context for generating this wiki page:
Feed parsing in htmlparser2 provides a specialized interface for processing syndication formats including RSS, Atom, and RDF. While the core of the library is a generic HTML/XML parser, it includes high-level utilities to transform XML feed structures into normalized JavaScript objects. This functionality is primarily exposed through the parseFeed method, which abstracts the underlying DOM construction and feed extraction logic.
Sources: src/index.ts:79-88, README.md:213-221
Architecture and Data Flow
The feed parsing process operates as a pipeline starting from a raw string and ending with a normalized Feed object. The parseFeed function internally utilizes parseDocument with xmlMode enabled by default to ensure correct handling of XML-specific syntax like self-closing tags and CDATA sections.
The diagram shows the sequential transformation from a raw XML string to a structured Feed object.
Sources: src/index.ts:84-88, src/Parser.ts:251-264
Key Components
| Component | Description |
|---|---|
parseFeed | The primary entry point for feed parsing, accepting a string and optional Options. |
xmlMode | A critical parser option (defaulted to true for feeds) that ensures strict XML parsing. |
getFeed | A utility from domutils that extracts feed metadata and items from a DOM structure. |
Feed Type | The resulting data structure containing type, title, link, description, updated, author, and items. |
Sources: src/index.ts:79-88, README.md:219-224
Data Structures and Formats
htmlparser2 supports multiple syndication formats by recognizing their unique XML structures during the DOM traversal phase.
Supported Feed Formats
The library includes test fixtures and logic for three primary formats:
- RSS 2.0: Defined by tags like
<rss>,<channel>, and<item>. - Atom: Defined by tags like
<feed>,<entry>, and namespaces likehttp://www.w3.org/2005/Atom. - RDF/RSS 1.0: Defined by
<rdf:RDF>and<item>elements.
Sources: src/FeedHandler.spec.ts:8-33, src/fixtures/Documents/RSS_Example.xml:3-5, src/fixtures/Documents/Atom_Example.xml:3-4, src/fixtures/Documents/RDF_Example.xml:2-3
The Normalized Feed Object
When a feed is successfully parsed, it returns a normalized object. This prevents developers from needing to handle the structural differences between RSS and Atom manually.
The class diagram represents the logical mapping of feed data after normalization.
Sources: README.md:219-221, src/fixtures/Documents/RSS_Example.xml:7-40
Implementation Details
The parseFeed function is a convenience wrapper. It enforces xmlMode: true because feeds rely on XML syntax that differs from standard HTML (e.g., case sensitivity and specific entity handling).
// src/index.ts:84-88
export function parseFeed(
feed: string,
options: Options = parseFeedDefaultOptions,
): Feed | null {
return getFeed(parseDocument(feed, options).children);
}
Sources: src/index.ts:84-88
Parser Configuration for Feeds
When parsing feeds, the Parser class adjusts its behavior based on ParserOptions. The most significant settings for RSS parsing are:
- lowerCaseTags: Typically
falseinxmlModeto preserve XML tag casing. - recognizeCDATA: Enabled in
xmlModeto allow content like<description><![CDATA[...]]></description>. - recognizeSelfClosing: Enabled in
xmlModeto handle tags like<link href="..." />.
Sources: src/Parser.ts:251-264, README.md:144-156
Conclusion
Feed parsing in htmlparser2 leverages the high-performance core parser to provide a simplified API for syndication content. By defaulting to xmlMode and integrating with domutils.getFeed, it offers a reliable way to ingest RSS, Atom, and RDF data into standardized JavaScript structures.
Sources: src/index.ts:82-88, README.md:213-221