Decoding API

The Decoding API is a high-performance system within the entities library designed to convert HTML and XML entities back into their corresponding Unicode characters. It supports both synchronous full-string decoding and stateful streaming decoding. The API utilizes a highly optimized binary trie structure for named entity lookups and follows WHATWG HTML specifications for handling numeric and legacy character references.

The API is categorized into two main implementation layers: high-level convenience functions for standard document processing and a low-level EntityDecoder class for advanced streaming scenarios where entities might be split across multiple data chunks. Sources: src/index.ts, src/decode.ts, readme.md

Core Architecture and Data Structures

Binary Trie Structure

For named entity resolution, the system uses a Uint16Array as a binary trie. This trie is generated from JSON maps during the build process to minimize runtime memory overhead and maximize lookup speed. The trie supports features such as "compact runs" to handle long, non-branching entity names efficiently. Sources: src/decode.ts:405-410, scripts/write-decode-map.ts

Entity Decoding Modes

The API behavior varies based on the DecodingMode, which determines how strictly the parser requires semicolon termination and how it handles legacy entities.

Sources: src/decode.ts:69-80, src/index.ts:47-60

Decoding Flow Logic

The decoding process starts when an ampersand (&) is encountered. The system then branches based on whether the following characters indicate a numeric reference or a named entity.

Main API Components

High-Level Functions

The library provides several functions for common decoding tasks.

• decode(input, options): The primary entry point that switches between HTML and XML levels. Sources: src/index.ts:67-79
• decodeHTML(input, mode): Decodes HTML entities using the full HTML trie. Sources: src/decode.ts:805-814
• decodeXML(input): Uses a hand-coded fast path for the 5 standard XML entities (&, <, >, ", ') and numeric entities. Sources: src/decode.ts:835-917

Streaming Decoder: EntityDecoder

The EntityDecoder class allows for processing fragments of data. It maintains internal state across multiple calls to the write() method.

Error Handling

The API can optionally produce errors for spec-non-compliant entities through the EntityErrorProducer interface. It tracks:

• Missing semicolons after character references.
• Absence of digits in numeric references.
• Validation of numeric values (e.g., replacement of invalid codepoints via replaceCodePoint).
  Sources: src/decode.ts:85-91, src/decode.spec.ts:370-435

Technical Implementation Details

Numeric Entity Resolution

Numeric entities are parsed into integers and then processed by replaceCodePoint. Values exceeding U+10FFFF are mapped to the replacement character U+FFFD.
Sources: src/decode.spec.ts:160-170, src/decode.ts:316-320

Trie Navigation

The determineBranch function is the core logic for traversing the binary trie. It supports three lookup types:

1. Single Branch: Direct character match.
2. Jump Table: O(1) lookup for a range of consecutive character codes.
3. Packed Dictionary: Binary search on sorted keys for sparse branches.
   Sources: src/decode.ts:511-550

Attribute Mode Logic

In DecodingMode.Attribute, the decoder implements specific termination logic. If a named entity is not terminated by a semicolon, it is rejected if the next character is an equals sign (=), an alphanumeric character, or a digit, to prevent accidental decoding of URL parameters like &not=1. Sources: src/decode.ts:55-60, src/decode.spec.ts:187-205

Summary

The Decoding API provides a robust and spec-compliant way to handle character references in HTML and XML. By utilizing a binary trie for named entities and a stateful machine for streaming, it achieves high performance while maintaining flexibility for various parsing contexts, including legacy HTML and strict XML environments. Sources: readme.md, src/decode.ts