Home

Relevant source files

The following files were used as context for generating this wiki page:

• readme.md
• package.json
• src/index.ts
• src/decode.ts
• src/encode.ts
• src/escape.ts
• scripts/trie/README.md

The entities library is a high-performance JavaScript solution for encoding and decoding XML and HTML entities. It is designed to be fast, configurable, and tree-shakeable, supporting various standards including XML, HTML4, and HTML5 character references. The project is widely utilized by major industry tools such as htmlparser2, the AWS SDK, and commonmark.
Sources: readme.md:1-25, package.json:2-15

The core architecture relies on optimized radix trees (tries) mapped to typed arrays (Uint16Array) to minimize memory consumption while maintaining high traversal speeds. It provides distinct modules for encoding, decoding, and escaping, allowing developers to tailor the output based on target environments like UTF-8 or ASCII-only systems.
Sources: scripts/trie/README.md:1-25, src/index.ts:1-93

Core API and Configuration

The library exposes a high-level API through the main entry point, providing unified encode and decode functions, as well as specialized functions for specific levels (XML vs HTML) and modes.

Entity Levels and Modes

The library categorizes operations into two primary levels:

• XML: Supports only the standard five XML entities (&, <, >, ", ') and numeric references.
• HTML: A superset of XML supporting all named character references defined in the HTML spec.
  Sources: src/index.ts:7-15

Option	Type	Description
EntityLevel.XML	Enum	Support only XML entities.
EntityLevel.HTML	Enum	Support HTML entities (superset of XML).
DecodingMode.Legacy	Enum	Supports entities not terminated with a semicolon.
DecodingMode.Strict	Enum	Requires all entities to be terminated with a semicolon.
DecodingMode.Attribute	Enum	Applies HTML spec rules for entities within attributes.
Sources: src/index.ts:7-15, src/decode.ts:60-72

Main Logic Flow

The following diagram illustrates the high-level logic used when calling the general encode or decode functions:

Citations: src/index.ts:80-115

Decoding Architecture

Decoding is performed using an EntityDecoder class that supports both full-string decoding and partial/streaming decoding. The system uses a binary trie to traverse entity names efficiently.
Sources: src/decode.ts:85-115

Trie Data Structure

The trie is stored in a Uint16Array. Each node starts with a 16-bit header containing:

• Value Length (Bits 15-14): Encodes if a value is present (0 to 3 units).
• Flag 13: Dual-use bit for semicolon requirements or compact run indicators.
• Branch/Span (Bits 12-7): Information about child nodes.
• Jump/First Char (Bits 6-0): Offset for jump tables or first character of a run.
  Sources: scripts/trie/README.md:43-65

Decoding State Machine

The EntityDecoder operates as a state machine to handle numeric and named entities:

Encoding and Escaping

Encoding focuses on converting characters into their corresponding named or numeric entities. The library provides optimized "scan" paths that combine inline bitset checks with regex jumps to skip clean spans of text.
Sources: src/encode.ts:245-280

Encoding Modes

• UTF-8: Only escapes characters strictly required for XML validity (e.g., < to <).
• ASCII: Escapes all non-ASCII characters and characters not valid in the chosen level.
• Extensive: Encodes all characters that have an equivalent named entity.
  Sources: src/index.ts:20-50

Performance Optimization (Inline Scan)

The encoder uses an INLINE_SCAN_WINDOW (16 characters) to check for escapable characters using a bitset before falling back to a stateful global regex (HTML_ENCODE_RE or xmlEncodeRegex). This approach minimizes the overhead of regex execution for dense markup.
Sources: src/encode.ts:238-255, src/escape.ts:40-50

Security and Threat Model

The library includes a dedicated security policy focusing on inputs as untrusted strings.

• Denial of Service (DoS): The architecture avoids quadratic behavior through linear trie traversal and optimized scanning.
• XSS Prevention: entities ensures that characters are correctly escaped to prevent context-breaking in HTML/XML, though it is not a full HTML sanitizer.
• Spec Compliance: The decoder follows WHATWG HTML named character reference specifications.
  Sources: SECURITY.md:15-40

Summary

The entities library provides a robust, low-level primitive for character reference manipulation. By utilizing binary tries for decoding and bitset-accelerated scanning for encoding, it achieves top-tier performance while maintaining compliance with HTML5 and XML standards. Its design allows for significant bundle size savings through tree-shaking by providing direct function exports for specific use cases.
Sources: readme.md:45-65, src/index.ts:120-145