High-Level Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• src/index.ts
• readme.md
• src/decode.ts
• src/encode.ts
• src/escape.ts
• scripts/trie/README.md
• scripts/write-decode-map.ts

The entities library is a high-performance JavaScript utility designed for encoding and decoding XML and HTML entities. Its primary purpose is to transform characters into their corresponding entity representations (e.g., < to <) and vice versa, supporting various levels of strictness and character sets. The architecture is optimized for speed and memory efficiency, utilizing pre-compiled radix trees (tries) and specialized scan paths to process strings.

The project is structured around three main pillars: Decoding, Encoding, and Escaping. It provides both high-level entry points for general use and low-level, tree-shakable functions for performance-critical applications. The library is "side-effect free," facilitating optimal bundling in modern web environments.

Sources: readme.md:5-15, src/index.ts:1-10, package.json:28-40

Core Components and Modules

The architecture is divided into specialized modules that handle specific logic paths. This separation allows for tree-shaking and minimizes the bundle size for users who only require a subset of functionality.

1. API Entry Point (src/index.ts)

The main entry point provides unified encode and decode functions that act as dispatchers to specialized implementation functions based on provided options.

Enum / Interface	Description
EntityLevel	Defines whether to support XML or HTML (superset) entities.
EncodingMode	Determines strategy: UTF8, ASCII, Extensive, Attribute, or Text.
DecodingMode	Configures behavior: Legacy (no semicolon), Strict, or Attribute.

Sources: src/index.ts:9-55

2. Decoding Engine (src/decode.ts)

The decoding logic supports both synchronous string processing and a streaming interface through the EntityDecoder class. It uses a Uint16Array trie for lookups.

Data Structures: Radix Trees (Tries)

To achieve high performance and low memory consumption (~250Kb vs ~8.5Mb for objects), entities maps its entity tries into a single Uint16Array.

Node Layout in Uint16Array

Each node in the trie is represented by a 16-bit header and optional data words.

• Bits 15..14: Value length (0-3).
• Bit 13: Dual-use flag (Semicolon required OR Compact run flag).
• Bits 12..7: Branch length or span.
• Bits 6..0: Jump table offset or first character.

Compact Runs

When a path has a linear chain of single-child nodes, they are collapsed into a "compact run" to reduce pointer chasing. The first character is in the header, and subsequent characters are packed two per uint16 word.

Sources: scripts/trie/README.md:40-90, src/decode.ts:316-350

Encoding and Escaping Logic

The encoding system uses a dual-path approach to maximize throughput: an Inline Scan for short gaps and a Regex Jump for long spans of clean text.

Encoding Architecture

1. Fast Path (ASCII): A flat lookup table asciiEntities (128 entries) allows O(1) lookups for ASCII characters.
2. Slow Path (Non-ASCII/Trie): Consults the htmlTrie for multi-character entities or astral code points.
3. Scan Optimization: The encodeHTMLTrieRe function uses an INLINE_SCAN_WINDOW (16 characters) to check for encodable characters before falling back to a global regex.

Escaping Strategy

Escaping functions are specialized versions of encoding that target specific subsets of characters defined by the HTML spec for text and attributes.

Function	Targeted Characters	Source
escapeUTF8	" & ' < >	src/escape.ts
escapeAttribute	" & \u00A0	src/escape.ts
escapeText	& < > \u00A0	src/escape.ts

Sources: src/escape.ts:160-195

Build and Pre-processing Pipeline

The library relies on pre-generated data files to avoid runtime overhead.

• scripts/write-decode-map.ts: Converts JSON maps into the packed Uint16Array binary trie used for decoding.
• scripts/write-encode-map.ts: Serializes the encoding trie into a compact string format that is parsed once at startup in src/encode.ts.
• Dictionary Encoding: The trie generator uses a dictionary-based delta-encoding scheme to reduce the size of the generated data files.

Sources: scripts/write-decode-map.ts:30-60, scripts/write-encode-map.ts:50-80, package.json:48-52

Summary

The high-level architecture of entities centers on extremely efficient data structures (binary tries) and a hybrid scanning mechanism. By offloading complex trie construction to a build-time script and using typed arrays for traversal, it provides a performant, tree-shakable solution for entity processing in JavaScript environments.

Sources: readme.md:17-25, src/decode.ts:1-20