Generated Code Assets

Relevant source files

The following files were used as context for generating this wiki page:

• src/generated/decode-data-html.ts
• src/generated/decode-data-xml.ts
• src/generated/encode-html.ts
• scripts/write-decode-map.ts
• scripts/write-encode-map.ts
• src/decode.ts
• src/encode.ts
• scripts/trie/README.md

Generated Code Assets in the entities library consist of highly optimized, pre-compiled data structures—specifically tries—used for the rapid encoding and decoding of HTML and XML entities. These assets are programmatically generated from source JSON maps to minimize runtime memory consumption and maximize traversal speed. By transforming entity mappings into compact binary formats, the library achieves significant performance gains over standard object-based lookups.

The generation process involves specialized scripts that convert human-readable entity definitions into Uint16Array structures or compressed strings. These assets are stored in the src/generated/ directory and are consumed by the core decode and encode modules. This architecture allows the library to support thousands of HTML5 entities while maintaining a small bundle size and high execution efficiency.

Binary Trie Architecture for Decoding

The decoding system utilizes a radix tree (trie) mapped to a Uint16Array. This structure avoids the overhead of JavaScript object allocations for each node, which previously resulted in ~8.5MB of memory usage; the current binary implementation reduces this to approximately 250KB.

Node Layout and Bit Manipulation

Each node in the binary trie starts with a 16-bit header word that encodes metadata about the node's value and its branching logic.

Bits	Description
15..14	Value Length Field: 0 (No value), 1 (Inlined), 2 (One code unit next), 3 (Two code units next).
13	Dual-use Flag: Semicolon-required flag (if value > 0) or Compact Run flag (if value == 0).
12..7	Branch Length / Span: Number of branches or length of a compact run.
6..0	Jump Table Offset / First Char: First character of a run or offset for jump tables.

Sources: scripts/trie/README.md:37-65, src/decode.ts:400-420

Build Process for Decode Tries

The scripts/write-decode-map.ts utility processes XML and HTML JSON maps to produce the binary trees. It uses a custom encoding scheme to further compress the trie data into a string format that is decoded at runtime using decodeTrieDict.

The generator uses a dictionary-based delta encoding combined with Run-Length Encoding (RLE) for high-frequency values, resulting in data that is ~24% smaller than raw base64.
Sources: scripts/write-decode-map.ts:31-60, scripts/write-decode-map.ts:208-220

Compact Encode Trie

The encoding asset, found in src/generated/encode-html.ts, uses a string-based serialized trie format. Unlike the decoding trie which is optimized for character-by-character consumption, the encoding trie is designed for mapping Unicode code points back to entity names.

Serialization Format

The serialized string follows a specific pattern to minimize size: [gap]name;[{children}].

• Gap: A base-10 integer representing the difference between the current code point and the previous one.
• Name: The entity name (e.g., amp).
• Children: Nested blocks enclosed in {} for multi-code-point entities.

Sources: src/encode.ts:48-62, scripts/write-encode-map.ts:63-80

Multi-Code-Point Support

The encoder supports complex entities (e.g., < + U+20D2 → <⃒). During the generation phase, astral characters (code points > U+FFFF) are flattened into the trie or handled as surrogate pair sequences to avoid large gaps in the data structure.

Key Components and Files

File Path	Role	Key Export / Logic
src/generated/decode-data-html.ts	HTML Decode Asset	htmlDecodeTree (Uint16Array)
src/generated/decode-data-xml.ts	XML Decode Asset	xmlDecodeTree (Uint16Array)
src/generated/encode-html.ts	HTML Encode Asset	Default export string (Serialized Trie)
scripts/write-decode-map.ts	Decode Generator	encodeTrieData, convertMapToBinaryTrie
scripts/write-encode-map.ts	Encode Generator	getTrie, serializeTrie

Sources: src/generated/decode-data-html.ts:5, src/generated/encode-html.ts:3, scripts/write-decode-map.ts:223-228, scripts/write-encode-map.ts:13-25

Performance Optimizations in Generated Assets

The generated assets are designed to work with specific runtime "fast paths":

1. Bitsets: HTML_BITSET and XML_BITSET_VALUE allow for O(1) checks to determine if a character needs encoding before attempting trie traversal.
2. Jump Tables: For dense ranges of characters, the trie header points to a jump table where the next node index is retrieved by simple subtraction and array access (nodeIndex + char - jumpOffset).
3. Compact Runs: Linear paths in the trie (sequences of characters with no branches) are collapsed into a single header and a packed byte string, reducing the number of iterations required to match long entity names like &CounterClockwiseContourIntegral;.

Sources: src/encode.ts:145-155, src/decode.ts:535-550, scripts/trie/README.md:95-105

Generated code assets represent the core efficiency of the entities library. By moving the complexity of trie construction and optimization to a build-time script, the library provides a high-performance solution for entity processing that is both memory-efficient and fast, as evidenced by its use of Uint16Array and bit-packed headers.