perf(tokenizer): skip runs of text in one loop when decoding entities
With the default decodeEntities: true, stateText handles every text character through one iteration of the parse loop and its state switch. With decodeEntities: false it already skips ahead with fastForwardTo. This change makes stateText scan to the next < or & (or the end of the chunk) in a local loop, so long runs of text cost one call instead of one per character. Callbacks, startIndex/endIndex and the DOM are unchanged; no callback fires during the scan, so pausing and resetting from callbacks behave as before.
The gain shows on text-heavy input such as books, articles and feeds. Markup-heavy pages spend most of their characters in tags and attribute values, so they stay flat.
| Input | Before | After | Speedup |
|---|---|---|---|
Gutenberg book, 850 KB, parseDocument | 7.17 ms | 4.66 ms | 1.54x |
same, Parser with no-op callbacks | 5.50 ms | 3.00 ms | 1.84x |
Atom feed, 100 KB, xmlMode, parseDocument | 0.85 ms | 0.69 ms | 1.23x |
NYT RSS, 57 KB, xmlMode, parseDocument | 0.55 ms | 0.48 ms | 1.15x |
| Wikipedia, GitHub, HN pages | ~1.0x | ||
| 200 B snippet | 3.7 us | 3.8 us | ~1.0x |
Thread CPU time, median of 11 alternating rounds with both builds loaded in one process (Node 24, Ryzen 5 5600X). Text with one character between tags and a& repeated stay within noise.
I checked old against new on random documents across HTML, xmlMode and decodeEntities: false, written whole, in small chunks and one character at a time, with pauses from callbacks: callbacks, indices and DOM were identical. I added a test for text and entities split across chunks.
Summary by CodeRabbit
- Bug Fixes
- Improved parsing of text and entities across consecutive input chunks, including entities split at the end of a chunk.
- Tests
- Added coverage for emitted events and offsets when chunks include text, entities, and tags.