fix(tokenizer): keep the trailing `?` of XML processing instructions

Closed
MF
7d ago
+74-21

Problem

Since #2382, <?xml version="1.0"?> loses the ? of its ?> closer:

import { parseDocument } from "htmlparser2";
import render from "dom-serializer";

const doc = parseDocument('<?xml version="1.0"?><a/>', { xmlMode: true });
console.log(doc.children[0].data); // ?xml version="1.0"
render(doc, { xmlMode: true });    // <?xml version="1.0"><a/>

dom-serializer writes a processing instruction as <${data}>, so the second
? disappears and the round trip is no longer well-formed XML. htmlparser2 10.x
kept it.

Cause

stateInProcessingInstruction reads the ? of the closer into
this.sequenceIndex, then ends the instruction with
onprocessinginstruction(sectionStart, this.index - 1). Parser#getSlice uses
an exclusive end index, so index - 1 stops at the ? instead of including it.
The HTML-mode branch passes this.index (the >), which includes it.

Fix

Pass this.index in the XML branch too, restoring the pre-11 data. The
processing-instruction callbacks now also report the full instruction: the next
node starts one character later, and the event's end offset points at the >.

getInstructionName is unaffected — it cuts at the first whitespace — so this
matches what 10.x reported for both data and name.

Tests

  • Parser.spec.ts: the trailing ? survives a ?> split across two chunks.
  • index.spec.ts: parseDocument keeps it in data, which is what
    dom-serializer needs for the round trip.
  • Updated the three affected event snapshots (Atom/RDF/RSS feeds, tokenizer
    ?> termination test); all changes are the added ? and the shifted offsets.

npm test (217 tests, eslint, tsc --noEmit, biome) passes.

Closes #2562.

Summary by CodeRabbit

  • Bug Fixes
    • XML processing instructions now preserve the closing question mark in their data, including when input arrives in separate chunks. The question mark is excluded from the instruction name, while the data retains the original text and punctuation. This keeps the parsed instruction’s name and data correctly separated in XML mode and ensures consistent results for both complete and chunked input.