fix(tokenizer): keep the trailing `?` of XML processing instructions

RY
9d ago
+59-21

Fixes #2562.

Problem

In xmlMode, stateInProcessingInstruction reports the end of a processing instruction as this.index - 1 when it reaches the > of ?>. Parser#onprocessinginstruction treats endIndex as inclusive (getSlice(start, endIndex) for the data, startIndex = endIndex + 1 for the next node), so:

  • the data loses the trailing ? (?xml version="1.0" instead of ?xml version="1.0"?),
  • the instruction's endIndex points at the ? instead of the >,
  • the next node's startIndex points at the >.

dom-serializer writes a processing instruction as <${data}>, so a parse → serialize round trip turns <?xml …?> into <?xml …>.

The - 1 came in with #2382 (v11.0.0); v10.1.0 passed this.index here.

Fix

Pass this.index again, the same convention ondeclaration uses. This restores the data and offsets from before v11.

The issue left open whether htmlparser2 or dom-serializer should change. This PR fixes it on the htmlparser2 side, so dom-serializer should not also start appending ? (that would give ??). It also means the endIndex that the Tokenizer passes to its onprocessinginstruction callback for <?…?> in xmlMode goes up by one, back to the v10 value; code that adapted to the v11/v12 offsets would see the difference.

Tests

  • A new regression test in src/Parser.spec.ts checks the data and the start/end indices of the instruction and of the text node after it. It fails with the old line and passes with the fix.
  • Three snapshots had recorded the shortened data and offsets (Tokenizer, WritableStream, WebWritableStream); they are updated.
  • Full suite: 215 passed (214 before, plus the new test); npm run lint passes. Run on Node 24, Linux.
  • The issue's round trip (parseDocument in xmlMode, then render from dom-serializer 3.1.1) returns the input unchanged with the fix, and drops the ? without it.

Summary by CodeRabbit

  • Bug Fixes
    • Corrected position reporting for XML processing instructions, including the boundary before following text.
    • Processing-instruction names no longer include the closing ?, while callback data retains it.
  • Tests
    • Added regression coverage for processing-instruction names, callback data, and position reporting.