Web WritableStream

Relevant source files

The following files were used as context for generating this wiki page:

• src/WebWritableStream.ts
• src/WebWritableStream.spec.ts
• src/Parser.ts
• src/Tokenizer.ts
• src/WritableStream.ts
• package.json

WebWritableStream is a specialized implementation within the htmlparser2 library that makes the core Parser interface available as a standard Web Streams API WritableStream. It allows developers to process HTML/XML data streams in environments that support the Web Streams API, such as modern browsers, Deno, and Node.js (version 18.0 and higher).

This module is particularly useful for piping response bodies from fetch() calls or other ReadableStream sources directly into the parser without needing to manually handle chunk concatenation or character decoding for fragmented Unicode characters.

Sources: src/WebWritableStream.ts:3-17, src/WebWritableStream.spec.ts:60-77, package.json:80-82

Architecture and Data Flow

The WebWritableStream class extends the native WritableStream and encapsulates an instance of the Parser. It acts as a bridge between the Web Streams API and the library's internal Tokenizer and Parser logic.

Data Processing Pipeline

When data is written to the stream, it follows this path:

1. Input Reception: The stream accepts chunks of type string or Uint8Array.
2. Decoding: If the chunk is a Uint8Array, it is processed by a TextDecoder using the { stream: true } option to ensure that fragmented multi-byte characters (like Unicode emojis or symbols) are correctly reassembled across chunk boundaries.
3. Parsing: The resulting string is passed to the internal Parser instance's write method.
4. Tokenization: The Parser utilizes a Tokenizer to break the string into HTML/XML tokens.
5. Callbacks: The Parser triggers specific handler events (e.g., onopentag, ontext) based on the discovered tokens.

The following diagram illustrates the data flow from a ReadableStream through the WebWritableStream:

Decoding Strategy

The stream uses TextDecoder to handle binary data. During the close operation, the stream ensures that any remaining bytes stored in the decoder's internal buffer are flushed and passed to the parser before signaling completion.

Sources: src/WebWritableStream.ts:35-42

API and Configuration

Constructor

The WebWritableStream constructor requires an event handler and accepts optional parser configurations.

Parameter	Type	Description
cbs	Partial<Handler>	An object containing callback functions for parser events (e.g., onopentag, ontext, onclosetag).
options	ParserOptions	Configuration for the parser, such as xmlMode, lowerCaseTags, and decodeEntities.

Sources: src/WebWritableStream.ts:22-25, src/Parser.ts:182-220

Parser Options Summary

The underlying Parser behavior can be customized through the options object.

Option	Default	Description
xmlMode	false	Enables XML processing, affecting entity decoding and self-closing tags.
decodeEntities	true	Decodes HTML entities (e.g., & to &).
lowerCaseTags	!xmlMode	Automatically converts tag names to lowercase.
recognizeSelfClosing	xmlMode	Recognizes tags ending in /> as self-closing.

Sources: src/Parser.ts:182-220, README.md:121-131

Implementation Details

Class Structure

The implementation leverages the super call to the native WritableStream to define the underlying sink's write and close methods.

export class WebWritableStream extends WritableStream<string | Uint8Array> {
    constructor(cbs: Partial<Handler>, options?: ParserOptions) {
        const parser = new Parser(cbs, options);
        const decoder = new TextDecoder();
        const streamOption: TextDecodeOptions = { stream: true };

        super({
            write(chunk) {
                parser.write(
                    typeof chunk === "string"
                        ? chunk
                        : decoder.decode(chunk, streamOption),
                );
            },
            close() {
                const remaining = decoder.decode();
                if (remaining) parser.write(remaining);
                parser.end();
            },
        });
    }
}

Sources: src/WebWritableStream.ts:21-44

Comparison with Node.js WritableStream

While WebWritableStream is designed for the Web Streams API, htmlparser2 also provides a WritableStream class (in src/WritableStream.ts) designed for Node.js node:stream compatibility. The Node.js version uses StringDecoder instead of TextDecoder and extends Node's Writable class.

Sources: src/WritableStream.ts:18-22, package.json:71-74

Error Handling and State Management

The WebWritableStream manages the lifecycle of the parser based on the stream state:

• Writing after End: If parser.write() is called after parser.end() (via the stream closing), the parser triggers an onerror callback with the message .write() after done!.
• Aborting: Tests indicate that if a stream is aborted via writer.abort(), the parser stops processing, and pending events like ontext or onend are not called.
• Unicode Fragments: The stream is validated to handle fragmented UTF-8 sequences, such as the Euro symbol (€), which is encoded as 0xe2 0x82 0xac. The TextDecoder maintains state between write calls to correctly emit the full character.

Sources: src/Parser.ts:634-637, src/WebWritableStream.spec.ts:9-20, src/WebWritableStream.spec.ts:46-56

Usage Example

Piping a fetch request body into the parser:

import { WebWritableStream } from "htmlparser2/WebWritableStream";

const stream = new WebWritableStream({
    onopentag(name, attribs) {
        console.log("Opened:", name);
    },
    ontext(data) {
        console.log("Text:", data);
    }
});

const response = await fetch("https://example.com");
if (response.body) {
    await response.body.pipeTo(stream);
}

Sources: src/WebWritableStream.ts:8-17, src/WebWritableStream.spec.ts:60-77

Conclusion

The WebWritableStream acts as a modern, standards-compliant wrapper for htmlparser2. By bridging the library's high-performance parsing logic with the Web Streams API, it facilitates efficient, non-blocking processing of HTML and XML data in browser-based applications and modern JavaScript runtimes.