perf: decode and encode PNGs with @napi-rs/image

Merged
KD
16d ago
+1017-206
  • e2e, @e2e-dev/mobile, and @e2e-dev/decision read and write PNGs with @napi-rs/image instead of fast-png and pngjs (packages/e2e/src/internal/image.ts, packages/mobile/src/png.ts, packages/decision/src/png.ts). pixelmatch stays: the addon has no diff API.
  • Chromium's captures are 8-bit RGB. The addon widens them to RGBA itself (opacity(1) keeps every sample), which was most of the decode once the codec was native. Other color types go through a short layout step in JS.
  • Decoded pixels match the old libraries on the 76 PNGs the repo tracks and on a synthetic matrix of color types, depths, interlacing, palettes, and transparent colors. Two behaviors change on purpose and are in the changeset: gray PNGs with 1, 2, or 4 bits now decode, and a 16-bit gray or RGB PNG with a transparent color compares at 16 bits.
  • e2e loads the addon when the first PNG is read or written (createRequire, as oxc() does), so --version, --help, and a run that takes no screenshot never need the binding. A missing binding fails the call with the new IMAGE_CODEC_UNAVAILABLE (docs/reference/errors.mdx). The STARTUP table in package-imports.test.ts pins the lazy load.
  • @e2e-dev/mobile and @e2e-dev/decision import the addon with the package. They are opt-in, and a lazy load there would turn a missing binding into a secure screenshot silently withheld (redactSecure) or a zoom silently skipped (decode in overlay.ts). Stopping at import is the louder failure.

Speed

Main (0c1b3572) against this branch: both dist builds in one process, A/B then B/A, 11 rounds per cell, medians over three real screenshots per profile. Each case first checks that both sides produce the same pixels.

web 1280×720web 2560×1440 (DPR 2)phone 1170×2532 (DPR 3)device 804×1786
decode30.8 → 2.06 ms117 → 9.98 ms97.5 → 7.78 ms20.4 → 2.64 ms
encode32.9 → 5.32 ms134 → 16.4 ms106 → 19.9 ms50.8 → 6.66 ms
model-bound downscale, downscalePixels(768)50.4 → 9.87 ms147 → 28.0 ms120 → 23.4 ms38.8 → 11.2 ms
toHaveScreenshot capture, whole screen31.1 → 2.51 ms136 → 29.4 ms
mobile maskPng, 2 rects197 → 28.5 ms90.5 → 8.84 ms
decision zoomAround(400, 2x)39.3 → 5.36 ms39.4 → 5.38 ms39.1 → 7.68 ms37.6 → 5.55 ms

What is left at 2560×1440 is the JS box resample, unchanged here: 14.6 of the 28 ms.

pnpm bench:ab --suite testbed -- tests/visual.e2e.ts, 30 pairs. Every test ran the same steps with the same outcome, and every counter matched. The harness notes that the suite's tests differ between the builds because screenshot.e2e.ts was ported; visual.e2e.ts itself is unchanged.

mainthis PRchange (95% CI)
step expect.toHaveScreenshot454 ms340 ms-24.6% (-27.8% to -23.2%)
suite, sum of attempts2.24 s2.09 s-6.8% (-7.5% to -5.9%)
wall, process3.32 s3.14 s-5.1% (-5.6% to -4.5%)
e2e --version, 30 interleaved pairs194.7 ms188.7 ms-3.4%

Decisions for the reviewer

  • Platforms. The addon publishes 12 bindings and oxc, already in e2e, publishes 19. Linux on ppc64, s390x, and riscv64 (gnu and musl), armv7 musl, android arm32, and OpenHarmony have none. There toHaveScreenshot and any agent screenshot the runner has to resize (a side over 768 px) fail with IMAGE_CODEC_UNAVAILABLE, and mobile and decision fail at import. The changeset is a patch; whether this is a **Breaking:** minor is your call.
  • Encoded size. FilterType.Adaptive makes a Chromium capture about 15% larger than fast-png wrote it (55 to 63 KB at 1280×720) and a device screenshot about 20% smaller. "The smaller of no filter and adaptive" would never be larger than fast-png on a Chromium capture, costs about twice the encode time (5.3 to 9.7 ms at 1280×720), and touches the same three files. Not done here.
  • Shared layout code. The step from the addon's pixel layouts to 8-bit RGBA (LAYOUTS, narrow, spread) exists in e2e, mobile, and decision. mobile and decision reach e2e only through its public entries, so sharing it means a new export on e2e or e2e/engine, or a new package. readPngSize is copied in web and mobile for the same reason.
  • Install size. The binary is 15.5 MB on macOS arm64 and 21.1 MB on Linux x64 (8.3 MB download), against 1.25 MB for fast-png, fflate, and iobuffer. pngjs stays through pixelmatch.
  • Memory. The addon frees what a Transformer holds in a finalizer that Node runs a loop turn after V8 collects the owner, and it does not tell V8 about that memory. A loop that calls these functions back to back without yielding keeps about 70 MB per 2560×1440 capture: 120 calls peak at 8.4 GB where main stayed at 253 MB. Real runs yield between calls: worker peaks are 240 MB for visual.e2e.ts (main 315 MB) and 490 MB for screenshot.e2e.ts (main 427 MB).

Verified

Ran it locally: yes

  • The built CLI on the testbed, tests/visual.e2e.ts against the stored screenshots fast-png wrote: 4 passed, and no tracked file changed.
  • The failure path, with the platform binding left out (@napi-rs/image-darwin-arm64 unresolvable, which is what --omit=optional does), on a testbed config without decisionFilter:
    • tests/todos.e2e.ts passes: 7 passed, exit 0.
    • tests/visual.e2e.ts fails every test with exit 3:
 × |web| tests/visual.e2e.ts > screenshots > compares a block with a changing part masked
IMAGE_CODEC_UNAVAILABLE: e2e reads and writes screenshots with @napi-rs/image, and its native binding for darwin-arm64 (@napi-rs/image-darwin-arm64*) is not installed. npm skips it with --omit=optional, or when package-lock.json was written on another platform (npm/cli#4828): install again without --omit=optional, or delete package-lock.json and node_modules and run npm install. The package has no binding for Linux on ppc64, s390x, or riscv64
  • The testbed's own config imports @e2e-dev/decision, so there the same missing binding stops at config load, exit 2: failed to load config .../e2e.config.ts: Cannot find native binding. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). ...
  • The mobile benchmark I could not run here (no device). CI ran it on this head: the iOS simulator passed 55 deterministic tests and 21 agentic ones, the Android emulator 57 and 20, with one retry on Android's Gestures, which flaked the same way on #1576.
  • Not run: the agentic suites live against a model (CI replays their recordings), Windows, and Linux outside CI.

🤖 Generated with Claude Code