test(testbed): Playwright's page tests in the testbed suite

Merged
OK
13d ago
+8744-72

What and why

  • Playwright's own page tests (microsoft/playwright tests/page + tests/library/download.spec.ts at v1.64.1, our playwright-core pin), rewritten against the e2e public API, as ordinary testbed tests in apps/testbed/tests/. 32 files, ~500 tests: click, fill, check, select, keyboard, locators, actionability and scroll into view, frames, dialogs, route/fulfill/continue/fallback, waitForURL/Response, network idle, screenshots + masks, downloads, file chooser.
  • Each describe is tagged playwright (--tag playwright runs only them). Each file header names its source and lists the tests left out, with the reason.
  • Known differences are fail: 'reason' (#1539), so a closed gap fails the run until its mark goes: 16 tests. 6 fail with an error fail does not cover (an engine failure, or a refused argument), so they are skip: 'reason'.
  • The playground serves Playwright's assets and its test server's dynamic routes for any path it has no page for (apps/testbed/app/playwright/, with LICENSE + NOTICE; assets unchanged, ignored by lint and fallow). One app, one config, chromium only.
  • CI: on a pull request the Testbed step leaves the playwright tests out of the model-filtered run and a second step runs all of them (23s locally); a push to main runs everything once.
  • 6 testbed tests the ported ones cover are removed (custom dialog handler, route stub/abort, raw keyboard, scrollIntoView, waitForURL).
  • One engine fix the tests found: a fixture call that times out names what it waited for (operation timed out in browser.waitForResponse "foo.css"), and url-pattern steps quote string patterns.

Marked fail (open gaps unless noted):

  • elements inside a visibility: hidden iframe read as visible (3 tests)
  • uncheck refuses role=radio; inputValue on a non-input reads "" instead of throwing
  • route.fulfill body string-only; header overrides can't delete a header
  • ? in route globs matches one character (Playwright: literal)
  • a page deleting Map/Array breaks evaluate (main world)
  • two screenshot steps at once interfere
  • by design per the migrate guide: bare-string selectOption matches label only, textContent() is rendered text, isChecked() on a non-checkable reads false

Marked skip: multiple set-cookie in a fulfill (2), screenshot while navigating, screenshot with a stalled subframe, setInputFiles([]), timeout: 0.

CodeQL on the routes (apps/testbed/app/playwright/routes.mjs, test-only): the app listens on 127.0.0.1 only; delays and sizes are clamped, redirects stay on the app's two origins, credentialed CORS goes only to those origins, the multipart parse is linear.

Issue

None. #1503 stacks on this.

Verified

Ran it locally: yes

apps/testbed, chromium (macOS), on main 732ea8af9
e2e run --tag playwright            503 passed | 6 skipped      23s
pnpm test --exclude-tag playwright   76 passed | 1 skipped
pnpm test (all)                     579 passed | 7 skipped      29s
e2e + web unit, lint, fallow, typecheck, build clean

Checklist

  • Changeset added for any change to a published package, or the change is testbed, docs, or CI only.
  • Breaking change.
  • Docs updated in the same PR (error text only; no page describes it).
  • Contract change.
  • Engine contract change.