fix(mobile): fail an action agent-device refuses for an unknown screen origin as an action failure
GR
4h ago+44+44+01234567890123456789-11-11-01234567890123456789
- agent-device 0.21.37 (callstack/agent-device#4439) refuses an iOS ref action with
COMMAND_FAILED,details.reason: window-origin-unknown,dispatched: nowhen its capture can't place the window on screen. It happens on a full-screen iPhone app right after a large web view loads. While that screen stays up the runner reads the tree through its private AX fallback, which never gets the viewport back (measured over 150 s). - The engine reported the refusal as
ENGINE_FAILURE, an infrastructure error (exit 3), although nothing was sent and the device is fine. It now maps toNOT_ACTIONABLE(ACTION_FAILED, exit 1) instaleOr, next to the covered-target refusals from #1686. A refusal withdispatched: unknownstaysENGINE_FAILURE. - agent-device's hint names its CLI (
Take a fresh snapshot … or use screenshot …). The message carries the test's move instead:Nothing was sent. For a tap, tap the point instead with screen.tapAt().A point tap isn't aimed from the capture, so agent-device takes it; other actions have no point form. - No retry: re-observing returns the same capture state for as long as the screen stays heavy, so a retry only moves the failure to the deadline.
- The refused reasons and their hints are one map,
NOT_ACTIONABLE_REASONS.undefinedkeeps agent-device's hint, and a string replaces its CLI advice. - No docs change: callstack/agent-device#4573 (merged) makes agent-device accept a full-screen root, so the web view case goes away at the next agent-device bump, and the error hint names the move.
Issue
Fixes #1821
Verified
Ran it locally: yes
- Repro: gronxb/e2e-repro, a UIKit app with a
WKWebViewthat loads 3,000 links. The test reloads it and taps a native button, three rounds. iPhone 17 Pro Max simulator, iOS 26.5, agent-device 0.21.37. e2e runon main's published packages and on this branch's packede2e,@e2e-dev/core,@e2e-dev/mobile:
main (@e2e-dev/mobile@0.12.7) | this branch |
|---|---|
✗ locator.tap getByTestId("reload-web") — ENGINE_FAILURE, exit 3 | ✗ locator.tap getByTestId("reload-web") — ACTION_FAILED, exit 1 |
hint: Take a fresh snapshot after the accessibility channel recovers, or use screenshot to choose screen coordinates. | hint: Nothing was sent. For a tap, tap the point instead with screen.tapAt(). |
- The documented move works: the same three rounds with
screen.tapAt({ x: 220, y: 98 })pass on 0.21.37. errors.test.ts: the new case fails on main (ENGINE_FAILURE) and passes here.