fix(ios): make back the verified left-edge swipe
Summary
On a header-less screen with its own Back button (React Native native-stack, headerShown: false), iOS back refused: it guessed at buttons and found none. Now iOS back (with or without --in-app or --system) is the left-edge back swipe, verified by a change of the app's screen. An agent presses a visible control by its ref itself.
How the swipe is sent
- Simulator: one edge-masked HID touch, the
right-edgepreset; parity is pinned incontracts/fixtures/back-swipe-geometry.json. An XCTest drag starting 20 pt in popped 2 in 6; the edge touch popped 6 in 6. - Physical device: a hold-free synthesized fling, 200 ms, from 2 pt inside the leading edge to 0.9 of the width, safe on a tree XCTest cannot serialize. There is no XCTest drag any more. XCTest's press-and-drag holds at the edge for at least 0.1 s and then moves at 500 px/s, and it missed 2 in 10 on an iPhone 17 Pro. The fling popped 10 in 10 there (evidence).
- Preparation: both swipes are resolved before the sample taken just before the swipe, and every tree and window read happens then, so the gap holds only the contact.
How the change is verified
- Comparison: each sample is a 48×96 grayscale grid. Its top 7 rows are always left out: the portrait status bar and Dynamic Island, whether shown or hidden. The status bar is never read. A change means more than 1% of the remaining cells moved by more than 24 of 256 levels.
- Settle: time-based, at most 8 samples 0.25 s apart. The screen must be still for a contact allowance (0.2 s on a simulator, 0.4 s on a device) plus 0.25 s plus the slowest capture seen while settling, so slow captures settle longer.
- Credit: a change counts only from the just-before sample, and only when the gap from it to the after sample did not outrun the still span. All three times are taken when a capture completed. Otherwise the result is
back_swipe_unverified. - Log: every exit logs
AGENT_DEVICE_RUNNER_BACK_SWIPE_VERIFICATIONwithoutcome,code,changed,stableMs,justBeforeToAfterMs,gestureMs,maxCaptureMs,settleSamplesandsettled.
Reasons and removals
- Reasons:
back_swipe_no_change(sent, screen unchanged),back_swipe_unverified(sent, unknown outcome),back_swipe_not_dispatched(dispatched: no). - visionOS: refused by the daemon at the back capability fact with
UNSUPPORTED_OPERATION. The runner's visionOS arm answers only a direct request, withBACK_SWIPE_NOT_DISPATCHED. - Deleted: the button lookup, the blind tap,
backSystem(recorded in RUNNER_PROTOCOL.md), and the device XCTest drag. - Moved: the long-press, sequence and planned-gesture dispatch disclosure moved to #4474. This PR adds no envelope field.
- Rebased onto #4474 as one commit.
PreparedTouch(notSent:)is an exhaustive switch; #4474's.notDispatchedmaps to the same pre-touch refusal as.unsupported(BACK_SWIPE_NOT_DISPATCHEDfor back), so nothing traps at run time.
Validation
Physical-iPhone verification was skipped by maintainer decision. This PR merges on simulator evidence.
Simulator validation at 1a1d0ea, rebased onto #4474, on an iPhone 17e simulator with iOS 27:
- Runner XCTests: NavigationTests, SynthesizedInteractionTests and ScrollDragExecutionTests ran 74 tests, with 0 failures and 1 environment skip. They cover:
- swipe, no change, still-changing and slow-toggle screens;
- hidden status bar, lit panel, rejected-query route;
- slow host → performed, and 0.4 s captures with a 0.2 s contact → performed;
- a 1.5 s contact →
BACK_SWIPE_UNVERIFIED; - the device fling on readable and unserializable trees (no window query in the gap, no XCTest press);
- #4474's synthesized-interaction tests.
- Builds: the runner builds for iOS simulator, iOS device, macOS, tvOS and visionOS.
- Replays:
01-settings.adand02-deep-navigation.adpass (back1.65 s each), including under host CPU load at an earlier head.01-physical-lifecycle.adpasses when run on the simulator. - CI: the iOS 26.2 simulator smoke replays.
Earlier hardware data: iPhone 17 Pro, iOS 27.0.1, before the fling became the product path (evidence). The synthesized 200 ms fling popped 10 of 10. Verification outcomes were not measured on hardware at the final head.
Known risk: on heavy-tree screens, a physical-iPhone back may report BACK_SWIPE_UNVERIFIED even though the screen popped. That happens when the gap from the just-before sample to the after sample exceeds the device still span; one device capture took about 0.4 s. The runner log line AGENT_DEVICE_RUNNER_BACK_SWIPE_VERIFICATION records stableMs, justBeforeToAfterMs, gestureMs and maxCaptureMs, so the device still span can be sized from real numbers later, without weakening the gap check.
| Scenario (simulator, CLI = MCP) | base 799e9c0 | head |
|---|---|---|
| Header screen | → Home | → Home |
| Header-less Product | refused | → Products |
| Root | refused after blind tap | back_swipe_no_change |
| back ∥ back / back ∥ press | refused | → Home / → Products |
Risk: an app with a left-edge drawer may open the drawer instead.
🤖 Generated with Claude Code