PR #3727 - callstack/agent-device

Summary

find <locator> focus and the focus leg of find <locator> type aimed a point themselves, so they never passed the pipeline door where the #2589 keyboard guard runs: a match behind the visible keyboard reported Focused (x, y) while the key ate the touch. Closes #2622.

Landed as the issue's option 2: the guard plus the vocabulary it needs (InteractionAction, interactionVerb) move to @agent-device/selectors, which both zones already import, so R2 is satisfied by placement, not exemption. The seam calls that one guard against the tree find matched, at the center it dispatches, ahead of expireRefFrame — a refusal leaves the frame pinnable.

find label Form focus   # -> Ref @e2 is behind the visible keyboard and cannot be focused safely

25 files: the move, the seam fix, an ADR 0011 path row (find-focus-dispatch, 14 cells classified) with daemon-harness scenarios, one shared #2589 fixture in @agent-device/test-kit. The row declares four gaps that are this seam's pre-existing behavior, newly visible because the row exists: offscreen (findAct says ignore), parentOwnedTouchPoint (taps the rect center), nonHittable (promotion runs, annotation never does), targetStability (no #3471 gate). All four cite #3734, which owns closing them; outcomeObservation stays on #3335.

Stale in #2622: paths are now packages/daemon/..., and option 2's cost was overstated — the pipeline-level half of the moved test still drives the guard through createInteractionDevice, so the split was a rename plus a trimmed unit suite.

Live evidence

Devices are mine, not shared: iOS simulator 2622-keyboard-guard (iPhone 17 Pro, iOS 27.0, created for this run) and AVD kb2622_api35 (emulator-5580, started for this run). Both deleted afterwards. Agent Device Tester via Metro, Form tab, Gboard on Android. Transcripts at head a0135d0d6, the gate-tested SHA.

iOS — refusal on both legs, positive control above the band

Keyboard raised by focusing the email field, seeded with pre@live.test. Settings is the tab-bar button at y 795, centered inside the band.

$ agent-device snapshot -i --json | jq '.data'
keys: 34   topmost key y: 597   viewport: 402x874
covered app node: e25 button Settings {x: 286.8, y: 795, w: 90.2, h: 54}

$ agent-device get text @e12
pre@live.test

$ agent-device find label Settings focus --json
{
 "success": false,
 "code": "COMMAND_FAILED",
 "message": "Ref @e25 is behind the visible keyboard and cannot be focused safely",
 "reason": "tap_keyboard_occludes_target",
 "ref": "@e25",
 "rect": {"x": 286.8, "y": 795, "width": 90.2, "height": 54},
 "keyboardFrame": {"x": 0, "y": 590, "width": 402, "height": 284},
 "dispatched": "no"
}

$ agent-device find label Settings type x --json
{
 "success": false,
 "code": "COMMAND_FAILED",
 "message": "Ref @e25 is behind the visible keyboard and cannot be focused safely",
 "reason": "tap_keyboard_occludes_target",
 "ref": "@e25",
 "keyboardFrame": {"x": 0, "y": 590, "width": 402, "height": 284},
 "dispatched": "no"
}

$ agent-device get text @e12                                  # unchanged, no keystroke landed
pre@live.test

$ agent-device snapshot -i | grep -c '\[key\]'                # keyboard still up after both refusals
40

$ agent-device find label "Contact details" focus --json      # above the band
{ "success": true, "data": { "x": 201, "y": 247, "message": "Focused (201, 247)" } }

On this producer data.keyboard is null, so the refusal above came from the guard's tree-derived rule reading the key nodes in find's own capture. That is the direct answer to the fail-open question: with no measured band, find's tree still carries the keys and the guard refuses.

Android — both legs refuse on the measured-band producer

Landscape, Gboard. This is the producer the review asked about — the helper's measured IME window, not the tree rule:

$ agent-device snapshot --json
windows: 3   keyNodes: 0
keyboard fact carried into the capture: {"kind":"visible","frame":{"x":0,"y":74,"width":2264,"height":1006}}
in-band app node: e165 group "Settings" {x: 1863, y: 806, w: 442, h: 211}  bundle com.callstack.agentdevicelab

$ agent-device find label Settings focus --raw --last --json
{
 "success": false,
 "code": "COMMAND_FAILED",
 "message": "Ref @e300 is behind the visible keyboard and cannot be focused safely",
 "reason": "tap_keyboard_occludes_target",
 "ref": "@e300",
 "keyboardFrame": {"x": 0, "y": 74, "width": 2264, "height": 1006},
 "dispatched": "no"
}

$ agent-device find label Settings type x --raw --last --json
{
 "success": false,
 "code": "COMMAND_FAILED",
 "message": "Ref @e300 is behind the visible keyboard and cannot be focused safely",
 "reason": "tap_keyboard_occludes_target",
 "ref": "@e300",
 "keyboardFrame": {"x": 0, "y": 74, "width": 2264, "height": 1006},
 "dispatched": "no"
}

keyNodes: 0 is the point of that pair: with no keys in the tree, the tree-derived rule cannot fire, so those refusals can only be the measured band reaching the guard through match.keyboard.

Two notes on the Android surface, both reproducible:

The refusal needs --raw. In the interactive projection the same app node is marked interactionBlocked: covered by the helper's cross-window scan, so the projected route stops it one stage earlier with a different reason: find label Settings focus answers AMBIGUOUS_MATCH (the app node plus Gboard's own Settings key) and find label Settings focus --last answers reason: "target_covered". So on Android with this app the misfire is blocked by occlusion when the node is projected as covered, and by this guard when it survives as a match. In portrait no app node is centered in the band at all (adjustPan moves the window), so landscape is the reachable case.
ime set is required, settings put secure default_input_method is not enough. My first Android passes reported keyboard: {"kind":"absent"} with dumpsys input_method saying mInputShown=true, and the guard dispatched behind what looked like a keyboard. That was my setup, not the producer: the bound IME was the app's bundled headless com.callstack.agentdevice.imehelper, which reports itself shown while drawing no window — the screenshots from those runs have no keyboard in them, so there was nothing to occlude and the dispatch was correct. After adb shell ime set com.google.android.inputmethod.latin/... the band measures on every capture. No producer bug to report.

Validation

Head a0135d0d6, base 9ca333d5f. check:affected --run --head a0135d0d6: all runnable checks passed, exactCommit=true (receipt run-tQKoZm). check:layering: OK.
Fail-before / pass-after: with the guard call removed, the seam test fails 2 of 4 (response.ok true, focus dispatched); with it, 4 of 4. The 6 contract scenarios pass.
The four guarantee waivers cite #3734 (opened for exactly these cells); outcomeObservation stays on #3335.
CI obligation: Integration Tests, Coverage, device lanes at the head — not claimed locally.

🤖 Generated with Claude Code