fix(conformance): wait for Settings to detach before the tap-retry baseline

Merged
TH
4d ago
+26-0

Summary

The nightly tap-retry-if-no-change differential fails because agent-device records zero retries. Not a retry regression; the fixture is still inert. The Blacksmith migration (#4185, d47bc9d3b7) moved the lane to runners where iOS snapshots use the simulator-ax-bridge backend instead of xctest. That backend skips app quiescence: after the inert fullScreenModal reaches its final frame, Settings stays attached for a few hundred ms (49 nodes vs the inert surface's 7). Settle latched on that combined tree, the retried tap took its baseline from it, and the inert-only post-tap tree read as changed, so no retry.

The flow now asserts open-inert-surface is not visible before the retried tap, so the baseline comes from the inert surface. tapRetries >= 1 is unchanged. A device-free test pins that the wait comes before the tap. 2 files. Fixes #4230.

Validation

  • Base 90c959518e, lane runner (run.ts --only tap-retry-if-no-change) on an iPhone 17 Pro Max sim, iOS 27.0, AX bridge, with the Maestro engine stubbed (Maestro CLI is not installed on this host): 4 of 7 runs failed with tapRetries was 0. Signature dumps: every failure resolved the target on the 49-node tree.
  • Head b79d867df9: 13/13 runs passed with tapRetries reached 1, and the baseline always resolved on the 7-node tree.
  • invariants.test.ts: the new test fails on the base flow and passes at head.
  • Conformance Differential dispatched on this branch at b79d867df9 (real Maestro, simulator-ax-bridge): all 6 scenarios ok, tapRetries reached 1. https://github.com/callstack/agent-device/actions/runs/42185348064
  • pnpm check:affected --run --head b79d867df9: passed, exactCommit=true (receipt run-Rh7hW1).

šŸ¤– Generated with Claude Code