feat(settle): prototype an adaptive early exit behind an internal switch (off by default)

TH
3h ago
+1158-41

Summary

This is a prototype of an adaptive short confirmation for --settle, for review as is. It is off by default. It runs only when the daemon host environment sets AGENT_DEVICE_SETTLE_ADAPTIVE=1, which the daemon passes on as SettleParams.earlyExit: 'adaptive'. It adds no CLI flag and does not change the output shape. With the switch off, every path matches main; the loop refactor is behavior-neutral.

Flipping the default is not part of this PR. It needs:

  • the ADR 0011 settleObservation guarantee;
  • its interaction-guarantees.ts matrix cell;
  • the --settle help.

The rule (settle-adaptive-confirmation.ts):

  1. The first capture that differs from a fresh pre-action baseline gets one confirming capture 110 ms later.
  2. If that capture is equal and from the same producer/presented surface, settle.
  3. These conditions escalate to today's quiet window:
    • the two captures disagree;
    • a busy indicator is on screen (the classification from #4513, now shared in kernel);
    • the presented surface changed;
    • the 1.5 s broad-replacement floor applies;
    • the capture holds the baseline beside a new screen (a transition union);
    • the baseline was not captured by this action, and the daemon does not report the session observation current. Current means: the frame is active, its tree is the newest observation, and nothing outdated it (wait text, Android freshness reads).
  4. If nothing differs from the baseline, the quiet window applies as today.

Deadlines, quality checks, private-AX recovery and diff output are unchanged.

The PR touches 20 files, including a test-only Settle lab fixture screen (agent-device-test-app://settle-lab).

Results

Each cell is N=15 (tab: 30), with 2.5 s post-checks (diff snapshot against the stored settled tree) and blocks rotated across arms.

  • on/off: pass 2, built at 3e048a7f7c.
  • main: pass 1, built at 244d1ff683.
  • Wrong: settled, but the screen changed afterwards.

iOS (iPhone 17 Pro sim, snapshot -i frame, bridge producer, 300 ms window)

scenariowall p50/p95 ms on · off · mainsettle wait p50 on · offcaptures on · offwrong on · off · main
counter806/3080 · 1002/2628 · 1122/3625221 · 4172 · 20 · 0 · 0
fill2913/3959 · 3084/4176 · 3289/5403327 · 5052 · 21 · 1 · 0
tab1128/1170 · 1359/1562 · 1378/1610293 · 4972 · 20 · 0 · 0
push1804/2292 · 1903/3063 · 1901/2256935 · 9683 · 30 · 0 · 0
back1844/1985 · 2013/2262 · 1997/2500476 · 6582 · 20 · 0 · 0
open-automation1582/1817 · 1605/2165 · 1692/2699927 · 9273 · 32 · 2 · 0
back-to-settings1621/3686 · 1797/4041 · 1957/4550269 · 4752 · 20 · 0 · 0
sheet-present1203/1307 · 1220/1333 · 1298/1633518 · 5182 · 20 · 0 · 0
sheet-dismiss2414/2591 · 2428/2502 · 2503/30161680 · 16975 · 50 · 0 · 0
delay5001187/1323 · 1354/1495 · 1407/1472603 · 7783 · 30 · 0 · 0
delay800997/1331 · 1048/1298 · 1004/1134418 · 4352 · 215 · 15 · 15
overlay805/837 · 1071/1634 · 1036/1134225 · 4422 · 215 · 15 · 15
stage-spinner997/1382 · 1391/2256 · 1000/2485418 · 8122 · 314 · 6 · 10
stage-plain827/1105 · 1067/2375 · 1043/1485238 · 4342 · 215 · 10 · 10
toast-nav814/1560 · 1052/2959 · 1066/3162226 · 4352 · 215 · 14 · 13

Android (API 37 AVD, full snapshot frame, helper, 500 ms window)

scenariowall p50/p95 ms on · off · mainsettle wait p50 on · offcaptures on · offwrong on · off · main
counter542/1034 · 780/1124 · 901/1276233 · 5402 · 30 · 0 · 0
fill694/1185 · 1044/2695 · 1249/1851214 · 5602 · 30 · 0 · 0
tab1523/2239 · 1856/3021 · 1635/2473972 · 12644 · 30 · 0 · 0
push958/2972 · 1270/2412 · 1361/1956678 · 10073 · 40 · 0 · 0
back1005/1824 · 1392/2879 · 1381/1512791 · 11753 · 40 · 0 · 0
open-automation1024/2427 · 1414/1790 · 1474/1982779 · 11553 · 40 · 0 · 0
back-to-settings968/1689 · 1321/2020 · 1356/1735743 · 10963 · 40 · 0 · 0
sheet-present806/1112 · 1225/1449 · 1454/2124566 · 9713 · 40 · 0 · 0
sheet-dismiss587/754 · 1003/1196 · 1131/1875342 · 7312 · 30 · 0 · 0
delay5001008/1248 · 804/1546 · 1344/1933752 · 5454 · 32 · 8 · 3
delay800775/959 · 777/1615 · 796/906536 · 5383 · 315 · 14 · 15
overlay675/819 · 828/1650 · 1126/2180321 · 5612 · 315 · 12 · 10
stage-spinner825/1600 · 853/1547 · 1486/1689582 · 5783 · 312 · 9 · 6
stage-plain469/697 · 1059/1997 · 804/1657228 · 5702 · 315 · 8 · 11
toast-nav687/719 · 810/1854 · 865/2076456 · 5543 · 315 · 8 · 10

Delayed effects with an immediate acknowledgement (re-measured at 7dad132c87 after review). The Settle lab now shows a "Run N" marker as soon as an action starts, so these cells measure "acknowledgement now, result later". In the tables above, the delay500/delay800 rows measured "no change, then a late change".

platformscenariowall p50/p95 ms on · offsettle wait p50 on · offwrong on · off
iOSdelay500812/3506 · 1362/4003230 · 78415 · 0
iOSdelay800811/907 · 1001/1066231 · 42515 · 15
Androiddelay500483/740 · 1504/1793234 · 126615 · 2
Androiddelay800472/711 · 1703/1794228 · 146415 · 6

The early exit settles on the acknowledgement every time. Today's window still catches a 500 ms result on iOS, and most Android results.

Checkout flow (10 steps, 9 settled; N=15; p50/p95 total):

  • iOS: on 25.6/29.1 s, off 27.5/36.2 s, main 27.5/32.4 s.
  • Android: on 11.9/21.2 s, off 14.9/21.1 s, main 16.2/18.0 s.

Which escalations mattered:

  • Transition union. Pass 1 (210c5b1f6f) ran without it. There, iOS stack pushes settled mid-transition: 9/15 and 14/15 wrong, versus 0/15 for off. With it, both match off.
  • Frame currency. In pass 1, Android never armed: selector resolutions run in a different view than a full-snapshot frame.
  • Presented surface and broad floor. These keep the sheet cases at today's latency.
  • Busy indicator. It blocks the early exit but cannot lengthen today's window. The 600 ms spinner plateau beats 300/500 ms in every arm.

What the rule makes worse: an action that acknowledges immediately and finishes later (above) is always wrong with the switch on. Also, unmarked multi-stage effects go from often wrong to always wrong. These are the plain intermediate step, toast-then-navigate, and the fading overlay. Today's rule already misses them 8–15/15 times, as it does delay800.

Recommendation

Don't flip the default yet.

  • For single-stage effects (in-place, fill, tab, push, back, sheet), the rule saves ~200 ms (iOS) and ~300–400 ms (Android) of settle wait. On those it adds no wrong settles.
  • Both producers gain, so a per-producer default isn't justified.
  • Next steps before a default:
    • a busy-indicator hold (wait for it to leave);
    • a run on real apps.

A default change would need these updates (none are in this PR):

  • the ADR 0011 settleObservation guarantee;
  • the interaction-guarantees.ts matrix cell;
  • the --settle help and docs;
  • settle-observation.test.ts;
  • the TS/Swift parity table, if required.

Validation

  • The touched suites pass. Each escalation was mutation-checked.
  • Failing-first tests:
    • the @ref freshness test fails on 9492f88;
    • the requested-window floor test (quietMs: 2000) fails with the c423b99 guard.
      Both pass at the head.
  • Typecheck passes.
  • Measured on the iPhone 17 Pro simulator and the API 37 AVD; 1-minute load averages ranged from 5 to 86.
  • Caveats:
    • Android snapshot -i hides text, so Android uses full snapshots.
    • In the measured bundle, the overlay was laid out in flow; this is fixed in source.
    • The freshness fixes after the runs (9492f88, and observationCurrent at 421179e5d6) change only flows after wait text or after a newer observation, which the benchmark never takes.
  • CI at 8d80138216 was green (18 checks). The unrelated wait-target dedupe moved to #4618.
  • Device smoke lanes run once the PR is ready for review.

🤖 Generated with Claude Code