perf(agent): wait 1 s, not 2 s, after a caret key in a focused text field

KD
6d ago
+268-12
  • After press or pressKey, a settled look waits up to 2 s (FULL_CHANGE_WAIT_MS) for the tree to change shape. A caret key (ArrowLeft, ArrowRight, Home, End, Control/Meta+a, with Shift or Control) in a textbox or searchbox that already holds the focus usually changes nothing the tree records, so the whole window was spent.
  • That wait is now 1 s (HELD_STILL_MS) for those keys: changeWaitAfter in packages/e2e/src/agent/settle-policy.ts. The first version used 150 ms; review showed a selectionchange handler that reveals a toolbar 700 ms later was missed, so the window now covers it.
  • Kept at 2 s: taps, the first key or tap that gives a field the focus, combobox, Alt chords, Tab and the vertical keys, and every other action.
  • armChange (observation-feed.ts) keeps the later deadline when a window is still pending. Without it, tap(Save) then pressKey('End') in one turn would cut the tap's 2 s window. focusedNode() returns a node only when exactly one node has the focus.
  • Known limit: a handler that updates the tree more than 1 s after the key is read on the next look, as after any action with a window shorter than its delay.

Verified

Ran it locally: yes

  • Surface: the web benchmark's tests-agent/ replay with --strict-cache, through pnpm bench:ab --config e2e.agent.config.ts against the merge base (e27f492c), 6 pairs in random order.
  • The 7 tests that call the model on a replay (3 planted-bug judgments, 4 act completes … scenarios) are left out of bench:ab with --grep-invert, since their model calls differ between runs. They ran once on the 150 ms version of this branch with a key (CI=1 e2e run --config e2e.agent.config.ts --ai-trace, recordings read-only): 33 tests passed, no REPLAY_STALE. On this head CI's web agent job ran them with a key: 33 passed, 20 model calls. The --no-cache path was not run.
  • Behavior: every test ran the same steps with the same outcome (status, cache mode, replayed actions) on every run of both builds.
Metricmainthis branchchange (95% CI)
wall (process)68.71 s62.86 s-9.0% (-11.4% to -5.8%)
suite (sum of attempts)67.52 s61.67 s-9.2% (-11.6% to -5.9%)
observe (events)38.70 s32.78 s-15.3% (-15.6% to -15.0%)
action (events)25.89 s25.64 ssame (-7.8% to +6.4%)
act completes rich-text-editor20.49 s14.58 s-28.9% (-29.3% to -28.4%)
  • Review regression test: tests/integration/agent-caret-settle.test.ts has a fake-engine popup that opens 700 ms after press and after pressKey on a focused field. Both cases fail at the 150 ms window and pass at 1 s. The same file keeps a first tap, a tap on the focused field, a caret key before the field held the focus, and a batched tap then key at the full window.
  • e2e unit 3514 pass, agent integration 203 pass, pnpm check exit 0. The iOS and Android agent and deterministic suites passed in CI on an earlier head, since settle-policy.ts is shared with mobile.
  • The replay suite cannot catch a delayed effect: every changing look in it ends within about 120 ms, which is why the fake-engine file exists.

At 150 ms the replay took 59.75 s (-16.1%) and the rich-text step 9.40 s. Taps stay at 2 s for the same reason: a click can open a panel that renders late. Five dead looks of about 2.1 s (10.7 s) remain: right clicks in the context-menu trap, the upload tap, the gift-card amount tap, and the editor's bold tap.

šŸ¤– Generated with Claude Code