Stop writing echoed browser storage values back, ending the sync=True cross-tab ping-pong

Merged
MA
8d ago
+502-22

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

What and why

Found in the 0.10.0a3 pre-release re-verification as A3-11 (regression vs 0.10.0a2, not vs 0.9.12) and A3-12 (pre-existing on 0.9.12, a2 and a3).

The bug. Since #7493 the boot event's delta carries the browser storage values the tab sent, so that get_delta overrides see them, as update_vars_internal's delta always did. The frontend writes every client-storage value in a delta back to the browser, echoes included. If another tab stores a newer value of a sync=True var while this tab is booting (or while its update_vars_internal is in flight), the echo overwrites the newer value and fires a storage event. The other tabs then send their own stale echoes back, and the tabs never stop answering each other:

  • tens of thousands of websocket frames per 5 s;
  • the backend at 50–70 % CPU;
  • tabs left on mixed old values, so the user's last change is lost.

a2 was immune only because its boot did not echo anything. 0.9.12 echoed through its separate update_vars_internal and stormed the same way.

The fix is frontend-only, in state.js:

  1. Record the sync=True localStorage values a tab sends with hydrate_and_load and update_vars_internal, per state key. A delta value equal to one of them is its echo.
  2. Write a synced var's echo only over this tab's own later write: its last write to that storage name came after the value was sent, and storage still holds it (e.g. a handler's reply that the backend applied before the echoed event). Otherwise storage holds either the sent value (nothing to write) or a newer value from another tab, which is kept whether or not its storage event has arrived yet. The tab sends that newer value to its backend, so backend and storage end on it. Writes are tracked by storage name, which vars of different state keys may share.
    • Values that are not echoes, such as a handler's or a get_delta override's, are written as before. #7493's reconcile and the computed-var sanitising tests keep passing.
    • Cookies, session storage and localStorage vars without sync=True are written as before. Their tab never sends another tab's value to its backend, so skipping one of their values could leave a handler's value out of storage for good. Cookies keep renewing their max_age.
  3. On a localStorage storage event, queue the var and read localStorage.getItem(key) when the event is sent, rather than sending e.newValue: a tab can receive another tab's event after a later write, and the event can wait for a reconnect. Session storage events from same-origin frames are ignored.

The decision reads localStorage at the moment the echo arrives, so it does not depend on whether the socket reply or the other tab's storage event comes first. A synced value whose echo a get_delta override replaced stays recorded until a matching value arrives, bounded at 256 values per key. A later handler value equal to it is still written over this tab's own write. Over another tab's newer value it is skipped only when it arrives before the reply to this tab's sync of that value; the backend then applies the sync after the handler, so storage and backend end on the newer value.

Why the bookkeeping is needed. A variant without it (storage events send getItem and update the tab's state locally, every echo is still written) does not stop the loop: writing a stale echo flips storage and the other tab answers the flip. It stormed in 1/3 runs of the change-while-loading scenario and 2/3 of the 6-tab on_load stamp scenario (32–36k frames per 5 s), and the two-tab boot test ping-pongs. Since the backend's reply does not say which values are echoes, the frontend remembers what it sent. Exact correlation (the backend marking echoed keys in each StateUpdate) would need a new field and processor plumbing; this PR stays frontend-only.

Evidence

The bootecho / syncstamp apps and the multi-tab drivers of the pre-release campaign were run against 4cfafa359 and against the published 0.10.0a3:

scenariopublished 0.10.0a3this PR
tab 0 changes a synced var 5× while 6 more tabs load, dev3/4 runs storm (48k–61k frames, never converge, last value lost)0/5 storm, all converge on the last value, 71–83 frames, 0 frames in the quiet window
same, prod2/5 storm (campaign run)0/3 storm, all converge, 95 frames
on_load stamps a synced var in 6 tabs at once, dev (A3-12)4/4 storm (330k–478k frames)0/6 storm, 6/6 converge, 83–107 frames
same, 3 tabs, devstorms on 0.9.12, a2, a3 (campaign)0/6 storm, 6/6 converge, 33–51 frames
same, 6 tabs, prod—0/4 storm, 4/4 converge, 89–111 frames

Re-run on the current head (2063422c8): the change-while-loading scenario 0/3 storm in dev (65–83 frames) and 0/2 in prod (87–91 frames), the 6-tab on_load stamp 0/3 storm in dev (86–139 frames), all converging on the last value with no frames in the quiet window.

Realistic network. The campaign's independent verifier found the original bug with one click, 3 tabs and a 100 ms RTT proxy, in a stock Chromium driven over CDP with real, throttled background tabs. Its scenarios against this PR (dev, 100 ms RTT, at 4cfafa359):

verifier scenariopublished 0.10.0a3 (verifier)this PR
browser restart restores 3 tabs, one click (raw CDP, background tabs)3/3 endless ping-pong, 22–27k frames per 5 s0/3 storm, all converge; in every run a tab booted across the click and its boot delta echoed the old value
restore 7 tabs 300 ms apart, one clickstorm in every run where a tab booted across the click0/4 storm, all converge (the race fired in 4/4)
6 restored background tabs on on_load-stamping pages (A3-12)2/2 storm0/2 storm, all converge
4 tabs on stamping pages, then a dev backend reload (A3-12)2/2 storm0/4 storm, all converge

Tests

New Playwright tests in tests/integration/tests_playwright/test_hydration_storage.py, run in dev and prod:

  • test_synced_storage_echo_keeps_newer_value: tab B boots with the stored value while its websocket replies are held back, and tab A stores a newer value. Once the replies arrive, B must not write the old value. On main, B writes old, new, old, new.
  • test_synced_storage_event_sends_stored_value: a late storage event syncs the value stored now. On main the tab shows the event's stale value.
  • test_synced_storage_event_while_disconnected_sends_current_value: a storage event that waits for a reconnect syncs the value stored when it is sent.
  • test_synced_storage_echo_crossed_by_another_tab_is_not_written, with the newer value's storage event arriving before and after the echo: the echo of a value another tab replaced is not written back. Fails on main in both orders.
  • test_synced_storage_echo_after_own_write_is_written (through the same var, or through another var sharing its storage name) and test_replaced_storage_echo_does_not_hide_later_change guard the cases that must still write: an echo over the tab's own later write, and a handler storing a value whose boot echo an override replaced. The latter runs for a synced var and a var that is not synced, each with and without another tab storing a value before the handler. All of them pass on main. The var that is not synced, with another tab's write, failed on 33bf76643 (storage kept the other tab's value); the own-write cases fail if that case is dropped, and the shared-name case also fails when writes are tracked by state key.

All of these pass:

  • the whole test_hydration_storage.py, 32/32 in dev and prod;
  • the Selenium test_client_storage.py (including sync=True across tabs), test_upload.py, test_connection_banner.py and test_call_script.py, 23/23;
  • tests/units/compiler/test_state_js_template.py and tests/units/compiler/state_js.test.mjs. The template test's mock builds the reconnect hydrate the way ReflexEvent does, without a payload, which caught a reconnect TypeError in the first commit;
  • pre-commit: ruff, pyright, ty, biome.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SjCrbdguMkfe4bhbaREiwo