Stop writing echoed browser storage values back, ending the sync=True cross-tab ping-pong
All Submissions:
- Have you followed the guidelines stated in CONTRIBUTING.md file?
- Have you checked to ensure there aren't any other open Pull Requests for the desired changed?
Type of change
- Bug fix (non-breaking change which fixes an issue)
Changes To Core Features:
- Have you added an explanation of what your changes do and why you'd like us to include them?
- Have you written new tests for your core changes, as applicable?
- Have you successfully ran tests with your changes locally?
What and why
Found in the 0.10.0a3 pre-release re-verification as A3-11 (regression vs 0.10.0a2, not vs 0.9.12) and A3-12 (pre-existing on 0.9.12, a2 and a3).
The bug. Since #7493 the boot event's delta carries the browser storage values the tab sent, so that get_delta overrides see them, as update_vars_internal's delta always did. The frontend writes every client-storage value in a delta back to the browser, echoes included. If another tab stores a newer value of a sync=True var while this tab is booting (or while its update_vars_internal is in flight), the echo overwrites the newer value and fires a storage event. The other tabs then send their own stale echoes back, and the tabs never stop answering each other:
- tens of thousands of websocket frames per 5 s;
- the backend at 50–70 % CPU;
- tabs left on mixed old values, so the user's last change is lost.
a2 was immune only because its boot did not echo anything. 0.9.12 echoed through its separate update_vars_internal and stormed the same way.
The fix is frontend-only, in state.js:
- Record the
sync=TruelocalStorage values a tab sends withhydrate_and_loadandupdate_vars_internal, per state key. A delta value equal to one of them is its echo. - Write a synced var's echo only over this tab's own later write: its last write to that storage name came after the value was sent, and storage still holds it (e.g. a handler's reply that the backend applied before the echoed event). Otherwise storage holds either the sent value (nothing to write) or a newer value from another tab, which is kept whether or not its
storageevent has arrived yet. The tab sends that newer value to its backend, so backend and storage end on it. Writes are tracked by storage name, which vars of different state keys may share.- Values that are not echoes, such as a handler's or a
get_deltaoverride's, are written as before. #7493's reconcile and the computed-var sanitising tests keep passing. - Cookies, session storage and localStorage vars without
sync=Trueare written as before. Their tab never sends another tab's value to its backend, so skipping one of their values could leave a handler's value out of storage for good. Cookies keep renewing theirmax_age.
- Values that are not echoes, such as a handler's or a
- On a localStorage
storageevent, queue the var and readlocalStorage.getItem(key)when the event is sent, rather than sendinge.newValue: a tab can receive another tab's event after a later write, and the event can wait for a reconnect. Session storage events from same-origin frames are ignored.
The decision reads localStorage at the moment the echo arrives, so it does not depend on whether the socket reply or the other tab's storage event comes first. A synced value whose echo a get_delta override replaced stays recorded until a matching value arrives, bounded at 256 values per key. A later handler value equal to it is still written over this tab's own write. Over another tab's newer value it is skipped only when it arrives before the reply to this tab's sync of that value; the backend then applies the sync after the handler, so storage and backend end on the newer value.
Why the bookkeeping is needed. A variant without it (storage events send getItem and update the tab's state locally, every echo is still written) does not stop the loop: writing a stale echo flips storage and the other tab answers the flip. It stormed in 1/3 runs of the change-while-loading scenario and 2/3 of the 6-tab on_load stamp scenario (32–36k frames per 5 s), and the two-tab boot test ping-pongs. Since the backend's reply does not say which values are echoes, the frontend remembers what it sent. Exact correlation (the backend marking echoed keys in each StateUpdate) would need a new field and processor plumbing; this PR stays frontend-only.
Evidence
The bootecho / syncstamp apps and the multi-tab drivers of the pre-release campaign were run against 4cfafa359 and against the published 0.10.0a3:
| scenario | published 0.10.0a3 | this PR |
|---|---|---|
| tab 0 changes a synced var 5× while 6 more tabs load, dev | 3/4 runs storm (48k–61k frames, never converge, last value lost) | 0/5 storm, all converge on the last value, 71–83 frames, 0 frames in the quiet window |
| same, prod | 2/5 storm (campaign run) | 0/3 storm, all converge, 95 frames |
on_load stamps a synced var in 6 tabs at once, dev (A3-12) | 4/4 storm (330k–478k frames) | 0/6 storm, 6/6 converge, 83–107 frames |
| same, 3 tabs, dev | storms on 0.9.12, a2, a3 (campaign) | 0/6 storm, 6/6 converge, 33–51 frames |
| same, 6 tabs, prod | — | 0/4 storm, 4/4 converge, 89–111 frames |
Re-run on the current head (2063422c8): the change-while-loading scenario 0/3 storm in dev (65–83 frames) and 0/2 in prod (87–91 frames), the 6-tab on_load stamp 0/3 storm in dev (86–139 frames), all converging on the last value with no frames in the quiet window.
Realistic network. The campaign's independent verifier found the original bug with one click, 3 tabs and a 100 ms RTT proxy, in a stock Chromium driven over CDP with real, throttled background tabs. Its scenarios against this PR (dev, 100 ms RTT, at 4cfafa359):
| verifier scenario | published 0.10.0a3 (verifier) | this PR |
|---|---|---|
| browser restart restores 3 tabs, one click (raw CDP, background tabs) | 3/3 endless ping-pong, 22–27k frames per 5 s | 0/3 storm, all converge; in every run a tab booted across the click and its boot delta echoed the old value |
| restore 7 tabs 300 ms apart, one click | storm in every run where a tab booted across the click | 0/4 storm, all converge (the race fired in 4/4) |
6 restored background tabs on on_load-stamping pages (A3-12) | 2/2 storm | 0/2 storm, all converge |
| 4 tabs on stamping pages, then a dev backend reload (A3-12) | 2/2 storm | 0/4 storm, all converge |
Tests
New Playwright tests in tests/integration/tests_playwright/test_hydration_storage.py, run in dev and prod:
test_synced_storage_echo_keeps_newer_value: tab B boots with the stored value while its websocket replies are held back, and tab A stores a newer value. Once the replies arrive, B must not write the old value. Onmain, B writesold, new, old, new.test_synced_storage_event_sends_stored_value: a late storage event syncs the value stored now. Onmainthe tab shows the event's stale value.test_synced_storage_event_while_disconnected_sends_current_value: a storage event that waits for a reconnect syncs the value stored when it is sent.test_synced_storage_echo_crossed_by_another_tab_is_not_written, with the newer value'sstorageevent arriving before and after the echo: the echo of a value another tab replaced is not written back. Fails onmainin both orders.test_synced_storage_echo_after_own_write_is_written(through the same var, or through another var sharing its storage name) andtest_replaced_storage_echo_does_not_hide_later_changeguard the cases that must still write: an echo over the tab's own later write, and a handler storing a value whose boot echo an override replaced. The latter runs for a synced var and a var that is not synced, each with and without another tab storing a value before the handler. All of them pass onmain. The var that is not synced, with another tab's write, failed on 33bf76643 (storage kept the other tab's value); the own-write cases fail if that case is dropped, and the shared-name case also fails when writes are tracked by state key.
All of these pass:
- the whole
test_hydration_storage.py, 32/32 in dev and prod; - the Selenium
test_client_storage.py(includingsync=Trueacross tabs),test_upload.py,test_connection_banner.pyandtest_call_script.py, 23/23; tests/units/compiler/test_state_js_template.pyandtests/units/compiler/state_js.test.mjs. The template test's mock builds the reconnect hydrate the wayReflexEventdoes, without a payload, which caught a reconnectTypeErrorin the first commit;- pre-commit: ruff, pyright, ty, biome.
🤖 Generated with Claude Code