feat: let plugins encode state updates for the browser

BE
17h ago
+1247-50

Our app's state holds SQLAlchemy models, and many vars hold the same model: every row of a list points to one product, and relationships point back at their parents. Sent as plain JSON, a shared model is repeated in every place that holds it, and a cycle cannot be sent at all. So we encode each update before it is sent: models held more than once go into a table and are replaced by pointers, which the browser resolves again before applying the delta.

Today that needs a patched reflex, because there is no hook between the backend's last look at an update and the wire, and none in the browser before applyDelta. Since the first hydrate is diffed against the compiled defaults and deltas are merged in hydrate_and_load, an encoding can no longer be applied while the delta is built: a pointer means whatever its table says, so two equal-looking values may stand for different data.

This PR adds that hook to the plugin API:

  • Plugin.encode_state_update(update) gets the fields of each update that is sent (delta, events) and returns them encoded. It may add fields of its own next to delta. It runs once per update, where the update is sent, after the backend is done comparing and merging deltas: EventNamespace resolves the plugins that override the hook once, and emit_update and the upload stream encode with them. StateUpdate and its serializer are unchanged, and an update forwarded to another instance through redis travels plain and is encoded by the instance holding the socket.
  • Plugin.get_state_update_decoder() names a JS module whose default export decodes a received update. context.js imports the decoders and registers them in the app registry, and the update handler in state.js runs them before it reads the update.
  • The compiled initial state goes through both, as an update whose delta is the initial state, so a plugin only handles one format. The snapshot hashes for the first hydrate are still computed from the plain values.
  • Plugins encode in configuration order and decode in reverse.
  • A decoded update may hold one object in several places, in one substate or several, or contain cycles. A decoder that leaves such objects lists the substates holding them in the update's shared_substates. The update handler gives the deltas of those substates one shared set of pairs, and for those applyDelta pairs every object with the previous value it is first reached at and gives it one result, which every place holding it holds, across substates and on a cycle too: the previous value if the object deep-equals it with everything it reaches, else the object itself, or a copy sharing what is unchanged. Every other substate, and every delta of an app without decoders, takes the plain structural sharing path, unchanged.
  • An app without state gets no decoders: no update reaches it, and its empty initial state is not encoded.

Without plugins that override the hooks, updates are sent and applied as before: the send path checks an empty tuple, and the only changes to the output are an empty stateUpdateDecoders array in context.js and its registration.

Tests: only plugins overriding the hook encode, in configuration order, and serializing an update leaves the encoding to where it is sent (test_state.py); emit_update and the upload stream send the encoded update, or the update as is without encoders (test_app.py); the compiled context imports the decoders in reverse order, embeds the encoded initial state and keeps the hashes (test_compiler.py); context.js decodes the initial state before registering it (test_context_js_template.py); the update handler decodes before dispatching and reports a failing decoder (client_state_updates.mjs); and applyDelta shares objects a decoded delta holds twice, also across substates, keeps its cycles leading back to the objects the state holds, shares a decoded tree like a plain one, compares the substates a decoder does not list structurally, and keeps these invariants over random shared and cyclic graphs spread across substates (test_state_js_template.py). An update with only events reaches the encoders without a delta (test_state.py). The docs gain a section on encoding state updates.