perf(runtime): bound the waits past stale data and in front of a Server Action

SO
19h ago
+718-56

Summary

Three bounded waits on the path to the first byte:

  • one deadline for everything asked past a stale or expired data value;
  • a Server Action's generation read runs while its route's module loads;
  • data memos are sized without stringifying their value.

Why

  • Stale data. Past a stale or expired data value, the runtime waits for a later one. It was documented as 200 ms at the most (500 ms past expired), but each step had its own budget, and the last step had none. In the worst case that came to about 800 ms past a stale value and about 1.4 s past an expired one, mid-render and before the first byte of pages without a shell.
  • Server Actions. A Server Action's postponed state needs a record read: up to 200 ms, plus up to 200 ms for a newer record. It ran only after the route's module had been evaluated, so the two waits added up.
  • Data memos. Keeping a data value measured its answer with JSON.stringify, which copied the whole inline base64: 0.6 ms at 512 KiB, 9.7 ms at 8 MiB.

Changes

  • laterIfAny sets one deadline (200 ms, or 500 ms past expired). The later-value read, its judgement, the probe and heldNow each get what is left. The probe is not started once the deadline has passed. Past it, heldNow still takes what the isolate has in hand — the write that overtook the later value, tags already brought up — where it settles within the turn, rather than answering a miss the render would compute again; what the host has to answer is not waited for. Late answers from the authority are still kept as the data center's copy.
  • serveWithBody starts the record lookup alongside module evaluation, under the route renderedBy will reach (read from the entry tables), for Node entries with a cache and an action handed state. It uses the early result only when the routes match, and calls it as before otherwise. Edge pages are unaffected.
  • dataBytes counts each short string (ids, handler, tags, artifact id) as JSON writes it, escapes included, and the inline base64 by its length alone, plus a fixed allowance, at two bytes a character, as before.

Tests

  • later-generations.test.ts: a 150 ms read plus a 100 ms judgement answers at 200 ms; no probe once the deadline has passed; a stalled heldNow is a miss at the deadline; past the deadline, the write that overtook the later value is still answered.
  • action-postponed.test.ts: the record read starts before module evaluation finishes; the same under a member's prerender route; no read for an edge page.
  • data-memo-bytes.test.ts: inline and artifact-reference values are sized with nothing written as JSON, and never below their JSON size.
  • Each new test was checked to fail without the behaviour it covers.
  • Locally: the touched runtime suites plus cache-reads, cache-mutations, the action-posting suites and the cache-data-copies workers suite; eslint, oxfmt, tsc (runtime), check:vocabulary.

Notes

Existing per-step data-wait tests, the memo budget tests and the data-copy workers suite all pass unchanged.

Review fixes

  • cubic: short strings are counted with their JSON escapes; the early route uses hasEntry; the expired-deadline comment spells out its 1.4 s; the artifact-id test fails where the id goes uncounted; a new test covers escape-heavy tags.

Not changed

  • cubic (held-reads.ts, the past-deadline read may start host work no render waits for): it happens only where a write overtook the later value after the whole wait was spent; what it finds lands in the isolate's data memo and tag view for the reads after it; a memory-only path would split both readData and the tag check for that one case.