perf(runtime): catch a request up through a window of recent invalidations

SO
17h ago
+1604-215

Summary

A request behind a recent invalidation no longer waits for a fresh isolate to page through the scope's whole tag history. It asks the authority for the window of recent invalidations, in one round trip, beside the full pull. Full pulls also ask for pages of 1000 tags instead of 500.

Why

catchUpWithScope waits up to 1 s before routing a request whose scope revision is ahead of the isolate's view, when the latest invalidation is less than 10 minutes old. The only way to catch up was the delta. A fresh isolate reads it from the scope's beginning, oldest first, 500 rows a page, one round trip per page, and tag rows are never removed. On a scope with thousands of invalidated tags, every early request of a fresh isolate waited the full second, and the invalidations the wait was for arrived last.

Changes

  • GET /tags/recent?since&from&limit on the gateway. It returns the existing recent window (recentDelta): the newest invalidations past since that took effect from from on, at the scope revision, truncated when there were more.
    • It has its own contract schema (tagsRecentResponseSchema). A truncated window is not a page of the delta, and the delta's page rule would reject it or mistake it for a page boundary.
  • Function client and host: tagsRecent(since, from), under the same authority read deadline as tagsDelta; platform tags are filtered; an older gateway's 404 is reported as unsupported. Plus optional CacheHost.tagsRecent in the runtime host contract.
  • tagsDelta asks for limit=1000 (MAX_DELTA_PAGE_TAGS), which the route already clamps to.
  • TagState.recent / seen:
    • A window's marks go into tag history and judge a request's reads, which are timestamp-based.
    • They never raise knownRevision or checkedAt, so holdsWhole, heldRevisionOf and syncLocal behave as before.
    • A mark past a render's revision counts as missed by that render.
    • A truncated window adds its marks but does not move seen.
    • The marks are held apart from the tags the delta brings (#recent), so a fresh isolate's delta, paging the scope's whole history after them, cannot let them go; each is let go of once the delta brings its tag at the mark's revision or later — not merely once the delta's revision passes it, since the delta answers each tag's latest row. Past as many as are held apart (4096), the first is lost as a delta mark is (#lost, losses: not held whole, an earlier render answers no more), and the view stands at the delta's revision for seen until a pull has run to its end past it, or a window asked from where the delta stands brings the marks back.
  • catchUpWithScope starts the window (shared per isolate) and the full pull together. It goes on once seen reaches the scope revision, or once both have settled short of it — still for 1 s at most. The full pull keeps running behind the response.
  • Regenerations, captures, speculative renders and commits still claim the full pull's revision.
  • Two refactors move code unchanged to stay within the 500-line lint limit: tag mark helpers into tag-marks.ts, and readWhole into read-budget.ts.

Tests

  • Contract: a truncated window is accepted where the delta schema rejects it; revisions past the window, and history past a row's own revision, are refused; at most 1000 rows.
  • Route: since, from and limit parsing; truncation; ordering.
  • Client: limit=1000 on both reads; deadline parity; a 404 becomes unsupported; platform tags filtered.
  • Tag view: a window's marks survive the delta paging more tags than the view keeps, and survive reads of as many other tags; a window mark let go of past the bound drops seen to the delta's revision until the delta passes it; an evicted tag plus a window mark is not held whole; a window mark past a render's revision stops answeringAtOnce; checkedAt stays unset; a truncated window leaves seen unchanged; unsupported is asked once; a lost window mark is remembered.
  • Regeneration after a window still claims the delta's revision.
  • Catch-up: the request goes on once the window brings the view up, with the pull still out; a truncated window waits for the pull; an older gateway falls back to the delta and is asked no more; no new catch-up once seen covers the scope.
  • Each new test was checked to fail without the behaviour it covers.
  • Locally: the touched runtime, edge (scope-revision), cache and host suites; eslint, oxfmt, tsc (runtime, host, edge, cache), check:vocabulary.

Rollout

  • The gateway serves the new route once it is deployed; until then the runtime gets a 404, reports unsupported, and catches up through the delta alone, as before.
  • The native cache has no tagsRecent: nothing there carries a scope revision, so it would never be asked.

Review fixes

  • Codex (window marks evicted by the delta's history while seen stood at the window's revision): the marks are held apart from the delta's tags, as above.
  • Codex (a mark let go of as the delta's revision passed it, though the tag was invalidated again and comes later): a mark goes only once the delta brings its tag.
  • Codex, Greptile (a mark let go of past the bound left the tag held whole and an earlier render free to answer): it is recorded as lost.
  • Codex (seen came back once the delta's revision passed a lost mark, though the tag may come later): it comes back only after a whole pull past it, or a covering window.
  • cubic: a row's history may not pass the row's own revision; the route test asserts the exact revision; the cache README says the window is asked beside the delta.

Not changed

  • cubic (tags.ts, a retried window clearing the lost-mark state after only a truncated delta page passed it): while a mark is lost the view stands at the delta's revision, so the retried window is asked from there (since = revision). It holds every tag's row past that revision — the rows the pages still out would bring included — and the rows at or before it are ones the delta has brought: rows only move forward, so a row there now was there when the page covering it was read. Only a window that left none out clears the state, before its own marks go in.