pi-plugin: survive NUL-padded served ledger tails and surface real busy causes
pi-plugin: survive NUL-padded served ledger tails and surface real busy causes
Fixes #723 (the lkg_miss / "storage is busy" failures after the interrupted-writer race corrupts served-array ledgers)
Problem
The interrupted-writer race (#743/#723) can leave NUL-padded tails on the pi-plugin's served-array ledger files (both the identity ledger and the digest ledger). On the next read, JSON.parse throws an untyped SyntaxError, the reader fails closed on the whole file, and the session loses access to its served-array data. Downstream this surfaces as:
lkg_missrefusals even though every valid record in the file is intact;- refusals rendered only as "Magic Context storage is busy; send your message again" because all storage-path errors — including non-transient ones like this parse failure — were classified as
PiStorageBusyError.
Changes
Fail-open on NUL tails, fail-closed on real corruption — served-array-ledger.ts
- The identity and digest ledger readers tolerate a trailing run of NUL bytes after the last complete JSONL record: the last valid record is adopted, and the file is rewritten in place to the last valid record — guarded by a size stamp so the rewrite is skipped whenever the file changed since the read (another Pi process may be appending concurrently). Verified the tail was written by padding (record bytes are unmodified) in production before doing this; the test suite covers a hole-padded multi-record tail. (Correction: an earlier version of this description mentioned a
*.bakbackup that the diff does not create — repairs are done in place, and the*.bak-corrupt-nultailfiles referenced in the issue come from the manual production repair, not from this patch.) - Genuine content corruption (invalid JSON within record headers/lines) still fails closed, but now as a typed
PiServedEnvelopeUnavailablewith the underlying cause attached, instead of an unwrappedSyntaxError.
Writer hardening against zero-filled appends — served-array-ledger.ts
appendPendingLines previously trusted the success return of an append flagged
"a". It now:
- Strips a pre-existing hole tail before appending (cheap 64 KiB tail probe;
full read only when NULs are found), reusing the reader's stamp-guarded
repair so appending never grows a hole into the middle of the next read. - Verifies the region it just appended (bounded read-back at the recorded
offset) and, on a zero-filled region, rewrites exactly those bytes via a
positionedwriteSync— only our own offsets, so a peer append past the
region keeps its bytes. - If the rewrite still verifies as zero-filled, the failure is recorded in the
same diagnostics counters as any other write failure; the next pass's reader
repair remains the last resort.
Partial-last-line handling (the maintainer's save-before/after-send keying in
#723) is intentionally out of scope here: incomplete records already fail
closed with a typed error, and committed identities remain authoritative in
SQLite, so no tag number can be reused either way.
Honest busy refusal — pi-raw-fallback.ts + context-handler.ts
- The broad catch that mapped any error in the storage path to
PiStorageBusyErroris narrowed to genuinely transient causes (real SQLite busy state / lock contention). Other storage-path errors throw a refusal whose message carries the underlying cause instead of implying a retryable busy state.
Tests (TDD, all added first as failing)
served-array-ledger.test.ts: 7 new tests (4 reader + 3 writer: pre-append strip, zero-filled append rewritten end-to-end via a fault-injectedwriteStorageFileSync, region-confinement of the positioned repair) — NUL-only tail on identity ledger (adopts last record, repairs file on disk), NUL-only tail on digest ledger (repairs on read), genuine corrupt content still fails closed with the typed error, raw JSON syntax failure surfaces aspi-served-envelope-unavailable.context-handler-degraded-pass.test.ts: the non-transient storage failure path now produces a refusal whose message names the real cause.
Verification
- New tests: 8/8 fail without the patch, pass with it (TDD).
tsc --noEmitforpi-plugin: clean.- Full suite
packages/pi-plugin(2910 tests across 290 files) run twice — once with the patch, once on a pristine-upstream baseline in the same worktree: 81 failure names on the patched run vs 83 on baseline, with an 81-name overlap; the run-to-run deltas are flaky, timing-bound stress tests and WindowsEBUSYon a temp-dir cleanup, and the LKG suite minus the patch reproduces exactly the same standalone failure set, name for name. No regression attributable to this change.
Notes for reviewers
- The repair-on-read backs up before truncating and only ever strips bytes = the pad; it does not try to resample or reconstitute records.
- Update: production-observed (Windows 11, bun, @cortexkit/magic-context 0.49.x) both refusal signatures —
JSON Parse error: Unexpected NUL byteand the corresponding padded-tail byte counts on~/.omp/...served-arrayledger files — before this change; both were locally repaired manually beforehand, and the repair described here keeps the same invariant that any new race event is converted from a persistent refusal into a self-healing read.