Key the zone-map sidecar by the column's offset in its file

BR
1d ago
+184-6

The zone-map sidecar for a disk-backed column is named after the file alone (<file>.xyzones), and its header only checks the file's size and mtime plus the row count. Two columns that map the same file with the same length therefore share one sidecar, and whichever was folded last is loaded for both on the next build.

That happens with the ordinary ways of keeping a table on disk: the rows of a 2-D np.memmap, or several np.memmap(offset=...) columns packed into one file.

Repro (main @ 72a069f)

import numpy as np, xy
n = 1000
table = np.memmap("table.f64", dtype=np.float64, mode="w+", shape=(2, n))
table[0] = np.linspace(0, 1, n)        # x
table[1] = np.linspace(1000, 2000, n)  # y
table.flush(); del table

for build in range(2):
    mapped = np.memmap("table.f64", dtype=np.float64, mode="r", shape=(2, n))
    spec, _ = xy.scatter_chart(xy.scatter(x=mapped[0], y=mapped[1])).figure().build_payload()
    print(build, spec["x_axis"]["range"], spec["y_axis"]["range"])

0 [-0.03, 1.03] [970.0, 2030.0]
1 [970.0, 2030.0] [970.0, 2030.0]   # x axis now spans y's values; every point is off-screen

The first build is correct and writes the sidecar. Every later build (the next viewer restart, or a second figure in the same process) gives x the y column's min/max, so the chart comes up empty. With offset= columns the second column is wrong on its very first read, because the first column's sidecar is already there.

Fix

  • _ooc.backing_offset(arr) returns the byte offset of a column's first element in its file. It is measured from the root memmap, because a sliced memmap inherits its parent's .offset.
  • _zone_cache_path names every sidecar <file>.<offset>.xyzones. The offset is always written, .0 included, so a file/offset pair can't collide with a file whose name ends in .<digits>. A memmap-typed view with no mapping behind it counts as not disk-backed.
  • The sidecar magic goes from XYZONEC1 to XYZONEC2, so a v1 sidecar that may already hold another column's maps is refolded once instead of trusted.

In-RAM columns are not affected. Existing sidecars are refolded once because the name and magic changed.

Tests

  • tests/test_ooc.py::test_zone_map_cache_keeps_views_of_one_file_apart covers the 2-D table case above through xy.chart(xy.scatter(...)), with a cold build and then a warm build.
  • tests/test_ooc.py::test_zone_map_cache_keys_on_file_offset covers two np.memmap(offset=...) columns in one file, plus a file named <file>.<offset>.
  • tests/test_ooc.py::test_backing_offset_without_file_metadata_is_not_disk_backed was added after review.

Both fail on main (assert -5.0 == 7.0; the warm build returns x as (1000.0, 2000.0)) and pass with the fix. The full suite passes on this branch (4096 passed, 113 skipped; main is 4094 passed, 113 skipped). ruff check, ruff format --check, and ty check python report no new diagnostics compared with main.

Checklist

  • News fragment: news/524.bugfix.md
  • Spec: the sidecar note in spec/process/perf-audit-2026-07-22.md now says how the sidecar is named per column
  • No public API, payload, or import-boundary change

Summary by CodeRabbit

  • Bug Fixes
    • Charts built from multiple disk-backed columns in the same file now retain the correct axis ranges instead of reusing another column’s cached statistics. Each column’s cached statistics remain separate, including when columns map different regions of the same file, so charts show the appropriate ranges after being built again.
    • After a replacement cache is written, obsolete cache files are removed when possible. Cleanup failures do not interrupt processing.