fix(drivers): read byte-bounded Mongo cursors in 1,000-document batches

AN
20m ago
+13-5

Issue for this PR

Closes #1479

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Why. Byte-bounded MongoDB reads set cursor.batchSize(1), so every document cost one network round trip. A materialization of about 127k documents took 6 to 11 minutes, almost all of it latency, and could exhaust a caller's time budget before any analysis ran.

Root cause. The byte budget was enforced per document, and the cursor was given a batch size of one so it would never fetch past the cap. That bound was unnecessary: the budget already stops consumption at the first document past the cap, and a server reply never exceeds 16 MiB.

What changed. Byte-bounded reads now request reply batches of 1,000 documents (BOUNDED_READ_BATCH). Was: one document per round trip. Now: up to 1,000 per round trip, with the byte budget still stopping at the first document past the cap. Reads without a byte budget are unchanged.

Base. Stacked on feat/analyst-harness-dab-2026-10 (#1441), which introduced the byte-bounded read path. The bug is not on main.

Behaviour deltas. Only the batch size of byte-bounded MongoDB cursors changes. Results, truncation and the byte cap are unchanged. At most one extra reply batch (up to 1,000 documents, at most 16 MiB) may be fetched past the cap and discarded.

How did you verify your code works?

  • Drivers package, full suite: 638 pass, 0 fail. source-byte-limit.test.ts now asserts the batch size used for bounded reads.
  • packages/opencode typecheck: clean.
  • Live, against a real MongoDB server: a bounded read of a 127,600-document collection took 381ā€“643 s at batch size one and takes 6ā€“14 s with this change (ten recorded agent sessions on the fixed build).

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

šŸ¤– Generated with Claude Code

[!NOTE]
Low Risk
Targeted performance change to cursor batching only; byte-limit semantics and returned results stay the same aside from possible extra prefetched documents discarded by the budget.

Overview
Fixes a severe latency regression on byte-bounded MongoDB find/aggregate reads by no longer forcing cursor.batchSize(1) (one network round trip per document). Those reads now use a shared BOUNDED_READ_BATCH of 1,000 documents per server reply while the existing byte budget still stops consumption at the first document past maxResultBytes; unbounded reads are unchanged.

Tests in source-byte-limit.test.ts were updated to expect the new batch size and to clarify that mocks count documents the budget iterates, not driver prefetch past the cap.

Reviewed by Cursor Bugbot for commit 5bafec40d807389f2476ec352c2abbb281d273e5. Bugbot is set up for automated code reviews on this repo. Configure here.