fix: compact instead of failing when the pre-send context check trips
Issue for this PR
Closes #1806
Type of change
- Bug fix
- Documentation
What does this PR do?
Long sessions could fail before sending when a character-based estimate ran ahead of provider-reported usage. Large recursive file listings were counted like opaque strings, even when the provider had reported ample room.
The estimator now counts directory components separately for recognizable listing lines and retains the dense-text surcharge for long opaque names. The pre-send guard can use prior input usage plus growth for an unchanged request prefix. A changed prefix, system prompt, tool catalog, or provider instruction uses the full estimate.
A pre-send size refusal compacts and retries once when the history is eligible. The refusal remains an error when the retried request still cannot fit, there is no prior turn, or auto-compaction is off. A new or edited image or PDF blocks pre-send compaction until the current model has seen it.
How did you verify your code works?
- With two roughly 38k-character listings and 49k reported prompt tokens, main failed before sending. The previous guard compacted and retried. The updated guard sends once with the full 16,384-token reservation.
- Targeted suites cover already-seen and edited media, append-only and replaced inputs, model and agent changes, no usage, toolless and deadline steps, disabled compaction, reasoning budgets, small windows, and dedicated input limits.
- Locally, 786 targeted tests passed, along with typecheck and the strict marker check. Codex found no P0–P3 issues at
966b9a4461. The paired live-model release check is pending.
Cost side effects
An eligible size refusal can trigger one summarization call. The path estimator reduces unnecessary compaction on large listings.
Screenshots / recordings
No UI change.
Checklist
- I tested the change locally.
- I kept the change to request sizing and compaction.
- I checked cost side effects.
[!NOTE]
Medium Risk
Changes core request sizing, compaction triggers, and error recovery on every model call; mistakes could over-send oversized prompts or compact when media must be preserved.
Overview
Fixes sessions that died on pre-send context guard failures when character estimates overshot real provider usage (especially SQL/JSON and huge file listings).
The token estimator now treats directory separators in listing lines as breaks in dense ASCII runs so recursive path dumps are not wildly over-counted. Pre-send sizing can calibrate from the last turn’s reported input plus incremental growth when message/system/tool overhead is unchanged; otherwise it falls back to the full estimate (including when reported usage is below 60% of the estimate or metadata is missing).
Pre-send budget errors (OutputTokenBudgetError, input limits, and context-induced reasoning clamp failures) trigger auto-compaction and one retry when eligible; refused attempts do not count toward step limits. New or edited images/PDFs the model has not seen block that auto-compact path and surface guidance to run /compact. Docs add a Pre-Send Size Check section. Coverage is expanded with a large context-guard-compaction integration suite.
Reviewed by Cursor Bugbot for commit 966b9a446132cf8b642eba977c04e0654a4b64af. Bugbot is set up for automated code reviews on this repo. Configure here.