test: reproduce large→small model-switch overflow (GLM-5.2 512k → GPT-5.5 272k)

Closed
SN
4mo ago
+124-0

Bug

When switching from a large-context model to a smaller one mid-session (e.g. GLM-5.2 512k → GPT-5.5 272k), the historian does not reduce the context to fit the new model before sending the prompt, resulting in Input exceeds context window errors.

Root cause

The model-change branch in transform.ts:502-547 detects the switch and clears lastContextPercentage / lastInputTokens to 0. This suppresses every reduction path on the same pass:

  • Historian trigger (checkCompartmentTrigger): 0% < proactive floor → shouldFire=false
  • 95% emergency block (transform-compartment-phase.ts:328): 0% < 95% → no block
  • Overflow recovery bump (transform.ts:616-628): needsEmergencyRecovery was just cleared → no bump

The oversized prompt — sized for the old model's window — is sent to the new smaller model and rejected. Recovery only arms on the second pass, after the real overflow error fires the event handler's recordOverflowDetected. The user sees the error on the first request.

Fix direction

In the model-change branch (transform.ts:502), before clearing, compare sessionMeta.lastInputTokens (the old model's last measured input tokens) against the new model's context limit (resolveTrustedContextLimit). When oldInputTokens > newContextLimit, call recordOverflowDetected(db, sessionId, newContextLimit, newModelKey) to arm recovery — which makes the existing :616-628 bump-to-95% path fire on the same pass, so historian + emergency drops run before the prompt is sent.

Reproduction

This PR adds a failing test (transform.test.ts) reproducing the scenario:

  • Session at 300k input tokens on GLM-5.2 (58% of 512k — no trigger fired)
  • Switch to GPT-5.5 (272k) — 300k is ~110% of the new window
  • Asserts needsEmergencyRecovery === true and the oversized tool output is dropped

The test currently fails: needsEmergencyRecovery is false and the tool output stays active.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


Summary by cubic

Adds a failing test in packages/plugin/src/hooks/magic-context/transform.test.ts that reproduces an overflow when switching mid-session from a large-context model (GLM-5.2 512k) to a smaller one (GPT-5.5 272k).
The test expects emergency recovery to arm immediately and drop oversized tool output before send; it currently fails, confirming the model-change branch clears usage and allows an oversized prompt through.

Greptile Summary

This PR adds a single failing test that documents the large→small model-switch overflow bug: when a session built up 300k tokens on GLM-5.2 (512k) is switched to GPT-5.5 (272k), the model-change branch in transform.ts clears all pressure state to zero, which suppresses every reduction path so the oversized prompt is sent to the new smaller model unchanged.

  • The test correctly reproduces the bug mechanism — liveModelBySession holds the new model while the last assistant message still carries the old model, triggering the model-change branch, and the session meta carries lastInputTokens: 300_000 against a 272k new window.
  • Two structural problems prevent the test from turning green once the fix lands: the test does not write a synthetic models.json with GPT-5.5's context limit (so resolveTrustedContextLimit returns undefined and the fix's overflow comparison never fires), and the OpenCode DB is seeded with id: \"m-raw-assistant-old\" while the live messages array uses id: \"m-assistant-old\" (a cross-reference mismatch that can affect message-index reconciliation and tool-tag eligibility).

Confidence Score: 2/5

The test is safe to merge in isolation — no production code is touched — but as written it will not turn green when the corresponding fix lands, leaving CI in a permanently failing state for this case.

The test correctly identifies the bug and the right observable outcomes, but cannot fulfill its role as a reproducer: without a models.json fixture carrying GPT-5.5's context limit, the fix mechanism gets undefined for the new model and skips arming recovery entirely. A separate OpenCode DB / live-messages ID mismatch could cause the tool-tag assertion to fail for an unrelated reason even if recovery were somehow armed. Both issues need to be resolved before the test can serve as a reliable regression guard.

packages/plugin/src/hooks/magic-context/transform.test.ts — specifically the new describe block starting at line 2551.

Important Files Changed

FilenameOverview
packages/plugin/src/hooks/magic-context/transform.test.tsAdds a failing regression test for the large→small model-switch overflow bug; has two blocking structural issues: no synthetic models.json mock for GPT-5.5 (so resolveTrustedContextLimit returns undefined and the proposed fix never arms recovery), and an OpenCode DB message ID mismatch (m-raw-assistant-old vs m-assistant-old in the live messages array) that could break tool-tag eligibility checks.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant TF as transform.ts
    participant MC as model-change branch (502-547)
    participant OVF as overflow state (DB)
    participant LCU as loadContextUsage
    participant BUMP as 95% bump path (616-628)
    participant ED as emergency-drop

    TF->>MC: "liveModel=GPT-5.5, lastAssistant=GLM-5.2 → mismatch"
    MC->>OVF: clearEmergencyRecovery()
    MC->>MC: contextUsageMap.delete(sessionId)
    Note over MC: (BUG) no recordOverflowDetected called
    Note over MC: (FIX) resolveTrustedContextLimit(GPT-5.5)<br/>needs models.json mock — returns undefined without it
    MC->>LCU: loadContextUsage → 0%
    LCU-->>TF: "percentage=0%"
    TF->>OVF: "getOverflowState → needsEmergencyRecovery=false"
    Note over BUMP: 0% < 95% but needsEmergencyRecovery=false → no bump
    TF->>ED: skipped — no 95% bump
    Note over ED: tool output stays active → prompt sent oversized → overflow error
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant TF as transform.ts
    participant MC as model-change branch (502-547)
    participant OVF as overflow state (DB)
    participant LCU as loadContextUsage
    participant BUMP as 95% bump path (616-628)
    participant ED as emergency-drop

    TF->>MC: "liveModel=GPT-5.5, lastAssistant=GLM-5.2 → mismatch"
    MC->>OVF: clearEmergencyRecovery()
    MC->>MC: contextUsageMap.delete(sessionId)
    Note over MC: (BUG) no recordOverflowDetected called
    Note over MC: (FIX) resolveTrustedContextLimit(GPT-5.5)<br/>needs models.json mock — returns undefined without it
    MC->>LCU: loadContextUsage → 0%
    LCU-->>TF: "percentage=0%"
    TF->>OVF: "getOverflowState → needsEmergencyRecovery=false"
    Note over BUMP: 0% < 95% but needsEmergencyRecovery=false → no bump
    TF->>ED: skipped — no 95% bump
    Note over ED: tool output stays active → prompt sent oversized → overflow error

Reviews (1): Last reviewed commit: "test: reproduce large→small model-switch..." | Re-trigger Greptile

Greptile also left 3 inline comments on this PR.