fix(codex): scope token dedup key to fork parent so subagent replays collapse

Merged
RE
5mo ago
+29-5

Codex subagent/fork fan-out replays the parent's token_count history into every child file. Each replayed row has the same cumulative total but a distinct per-file session id, and the dedup key is scoped by session id, so the copies never collapse and every sibling gets counted. One real day reported 12.9B tokens and tripped the daily submit cap; deduped it is ~1B (#679).

Scope the codex token_count dedup key to forked_from_id instead of the child's own session id. Sibling replays share one key and collapse. Non-forked sessions keep their own id, so unrelated work never merges.

From 175 real session files on the affected day:

  • 87.9% of token_count rows are cross-file duplicates (4,450 rows replayed across 16 files each, etc).
  • counting each distinct cumulative row once drops the day from 13.7B to 1.65B.
  • 0 cumulative rows are shared across unrelated fork families, so parent-scoping only collapses real replays.

This changes test_..._deduplicates_parent_replay_across_forks. Its two sibling forks each do one own turn that lands on a byte-identical cumulative total, and it asserted both survive; with parent-scoped keys they collapse, so the assertion is updated. In real data, identical sibling cumulative vectors are the replay signature rather than independent work, since the cumulative encodes each fork's divergent context size. If you would rather preserve that case, the alternative is a cross-file pass that collapses only contiguous shared runs instead of single rows; happy to switch.

Closes #679.

Summary by cubic

Fix token overcounting in Codex by scoping token_count dedup keys to the fork parent (forked_from_id) so sibling subagent replays collapse. Also bump the message cache schema to reparse stale dedup_keys and apply the fix to previously scanned sessions; closes #679.

  • Bug Fixes
    • Scope the dedup key to forked_from_id to collapse identical cumulative token_count rows across sibling forks; fallback to the session id for non-forked sessions, so unrelated sessions never merge.
    • Bump message cache schema to 18 to force a reparse of cached messages with old per-child dedup_keys.
    • Update test to reflect collapsed sibling turns (3 messages; totals 140/14), preventing inflated daily counts.