Bug Bench
Bugs found by cubic in AI-written code, grouped by model.
As of Nov 9, 2026, GPT-6 Luna had the lowest relative bug rate on Bug Bench (0.31, 69% fewer counted bugs than average), among 7 ranked AI coding models.
Ranking window Jul 20 to Nov 8, 2026
Relative bug rate
1.00averageabove averagelower is better
| Rank | Model | Rate |
|---|---|---|
| 1 | GPT-6 Lunanew this week | 0.31 |
| 2 | GPT-6.1 SolDown 1 since last week | 0.74 |
| 3 | Claude Fable 5.1Down 1 since last week | 0.88 |
| 4 | GPT-6 AstraDown 1 since last week | 0.95 |
| 5 | Claude Sonnet 5.5Down 1 since last week | 1.04 |
| 6 | Claude Opus 5.5 | 1.11 |
| 7 | Grok 4.7Up 1 since last week | 1.61 |
| – | Claude Opus 5 | No longer ranked |
| – | GPT-5.6 Sol | No longer ranked |
| – | Claude Opus 4.8 | No longer ranked |
Adoption
Share of AI-attributed added linesJul 20 to Nov 8, 2026 · 16 complete weeks
Models
| Week | Claude Opus 5.5 | GPT-6.1 Sol | GPT-6 Astra | Claude Sonnet 5.5 | Claude Fable 5.1 | GPT-6 Luna | Grok 4.7 | Claude Opus 5 | GPT-5.6 Sol | Claude Opus 4.8 | Other |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Week of Jul 20, 2026 | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 5.0% | 33.8% | 26.4% | 34.8% |
| Week of Jul 27, 2026 | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 31.1% | 25.1% | 10.6% | 33.2% |
| Week of Aug 3, 2026 | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 46.4% | 22.7% | 6.0% | 24.9% |
| Week of Aug 10, 2026 | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 38.6% | 23.3% | 4.5% | 33.7% |
| Week of Aug 17, 2026 | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 26.7% | 35.4% | 1.8% | 36.1% |
| Week of Aug 24, 2026 | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% | 30.3% | 35.2% | 3.8% | 30.7% |
| Week of Aug 31, 2026 | 0.0% | 0.0% | 3.2% | 0.0% | 17.5% | 0.0% | 0.0% | 27.0% | 28.3% | 2.1% | 21.8% |
| Week of Sep 7, 2026 | 0.0% | 0.0% | 22.7% | 0.0% | 18.3% | 0.0% | 0.0% | 26.6% | 13.3% | 1.7% | 17.2% |
| Week of Sep 14, 2026 | 0.0% | 0.0% | 19.6% | 0.0% | 19.0% | 0.0% | 0.0% | 37.9% | 6.8% | 1.9% | 14.9% |
| Week of Sep 21, 2026 | 28.7% | 0.0% | 13.4% | 0.0% | 11.0% | 0.4% | 0.5% | 21.8% | 4.3% | 0.5% | 19.2% |
| Week of Sep 28, 2026 | 53.4% | 4.6% | 9.5% | 2.5% | 6.2% | 0.4% | 0.7% | 4.5% | 2.1% | 0.2% | 16.0% |
| Week of Oct 5, 2026 | 60.4% | 8.7% | 8.1% | 3.5% | 3.9% | 0.7% | 0.3% | 1.9% | 0.7% | 0.2% | 11.6% |
| Week of Oct 12, 2026 | 59.1% | 9.6% | 4.7% | 4.8% | 3.3% | 1.3% | 0.4% | 2.2% | 0.4% | 0.5% | 13.7% |
| Week of Oct 19, 2026 | 59.6% | 12.6% | 5.9% | 4.5% | 4.2% | 1.7% | 0.3% | 1.2% | 0.2% | 0.1% | 9.6% |
| Week of Oct 26, 2026 | 61.6% | 10.7% | 4.8% | 2.6% | 6.5% | 0.4% | 1.2% | 0.8% | 0.4% | 0.6% | 10.3% |
| Week of Nov 2, 2026 | 66.2% | 9.9% | 4.5% | 3.7% | 2.8% | 0.7% | 0.6% | 0.5% | 0.3% | 0.2% | 10.6% |
Week of Nov 2, 2026
- Claude Opus 5.5
- 66.2%
- GPT-6.1 Sol
- 9.9%
- GPT-6 Astra
- 4.5%
- Claude Sonnet 5.5
- 3.7%
- Claude Fable 5.1
- 2.8%
- GPT-6 Luna
- 0.7%
- Grok 4.7
- 0.6%
- Claude Opus 5
- 0.5%
- GPT-5.6 Sol
- 0.3%
- Claude Opus 4.8
- 0.2%
- Other
- 10.6%
Harnesses
| Week | Claude Code | Codex | Cursor | OpenCode | Pi | Other |
|---|---|---|---|---|---|---|
| Week of Jul 20, 2026 | 53.8% | 37.0% | 6.2% | 1.0% | 1.8% | 0.2% |
| Week of Jul 27, 2026 | 60.9% | 28.2% | 8.8% | 0.5% | 1.3% | 0.1% |
| Week of Aug 3, 2026 | 71.0% | 23.4% | 4.2% | 0.8% | 0.3% | 0.3% |
| Week of Aug 10, 2026 | 66.3% | 25.0% | 4.9% | 2.3% | 1.3% | 0.3% |
| Week of Aug 17, 2026 | 49.6% | 36.9% | 10.6% | 1.7% | 1.0% | 0.2% |
| Week of Aug 24, 2026 | 53.9% | 38.9% | 3.9% | 1.4% | 1.5% | 0.4% |
| Week of Aug 31, 2026 | 60.5% | 34.5% | 3.2% | 0.9% | 1.0% | 0.0% |
| Week of Sep 7, 2026 | 56.6% | 37.3% | 3.1% | 1.9% | 1.0% | 0.1% |
| Week of Sep 14, 2026 | 66.7% | 27.4% | 3.3% | 2.1% | 0.2% | 0.3% |
| Week of Sep 21, 2026 | 72.0% | 23.3% | 2.9% | 0.9% | 0.4% | 0.5% |
| Week of Sep 28, 2026 | 76.5% | 18.6% | 3.5% | 0.9% | 0.3% | 0.3% |
| Week of Oct 5, 2026 | 78.0% | 19.0% | 1.6% | 0.7% | 0.5% | 0.3% |
| Week of Oct 12, 2026 | 79.1% | 18.1% | 1.3% | 0.7% | 0.5% | 0.4% |
| Week of Oct 19, 2026 | 76.4% | 20.5% | 1.3% | 0.4% | 1.1% | 0.3% |
| Week of Oct 26, 2026 | 79.8% | 17.0% | 2.0% | 0.5% | 0.6% | 0.1% |
| Week of Nov 2, 2026 | 79.4% | 16.1% | 1.9% | 1.8% | 0.8% | 0.0% |
Week of Nov 2, 2026
- Claude Code
- 79.4%
- Codex
- 16.1%
- Cursor
- 1.9%
- OpenCode
- 1.8%
- Pi
- 0.8%
- Other
- 0.0%
Methodology
A counted bug is a finding on AI-written code that a developer upvoted or an AI coding agent addressed. Relative bug rate compares counted bugs per 1,000 attributed lines with the overall rate (1.00). That overall rate covers all AI-attributed lines in the window — including unranked and unspecified models — not an average of the models shown.
Adoption measures AI-attributed added lines in merged PRs reviewed by cubic. Small weekly cohorts are grouped into Other. Aggregates are anonymized. Bug Bench v0.1.
Bug Bench is observational: teams choose their models, so a rate reflects the code each model was used for as well as the model. A model is ranked once enough of its code has been reviewed across many organizations. JSON, CSV, and Markdown downloads are free to use under CC BY 4.0.
Questions and answers
- Which AI coding model writes the fewest bugs?
- As of Nov 9, 2026, GPT-6 Luna had the lowest relative bug rate on Bug Bench (0.31, 69% fewer counted bugs than average), among 7 ranked AI coding models.
- Which AI coding model has the most bugs?
- As of Nov 9, 2026, Grok 4.7 had the highest relative bug rate among 7 ranked models (1.61, 61% more counted bugs than average).
- How does Bug Bench measure bugs?
- It counts cubic review findings on AI-written code that a developer upvoted or an AI coding agent addressed, per line, relative to the average across all AI-written code. 1.00 is average; lower is better.
- Which AI coding agent writes the most code?
- Claude Code wrote 79.4% of AI-attributed lines in merged pull requests reviewed by cubic in the latest complete week.