Updated
Which AI models write the most bugs, by type
The AI coding models with the most and the fewest bugs of each type.
- Logic bugsMost: Grok 4.7 · 1.45 · Fewest: GPT-6 Luna · 0.28
- Tests and documentationMost: Grok 4.7 · 1.62 (statistically tied with 1 other) · Fewest: GPT-6 Luna · 0.44
- Accessibility and user experienceMost: Grok 4.7 · 4.56 · Fewest: Claude Fable 5.1 · 0.68 (statistically tied with 2 others)
- Security and privacyMost: Claude Sonnet 5.5 · 1.51 (statistically tied with 1 other) · Fewest: GPT-6.1 Sol · 0.60 (statistically tied with 2 others)
- Error handlingMost: Claude Opus 5.5 · 1.10 (statistically tied with 1 other) · Fewest: GPT-6.1 Sol · 0.64 (statistically tied with 3 others)
- PerformanceMost: GPT-6 Astra · 1.26 (statistically tied with 3 others) · Fewest: Claude Fable 5.1 · 0.72 (statistically tied with 2 others)
- Business rules and complianceMost: Claude Opus 5.5 · 1.53 (statistically tied with 1 other) · Fewest: GPT-6.1 Sol · 0.60 (statistically tied with 3 others)
- Configuration and dependenciesMost: Claude Sonnet 5.5 · 1.49 (statistically tied with 1 other) · Fewest: Claude Fable 5.1 · 0.74 (statistically tied with 2 others)
- Type safety and build errorsMost: GPT-6.1 Sol · 2.94 · Fewest: GPT-6 Astra · 0.72 (statistically tied with 2 others)
- Code maintainabilityMost: GPT-6 Astra · 1.26 (statistically tied with 1 other) · Fewest: GPT-6.1 Sol · 0.69 (statistically tied with 2 others)
- Architecture and designNo clear difference (statistically tied)
- Code styleNo clear difference (statistically tied)