Updated

Claude Sonnet 5.5 vs GPT-6.1 Sol

GPT-6.1 Sol had 29% fewer counted bugs per line than Claude Sonnet 5.5 (0.73 vs 1.03).

Claude Sonnet 5.5 and GPT-6.1 Sol compared on relative bug rate, by bug type and language, and on price and usage.
Claude Sonnet 5.5GPT-6.1 Sol
Bug rate
Relative bug rate1.030.73
Rank#5 of 7#2 of 7
By bug type
Security and privacy1.510.60
Logic bugs0.850.61
Error handling(statistically tied)0.950.64
Performance(statistically tied)1.050.98
Business rules and compliance(statistically tied)1.190.60
Configuration and dependencies1.490.88
Accessibility and user experience(statistically tied)1.310.83
Tests and documentation1.240.93
By language
TypeScript(statistically tied)0.940.99
JavaScript2.430.52
Python1.280.53
Go0.231.95
Swift1.330.05
SQL(statistically tied)0.120.34
Shell(statistically tied)2.051.55
Details
ProviderAnthropicOpenAI
ReleasedSep 28, 2026Sep 29, 2026
List price$2 input / $10 output per 1M tokens$2 input / $10 output per 1M tokens
Share of AI-written code, week of Nov 23.7%10.0%

1.00 = average · lower is better · = statistically tied

More comparisons

Questions and answers

Which has fewer bugs, Claude Sonnet 5.5 or GPT-6.1 Sol?
GPT-6.1 Sol had 29% fewer counted bugs per line than Claude Sonnet 5.5 (0.73 vs 1.03). GPT-6.1 Sol ranks #2 of 7 and Claude Sonnet 5.5 #5.
Which is cheaper, Claude Sonnet 5.5 or GPT-6.1 Sol?
Both list at $2 input / $10 output per 1M tokens.
Does it depend on the programming language?
In TypeScript, they were statistically tied (0.94 and 0.99); in JavaScript, GPT-6.1 Sol had the lower rate (0.52 vs 2.43); in Python, GPT-6.1 Sol had the lower rate (0.53 vs 1.28); in Go, Claude Sonnet 5.5 had the lower rate (0.23 vs 1.95); in Swift, GPT-6.1 Sol had the lower rate (0.05 vs 1.33); in SQL, they were statistically tied (0.12 and 0.34); in shell scripts, they were statistically tied (2.05 and 1.55).
Does it depend on the kind of bug?
For security and privacy bugs, GPT-6.1 Sol had the lower rate (0.60 vs 1.51); for logic bugs, GPT-6.1 Sol had the lower rate (0.61 vs 0.85); for error-handling bugs, they were statistically tied (0.95 and 0.64); for performance bugs, they were statistically tied (1.05 and 0.98); for business-logic bugs, they were statistically tied (1.19 and 0.60); for config and dependency issues, GPT-6.1 Sol had the lower rate (0.88 vs 1.49); for accessibility and UX issues, they were statistically tied (1.31 and 0.83); for testing and documentation issues, GPT-6.1 Sol had the lower rate (0.93 vs 1.24).