Expired50% confidenceFactExact time
The Pass@1 rate of frontier models such as GPT-4 and Claude on code generation tasks is typically between 60% and 80%, meaning 1 to 2 out of every 5 modifications may have issues.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Four Practical Claude Code Commands to Double Your AI Programming Efficiency
bilibiliopenclaw养成记6/3/2026
Related Claims
UnverifiedGPT-4 and Claude 3 achieve pass@1 rates exceeding 85% on the HumanEval code benchmark.77% similarPartially VerifiedAll three GPT-5.6 models achieved a perfect score of 10 on the long-horizon agent fine-tuning task, while GPT-5.5 and Opus 4.7 could only score two or three points on that task.73% similarUnverifiedFor the most difficult coding tasks, GPT 5.5 and Opus 4.8 currently each account for approximately 50% of usage, whereas a few months ago Anthropic's models held a clear advantage.73% similarUnverifiedTaking GPT-4 as an example, a single complex Agent task may involve 5–20 LLM calls.72% similarUnverifiedStanford University's AgentBench benchmark test in 2024 showed that GPT-4-level models have an end-to-end success rate below 30% on complex tasks requiring cross-system coordination.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/52890API
curl https://kongchang.com/api/v1/knowledge/claims/52890MCP
get_claim(id=52890)