Unverified50% confidenceBenchmarkExact time
Even the most advanced current models such as GLM-5.2, Claude-Sonnet-4.6, and GPT achieve only just over half marks on this benchmark.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
UnverifiedState-of-the-art models such as GLM-5.2, Claude-Sonnet-4.6, and GPT-5 achieve only approximately 50–54% Pass@8 on the hard subset.79% similarUnverifiedAccording to hands-on evaluations, GLM-5.2's overall performance has reached the level of GPT-5.x and the Claude series flagship models.79% similarUnverifiedEarly GPT-4 achieved a resolution rate of less than 2% on SWE-bench, while recent top models have surpassed 50%.78% similarUnverifiedEarly GPT-3's HumanEval Pass@1 was below 10%, while top models such as GPT-4o and Claude 3.5 Sonnet have now consistently exceeded 85%.77% similarPartially VerifiedGPT-5.6 is a notable improvement over GPT-5.5, but has not yet reached the level of a "cross-generational" leap and has not resolved the model's original weaknesses.77% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/895643API
curl https://kongchang.com/api/v1/knowledge/claims/895643MCP
get_claim(id=895643)