Expired50% confidenceFactExact time
SWE-Bench Pro has a false positive rate of 8.5%, meaning the verifier accepts incorrect implementations at that rate.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
DeepSWE Benchmark Reveals the Truth: GPT 5.5 Leads Opus 4.7 by a Wide Margin
bilibili7k的每日搬运5/29/2026
Related Claims
UnverifiedSWE-Bench Pro has a false positive rate of 8.5% and a false negative rate as high as 24%, while DeepSWE has a false positive rate of only 0.3% and a false negative rate of only 1.1%.76% similarUnverifiedSWE-Bench Pro has a false negative rate of 24%, meaning 1 out of every 4 correct answers is incorrectly marked as a failure.75% similarUnverifiedFrontier Code's false positive rate is 6.9%, a reduction of approximately 81% compared to SWE-Bench Pro's 36.0%.64% similarUnverified接受率随位置逐级衰减：GSM8K从位置0的0.92下滑至约0.53，MTBench从0.78急跌至0.13，因误差在连续猜测中累积62% similarUnverifiedCompared to GPT-5.2, GPT-5.4 reduces the probability of an individual statement being incorrect by 33% and the probability of an entire response containing any error by 18%.58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/46947API
curl https://kongchang.com/api/v1/knowledge/claims/46947MCP
get_claim(id=46947)