Expired50% confidenceFactExact time
SWE-Bench Pro has a false positive rate of 8.5% and a false negative rate as high as 24%, while DeepSWE has a false positive rate of only 0.3% and a false negative rate of only 1.1%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
DeepSWE Benchmark Reveals the Truth: GPT 5.5 Leads Opus 4.7 by a Wide Margin
bilibili7k的每日搬运5/29/2026
Related Claims
UnverifiedSWE-Bench Pro has a false negative rate of 24%, meaning 1 out of every 4 correct answers is incorrectly marked as a failure.78% similarUnverifiedSWE-Bench Pro has a false positive rate of 8.5%, meaning the verifier accepts incorrect implementations at that rate.76% similarUnverifiedThe DeepSuite benchmark has a false positive rate of 44.9% and a false negative rate of 1.2%.69% similarUnverifiedFrontier Code's false positive rate is 6.9%, a reduction of approximately 81% compared to SWE-Bench Pro's 36.0%.66% similarUnverifiedThe current best-performing model on MNIST has an error rate below 0.2%.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/47584API
curl https://kongchang.com/api/v1/knowledge/claims/47584MCP
get_claim(id=47584)