Expired60% confidenceFactTime unknown
Devin 2.0 resolved 13.86% of real-world programming problems in the SWE-bench benchmark, compared to only 1.96% for previous AI models.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
5/31/2026
First Seen
Valid until: 8/29/2026(expired)
Sources
Related Entities
Related Claims
VerifiedDevin resolved 13.86% of real GitHub Issues in the SWE-bench test, surpassing the performance of Claude 2.72% similarUnverifiedThe SWE-bench benchmark from Princeton University in 2023 showed that the most advanced AI programming tools at the time achieved a success rate of only about 13% on real GitHub issue fix tasks.70% similarUnverifiedIn testing, small models of approximately 1–2B parameters were unable to accurately locate and complete complex tool-calling tasks within lengthy system prompts.68% similarUnverifiedA Reddit developer used a genetic algorithm combined with a multilayer perceptron (MLP) to optimize the boarding problem, achieving a result 9.6% faster than the Steffen method in simulation.67% similarUnverified4 basic rules can reduce the AI programming error rate from 41% to 11%67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/3194API
curl https://kongchang.com/api/v1/knowledge/claims/3194MCP
get_claim(id=3194)