SWE-Bench Verified
A subset of SWE-Bench consisting of 500 tasks manually selected for clear descriptions and unique answers, and the most frequently cited benchmark for software engineering capability.
Timeline (last 90 days)
Reflection AI self-reports that Beam scores 80.9 on SWE-Bench Verified, 80.1 on Terminal-Bench 2.1, and 97.8 on AIME 2026.
In 50 real issues from the SWE-Bench Verified benchmark, Saluki successfully resolved 30, while the original model resolved 33, retaining approximately 91% of real-world bug-fixing capability.
In the comparison table officially released by Qwen, Gemma 4 26b only fixed 17.4% of real GitHub issues on SWE-Bench Verified.
GLM 4.7 Flash is licensed under MIT and scored 59.2 on SWE-Bench Verified.
The SAO team selected three high-difficulty benchmarks—SWE-Bench Verified, BeyondAIME, and IMOAnswerBench—for validation
According to the SWE-Bench Verified benchmark, Qwen 2.5 Coder 32B achieves a coding ability score of 61.7
All Facts (8)
SWE-Bench Verified是经人工验证的500道高质量子集，SWE-Bench Lite是300道较简单的子集
85%UnverifiedSWE-Bench Verified was proposed by Princeton University in 2023 and contains a total of 500 tasks.
60%UnverifiedReflection AI self-reports that Beam scores 80.9 on SWE-Bench Verified, 80.1 on Terminal-Bench 2.1, and 97.8 on AIME 2026.
50%UnverifiedIn 50 real issues from the SWE-Bench Verified benchmark, Saluki successfully resolved 30, while the original model resolved 33, retaining approximately 91% of real-world bug-fixing capability.
50%UnverifiedIn the comparison table officially released by Qwen, Gemma 4 26b only fixed 17.4% of real GitHub issues on SWE-Bench Verified.
50%UnverifiedGLM 4.7 Flash is licensed under MIT and scored 59.2 on SWE-Bench Verified.
50%UnverifiedThe SAO team selected three high-difficulty benchmarks—SWE-Bench Verified, BeyondAIME, and IMOAnswerBench—for validation
50%UnverifiedAccording to the SWE-Bench Verified benchmark, Qwen 2.5 Coder 32B achieves a coding ability score of 61.7
50%