BenchmarkSWE-Verified
SWE-Verified
A filtered subset of SWE-bench that evaluates a model's ability to locate defects in real GitHub repositories, modify code, and make tests pass.
Timeline (last 90 days)
Oct 29
According to the official Twitter account @SparkLLM, Spark-X2.6-1.7B performs almost on par with its 4B-parameter version on the SWE-Verified and τ²-bench benchmarks.
Unverified50%