SWE-Bench Pro
SWE-Bench Pro is a benchmark for evaluating the software engineering capabilities of AI models and represents an advanced version of the SWE-Bench series. By providing real-world code repository issues and bug-fixing tasks, it assesses AI systems' practical engineering abilities in code comprehension, issue localization, and patch generation, aiming to measure more rigorously and comprehensively how well AI models resolve real-world software development tasks.
Core Facts
Qwen3.6 27B scored 77.2 on SWE-bench Verified.
Qwen3.8-Flash-Next achieved scores of 58.7 and 62.5 on the SWE-bench 1.1 and SWE-bench Pro evaluations, respectively.
SWE-Bench was released by a Princeton University research team and collects real issue reports and their corresponding pull request fix patches from GitHub.
Timeline (last 90 days)
On Qwen's model card, Qwen3.8 27b scored 61.7 on SWE-Bench Pro, 73 on Terminal Bench 2.1, and 90.3 on Live Code Bench.
The UP host stated that the Qwen3 coding model scored approximately 61.7 on SWE-Bench Pro, while the top-tier closed-source model used as a reference scored 53.4 (this data contains inconsistencies and is for reference only).
Qwen3.8 Flash Next scores 62.5 on SWE-Bench Pro, 73.9 on Cowork Bench, 84.5 on Android World, and 91.9/95.7 on Live Code Bench.
Alibaba's official figures show Qwen3.8 Flash scoring 62.5 on SWE-Bench Pro, leading Claude Opus 4.6 by approximately 9 points; 73.9 on Cowork Bench versus DeepSeek's 45.1; and 55.7 on Job Bench, exceeding Opus 4.6 by approximately 20 points.
Qwen3.8-Flash-Next achieved scores of 58.7 and 62.5 on the SWE-bench 1.1 and SWE-bench Pro evaluations, respectively.
Muse Glimmer在DeepSearch QA得分74.6，SWE-Bench Pro得分51.2
All Facts (7)
Qwen3.6 27B scored 77.2 on SWE-bench Verified.
70%VerifiedQwen3.8-Flash-Next achieved scores of 58.7 and 62.5 on the SWE-bench 1.1 and SWE-bench Pro evaluations, respectively.
65%VerifiedSWE-Bench was released by a Princeton University research team and collects real issue reports and their corresponding pull request fix patches from GitHub.
65%UnverifiedOn Qwen's model card, Qwen3.8 27b scored 61.7 on SWE-Bench Pro, 73 on Terminal Bench 2.1, and 90.3 on Live Code Bench.
50%UnverifiedQwen3.8 Flash Next scores 62.5 on SWE-Bench Pro, 73.9 on Cowork Bench, 84.5 on Android World, and 91.9/95.7 on Live Code Bench.
50%UnverifiedAlibaba's official figures show Qwen3.8 Flash scoring 62.5 on SWE-Bench Pro, leading Claude Opus 4.6 by approximately 9 points; 73.9 on Cowork Bench versus DeepSeek's 45.1; and 55.7 on Job Bench, exceeding Opus 4.6 by approximately 20 points.
50%UnverifiedMuse Glimmer在DeepSearch QA得分74.6，SWE-Bench Pro得分51.2
50%