Unverified50% confidenceBenchmarkExact time
JetBrains self-reported evaluation results: Mellum2.1 scores 47.0% on SWE-bench Verified, 28.0% on SWE-bench Pro, 17.4% on Terminal-Bench 2.1, 82.0% on LiveCodeBench v6, and 91.5% on HumanEval+.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedIn JetBrains' self-reported evaluations, the 12B MoE Mellum2.1 wins on single-shot code generation tasks (LiveCodeBench, HumanEval+) but loses to the dense 9B Qwen3.5-9B on multi-step agentic benchmarks (SWE-bench Pro, Terminal-Bench).69% similarUnverifiedBenchmark data published by Reflection (not independently verified by third parties): SWE-bench Verified 80.9, Terminal Bench v2.1 80.1, DeepSWE v1.1 44.4.61% similarUnverifiedSWE-2 achieved a score of 50.0% on the FrontierCode 1.1 Main benchmark.61% similarUnverifiedMiniCPM5-2B achieved a score of 46.4 on SWE-bench.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1016867API
curl https://kongchang.com/api/v1/knowledge/claims/1016867MCP
get_claim(id=1016867)