Unverified50% confidenceOpinionExact time
In JetBrains' self-reported evaluations, the 12B MoE Mellum2.1 wins on single-shot code generation tasks (LiveCodeBench, HumanEval+) but loses to the dense 9B Qwen3.5-9B on multi-step agentic benchmarks (SWE-bench Pro, Terminal-Bench).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedJetBrains self-reported evaluation results: Mellum2.1 scores 47.0% on SWE-bench Verified, 28.0% on SWE-bench Pro, 17.4% on Terminal-Bench 2.1, 82.0% on LiveCodeBench v6, and 91.5% on HumanEval+.69% similarUnverifiedBenchmarked against Qwen3.8-27B, Agens Volundr 32B leads on LiveCodeBench v6 (+4.2), HumanEval (+4.3), AIME 2025 (+2.9), and MATH-500 (+1.6).64% similarUnverifiedIn the MoE architecture, the Mixtral 8x7B model has a total of 46.7B parameters, but only approximately 12.9B parameters (2 experts) are activated per token, making inference costs close to those of a 13B dense model while achieving performance comparable to the 70B level.59% similarUnverifiedThe performance of Qwen2.5-Coder and DeepSeek-Coder-V2 on code benchmarks such as HumanEval and MBPP is already very close to that of top closed-source models such as Claude 3.5 Sonnet.59% similarUnverified在HumanEval、MBPP、GSM8K等基准测试上，Gemma 7B的表现显著优于同参数量级的LLaMA 2 7B和Mistral 7B59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1016869API
curl https://kongchang.com/api/v1/knowledge/claims/1016869MCP
get_claim(id=1016869)