Unverified50% confidenceBenchmarkExact time
Microsoft research shows that the overall MMLU score for 4-bit LLaMA 2 changes by only one-third of a percentage point, yet 13% of individual question answers changed.
1
Sources
50%
Confidence
Long-term
Relevance
9/16/2026
First Seen
Sources
Related Entities
Related Claims
Unverified一项针对GPT-4的研究显示，对MMLU题目进行语义等价改写后，模型准确率在部分子集上下降超过10个百分点69% similarUnverifiedAn EMNLP study found that 4-bit models suffer an average retrieval loss of 23% at 128,000 tokens, with 70B LLaMA losing one-third of its score while QWEN of the same scale loses almost none.63% similarUnverifiedThe score gap among multiple frontier models on MMLU has narrowed to the order of one percentage point, driving the emergence of more challenging benchmarks such as MMLU-Pro and ARC-AGI.62% similarUnverifiedThe lower the quantization bit depth, the more likely the model is to show degraded performance on detailed reasoning, but for most tasks at quantization levels above 4-bit, the experiential difference is not obvious to ordinary users.61% similarVerifiedOn mainstream evaluations such as MMLU, HumanEval, and GSM8K, the score gaps among leading large models are often only a few percentage points.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/925954API
curl https://kongchang.com/api/v1/knowledge/claims/925954MCP
get_claim(id=925954)