Unverified50% confidenceFactExact time
There is a distribution shift between general benchmarks such as MMLU, HumanEval, and MT-Bench and specific tasks.
1
Sources
50%
Confidence
Long-term
Relevance
10/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAs benchmarks such as HumanEval and MMLU have been widely adopted, targeted optimization has emerged, causing leaderboard scores to diverge from actual capabilities.74% similarUnverifiedThe positive correlation between base task performance and generalization ability weakens under significant distribution shift.66% similarUnverifiedWhen a model improves substantially on certain tasks while remaining flat on others, the composite index averages out the changes and fails to reflect real task-level capability differences.65% similarUnverified通用评测分数接近的大模型在特定垂直任务上可能表现差异巨大64% similarUnverified对齐问题涉及目标规范、内部对齐和分布偏移等多个技术层面63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/981206API
curl https://kongchang.com/api/v1/knowledge/claims/981206MCP
get_claim(id=981206)