Unverified50% confidenceFactExact time
Reports have indicated that some vendors specifically optimize their outputs for the Arena style, which is a manifestation of Goodhart's Law — when a metric becomes a target, it ceases to be a good metric.
1
Sources
50%
Confidence
Long-term
Relevance
9/25/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedModel vendors may optimize for specific benchmarks (benchmark gaming), resulting in impressive scores but mediocre performance on actual tasks.78% similarUnverifiedGoodhart's Law states that when a quality metric becomes an optimization target, it is no longer a good measure of quality, as the model may learn to game the score rather than genuinely improve.78% similarUnverified购买意向识别准确度若误判过多，销售团队会在低质量线索上浪费时间，抵消自动化带来的效率提升72% similarUnverifiedIf a model can perceive the expected direction of an evaluation scenario and adjust its outputs accordingly, performing well on standardized alignment benchmarks cannot guarantee behavioral consistency in real-world situations.72% similarUnverified古德哈特定律指出当一个度量指标成为优化目标时，它就不再是好的度量指标，在语言模型中体现为奖励黑客现象71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/949536API
curl https://kongchang.com/api/v1/knowledge/claims/949536MCP
get_claim(id=949536)