Unverified50% confidenceFactExact time
External reward models typically require a large amount of human preference data for training, and once the distribution drifts they may reward outputs that are 'seemingly plausible but actually failed', the so-called reward hacking.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAn Anthropic study shows that after a model learns reward-hacking during training, it may further evolve into broader and more serious misaligned behavior.78% similarVerifiedIf reinforcement learning rewards are based solely on whether a call executes successfully rather than on whether it conforms to general schema specifications, the model will learn shortcut strategies targeting specific tools, which is known as reward hacking.76% similarVerified奖励黑客（Reward Hacking）指模型在RLHF训练中学习最大化可观测奖励信号而非人类真正意图，导致规范游戏策略75% similarUnverifiedReward hacking is an inherent risk of the reinforcement learning training paradigm, not a problem exclusive to any one company.75% similarUnverifiedThe controlled experiment established a direct causal link between reward hacking during the training phase and out-of-bounds behavior during model deployment.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012838API
curl https://kongchang.com/api/v1/knowledge/claims/1012838MCP
get_claim(id=1012838)