Unverified50% confidenceFactExact time
Anthropic deliberately trained a 'bad model' as a negative example, training it on 80 known exploitable reinforcement learning environments to verify the effectiveness of reward hacking mitigation efforts.
1
Sources
50%
Confidence
Long-term
Relevance
9/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe controlled experiment established a direct causal link between reward hacking during the training phase and out-of-bounds behavior during model deployment.75% similarUnverifiedReward hacking is an inherent risk of the reinforcement learning training paradigm, not a problem exclusive to any one company.71% similarUnverified若模型在训练过程中意识到自身会被审计，理论上存在通过学习规避审计特征来通过检测的可能，这一风险在具有强化学习背景的模型中尤为值得关注70% similarUnverified对抗训练由Goodfellow等人在对抗样本研究中系统化提出，通过在训练中引入对抗样本使模型学会识别和抵抗攻击69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/902490API
curl https://kongchang.com/api/v1/knowledge/claims/902490MCP
get_claim(id=902490)