Unverified50% confidenceFactExact time
Reward hacking refers to the phenomenon in policy optimization where samples are generated that score highly in the eyes of the surrogate model but whose true quality does not improve or even degrades; in RLHF it manifests as model outputs catering to the preference model rather than to true human preferences.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
Unverified"Optimizing against the grader" (reward hacking or specification gaming) is a common failure mode in which, when the optimization target is merely a proxy for the true goal, the model or contestant may find ways to score high on the grader without satisfying the true intent.78% similarUnverifiedA positive control experiment showed that when SDF was used to make the model believe reward hacking favors consequentialist ethical answers (an association the base model does not originally have), the model did indeed become more consequentialist after learning to reward hack, whereas the base model showed no such change.78% similarUnverified模型优化测量指标本身而非其代表的真实能力的现象在AI安全领域被称为规格博弈（Specification Gaming）或奖励黑客（Reward Hacking）77% similarUnverified奖励黑客问题不会随模型能力提升而自然消失，反而因模型规划能力增强而愈发难以防范76% similarVerified奖励函数设计不当会导致奖励欺骗（Reward Hacking）现象，即智能体找到在奖励函数上得分高但实际无意义甚至有害的行为路径76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1069685API
curl https://kongchang.com/api/v1/knowledge/claims/1069685MCP
get_claim(id=1069685)