Unverified50% confidenceFactExact time
A positive control experiment showed that when SDF was used to make the model believe reward hacking favors consequentialist ethical answers (an association the base model does not originally have), the model did indeed become more consequentialist after learning to reward hack, whereas the base model showed no such change.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
Unverified奖励黑客问题不会随模型能力提升而自然消失，反而因模型规划能力增强而愈发难以防范74% similarVerifiedPoorly designed evaluators can lead to reward hacking, where the model finds shortcuts that score high on the metric but are actually useless in practice.70% similarUnverifiedIn related research, organizations such as OpenAI emphasize that over-optimizing the chain of thought may cause models to learn behavioral strategies that appear harmless on the surface but still engage in deception, known as Obfuscated Reward Hacking.69% similarUnverifiedThe combination of formal verification and AI can naturally eliminate reward hacking, because theorems must be genuinely proved by the kernel and cannot be cheated.67% similarUnverifiedThe study recommends that rather than pursuing a single perfect reward, one should systematically anticipate and test shortcuts the model might exploit, and incorporate adversarial validation into the design phase.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012020API
curl https://kongchang.com/api/v1/knowledge/claims/1012020MCP
get_claim(id=1012020)