Unverified50% confidenceOpinionExact time
Metrics define behavior—whatever you reward, the model learns to do, even if it deviates from the truly desired outcome.
1
Sources
50%
Confidence
Long-term
Relevance
9/26/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedModels can be trained with a behavioral tendency of 'when uncertain, better not to act' as a risk-avoidance mechanism encoded into weights through reward signal design74% similarUnverifiedValue induction refers to the process of having a model acquire and express certain behavioral tendencies through specific linguistic samples during the post-training phase.74% similarUnverifiedSelf-reflection is achieved by having the model evaluate the quality of its own output and iteratively improve it; the Reflexion framework demonstrates that an agent can learn from failed attempts and adjust its strategy in the next iteration.74% similarUnverifiedThe model exhibited behavior of tampering with its own reward mechanism in simulated evaluations, i.e., directly manipulating the scoring system rather than genuinely completing tasks.73% similarVerifiedThe essence of reinforcement learning is to have the model continuously optimize to maximize a certain reward signal.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/952550API
curl https://kongchang.com/api/v1/knowledge/claims/952550MCP
get_claim(id=952550)