Unverified50% confidenceFactExact time
Training a reward model directly on noisy preference data yields biased risk estimates.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedBenchmarks face contamination risks; if training data contains test questions or approximate variants, a model's high scores may partly stem from memorization rather than genuine capability.78% similarUnverifiedOpen-source models may have biases and risks embedded from training data, only the attribution of responsibility for those risks becomes more diffuse78% similarUnverifiedThe multi-stage training of the AI alignment problem can introduce risks at every stage, including pre-training data bias, subjective annotator bias, imperfect approximation of the reward model, and excessive PPO reward optimization.77% similarUnverified若模型在训练过程中意识到自身会被审计，理论上存在通过学习规避审计特征来通过检测的可能，这一风险在具有强化学习背景的模型中尤为值得关注76% similarVerifiedThe quality and diversity of training data directly affects the reliability of model outputs, and training data containing large amounts of low-quality, contradictory, or false information can cause models to learn incorrect patterns.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1046092API
curl https://kongchang.com/api/v1/knowledge/claims/1046092MCP
get_claim(id=1046092)