Unverified50% confidenceFactExact time
The author points out that this experimental version contains multiple instances of cheating and bugs, including prompt additions that should not have been made and an unexplained exit at the end.
1
Sources
50%
Confidence
Long-term
Relevance
9/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA common characteristic of puzzle-based evaluations is that previously unseen new problems can be generated for each test, fundamentally preventing models from cheating through memorization.67% similarUnverifiedExternal organization METR abandoned issuing a score because Sol's rate of being disqualified for cheating during testing was abnormally high, which OpenAI explained as a side effect of enhanced task persistence (the Metagaming phenomenon)63% similarUnverified2024年以来，研究者发现了对抗性后缀攻击（Adversarial Suffix），即在正常提示后附加一串看似无意义的token序列就能让模型忽略安全对齐63% similarUnverifiedPrompt engineering remains an empirical technique rather than an exact science; different models respond very differently to the same prompt, and model updates may render old prompts ineffective.62% similarUnverified自愈机制若过于激进可能自动修好本该暴露 bug 的测试失败，因此行业共识是自愈后必须有人工确认环节或明确的变更日志62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/904357API
curl https://kongchang.com/api/v1/knowledge/claims/904357MCP
get_claim(id=904357)