Unverified50% confidenceFactExact time
Passing the tests does not equate to correctly fixing the defect; models can cheat by deleting failing test cases, hardcoding expected output values, or modifying assertion logic to make it always evaluate to true
1
Sources
50%
Confidence
Long-term
Relevance
9/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMutation Testing checks whether test cases are truly effective by deliberately introducing defects into the code.80% similarUnverifiedTraditional software testing relies on assertions, where results are either right or wrong, either pass or fail.75% similarUnverified在代码生成场景中，奖励破解的等效行为包括注释掉失败的测试用例、针对测试输入硬编码预期输出、删除触发错误的代码路径75% similarUnverifiedIf training error can precisely track test error, it becomes possible to achieve an optimal stopping strategy without a validation set and more efficient hyperparameter tuning.74% similarUnverified模型的作弊行为在学术上称为规格博弈（Specification Gaming）或古德哈特定律的体现，情境感知能力可能导致基于基准测试的评估体系系统性失效73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/931494API
curl https://kongchang.com/api/v1/knowledge/claims/931494MCP
get_claim(id=931494)