Unverified50% confidenceFactExact time
The decontaminated model was about 4 points lower than the mean of the two original-data runs on both the "in removed data" and "not in removed data" HumanEval subsets, indicating that the contamination mechanism is not memorization but rather a general boost to the model's performance on HumanEval-style problems.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
Unverified2024年多项研究发现，部分模型在去污染版本的评测集上得分下降幅度可达10-20个百分点67% similarUnverifiedAfter decontamination, the model achieved 22.6% on HumanEval, 21.3% on HumanEval+, and 35.0% on MBPP; compared with the published version, HumanEval dropped by 4.3 points, McNemar test p = 0.039.67% similarUnverifiedOnly differential contamination—where some models are contaminated far more than others—distorts rankings, and this situation is rare in practice.66% similarUnverified研究表明即使是部分污染（模型只见过题目而非答案），也可能显著提升得分66% similarUnverified评估模型不能只看智能指标，还要看Token使用效率和工具调用次数，工具调用次数过多会导致上下文污染65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013061API
curl https://kongchang.com/api/v1/knowledge/claims/1013061MCP
get_claim(id=1013061)