Verified80% confidenceFactExact time
Benchmark data contamination refers to the phenomenon where, after a test set is made public, its questions may appear in the pre-training corpora of subsequent models, causing scores to reflect memorization rather than genuine reasoning ability.
11
Sources
80%
Confidence
Long-term
Relevance
7/8/2026
First Seen
Sources
基准测试即数据激活：让领域数据找到攀登方向
hackernewshackernews7/3/2026
Related Claims
Verified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力77% similarVerifiedBenchmark contamination refers to the phenomenon where test questions may have already appeared in training data, causing high scores to not truly represent generalization ability.77% similarUnverifiedOut-of-sample design can substantially eliminate the interference of memorization retrieval on evaluation by using contemporary works created after the model's training cutoff date or from controlled sources.75% similarUnverifiedIf the model memorized the answers to contaminated benchmarks during pre-training, information loss from compressing the context will not affect performance, rendering the reported 'lossless compression' conclusion meaningless.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/223327API
curl https://kongchang.com/api/v1/knowledge/claims/223327MCP
get_claim(id=223327)