Verified75% confidenceFactTime unknown
Benchmark Contamination is a significant challenge in the current field of large model evaluation, where publicly available test questions may already be included in model training data, causing evaluation results to be distorted.
3
Sources
75%
Confidence
Long-term
Relevance
6/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAround 2023, multiple academic benchmarks including BIG-bench and HELM were pointed out to have test set contamination issues, meaning the models' training data may have already included evaluation questions, leading to inflated scores.78% similarUnverifiedThe root causes of benchmark failure lie in two issues: training set contamination and question bank saturation.76% similarUnverifiedData leakage falls into two categories—target leakage and train-test contamination—and leads to systematic overestimation of model performance.74% similarVerifiedBenchmark contamination refers to the phenomenon where test questions may have already appeared in training data, causing high scores to not truly represent generalization ability.74% similarUnverifiedBenchmark gaming is widespread, as models may be exposed to test-set-related data during training or fine-tuning, resulting in inflated scores but insufficient generalization ability.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/22964API
curl https://kongchang.com/api/v1/knowledge/claims/22964MCP
get_claim(id=22964)