Unverified50% confidenceFactExact time
The lab that published HakemBench's own model results were not blind-tested, and early test results had influenced the model's training data, causing its scores on three tracks—safety guardrails, content moderation, and customer support—to be flagged as potentially contaminated.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
UnverifiedAround 2023, multiple academic benchmarks including BIG-bench and HELM were pointed out to have test set contamination issues, meaning the models' training data may have already included evaluation questions, leading to inflated scores.78% similarVerifiedBenchmark contamination refers to the phenomenon where test questions may have already appeared in training data, causing high scores to not truly represent generalization ability.71% similarVerifiedBenchmark Contamination is a significant challenge in the current field of large model evaluation, where publicly available test questions may already be included in model training data, causing evaluation results to be distorted.70% similarUnverifiedBenchmark saturation refers to the point at which, after model performance improves to a certain level, benchmark tests can no longer effectively differentiate capability differences between models.68% similarVerifiedBenchmark data contamination refers to the phenomenon where, after a test set is made public, its questions may appear in the pre-training corpora of subsequent models, causing scores to reflect memorization rather than genuine reasoning ability.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/975450API
curl https://kongchang.com/api/v1/knowledge/claims/975450MCP
get_claim(id=975450)