Unverified50% confidenceOpinionExact time
HumanEval has a limited sample size and has been extensively covered by training data, leading to suspicions of test set leakage in some models
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
Unverified预处理泄漏在小数据集上尤为显著，当训练集仅有数百张图像时加入测试集样本会明显改变均值和标准差估计78% similarUnverified在稠密检索中，批内负样本训练要求批次包含有信息量的负样本，随机采样会导致训练损失快速收敛但泛化性能差77% similarUnverifiedThere is an inherent tension in dangerous capability assessments between the coverage, update frequency, and confidentiality of test sets: overly public test sets become incorporated into training data and lose effectiveness, while overly confidential tests lack peer-review credibility.75% similarVerified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力74% similarUnverifiedResearchers can use n-gram overlap analysis and membership inference attacks to determine whether specific data appeared in a model's training set, but with the limitation of a high false positive rate.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/976192API
curl https://kongchang.com/api/v1/knowledge/claims/976192MCP
get_claim(id=976192)