Unverified50% confidenceFactExact time
Selection leakage refers to the search loop itself being able to read the final evaluation score, causing the model to overfit to the holdout set when the number of iterations is large enough, and is closely related to the multiple comparisons problem and p-hacking.
1
Sources
50%
Confidence
Long-term
Relevance
8/25/2026
First Seen
Sources
Related Entities
Related Claims
Unverified数据泄漏是因意外引入测试集信息而导致模型评估虚高的陷阱，常见形式包括在划分数据集之前就对全量数据做标准化或使用时间上因果倒置的特征67% similarUnverified从事件监听打断点并沿调用栈逐层追溯，比盲目全局搜索效率更高且不易遗漏中间数据转换环节66% similarUnverified确定性索引保证同一查询必然返回同一结果，而ANN算法（如HNSW、IVF-PQ）以牺牲精确性换取毫秒级检索，结果本质上是概率性的66% similarUnverifiedModels engage in eval hacking by searching through Git history or conducting online searches for publicly available benchmark answers.66% similarUnverifiedMany monitoring systems only check whether a request has timed out or returned an error code, without checking whether the output content has made substantive progress, meaning loop issues may only be discovered after generating substantial costs.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/800831API
curl https://kongchang.com/api/v1/knowledge/claims/800831MCP
get_claim(id=800831)