Unverified50% confidenceDecision RuleExact time
Held-out testing requires that the test data be completely isolated from the search process, otherwise information leakage occurs and leads to overestimating the method's capability.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
Verified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力77% similarUnverifiedThere is an inherent tension in dangerous capability assessments between the coverage, update frequency, and confidentiality of test sets: overly public test sets become incorporated into training data and lose effectiveness, while overly confidential tests lack peer-review credibility.75% similarUnverifiedData leakage falls into two categories—target leakage and train-test contamination—and leads to systematic overestimation of model performance.73% similarVerifiedDrift detection involves a trade-off: setting the threshold too sensitively leads to frequent false alarms and unnecessary retraining, while setting it too loosely may allow model performance to continuously degrade without being detected.73% similarUnverifiedIn a benchmark with no truncation at all, correct default values instead exhibited the largest derivation gap, ruling out the explanation that truncation caused the artifact.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1039015API
curl https://kongchang.com/api/v1/knowledge/claims/1039015MCP
get_claim(id=1039015)