Unverified50% confidenceCautionExact time
Naïve confidence-stratified comparisons are unreliable because uncontrolled detection strength can confound conclusions.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThere is an inherent tension in dangerous capability assessments between the coverage, update frequency, and confidentiality of test sets: overly public test sets become incorporated into training data and lose effectiveness, while overly confidential tests lack peer-review credibility.78% similarUnverifiedOnce a public benchmark attracts widespread attention, it risks being targeted for optimization, and high scores do not necessarily reflect generalized reasoning ability.78% similarUnverifiedThe overconfidence of a model answering directly while skipping retrieval is technically referred to as miscalibration, meaning there is a systematic deviation between confidence and actual accuracy.76% similarUnverified天花板效应指当测试工具的难度上限低于被测对象的实际能力时，分数分布呈现严重负偏态，导致测试的信度和效度显著下降75% similarVerifiedDrift detection involves a trade-off: setting the threshold too sensitively leads to frequent false alarms and unnecessary retraining, while setting it too loosely may allow model performance to continuously degrade without being detected.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/975873API
curl https://kongchang.com/api/v1/knowledge/claims/975873MCP
get_claim(id=975873)