Unverified50% confidenceOpinionExact time
Even with customized prompts, the strongest models' evidence retention on KEEP-Bench remains unreliable.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/20/2026
First Seen
Valid until: 1/18/2027
Sources
Related Entities
Related Claims
UnverifiedOn KEEP-Bench, the best closed-source model retained all required evidence in only 61.6% of cases.78% similarUnverifiedOn long traces, the model tends to prematurely lock onto a seemingly plausible diagnosis and stop, resulting in key evidence going unexamined.70% similarUnverifiedSome failure cases can be traced to the model continuing to reason and exhausting the budget that should have been used to give the final answer, ultimately failing to converge to a conclusion.69% similarUnverifiedNaïve confidence-stratified comparisons are unreliable because uncontrolled detection strength can confound conclusions.68% similarUnverifiedKEEP-Bench contains 4,102 cases covering three challenging scenarios: retaining evidence to support plausible referential resolution, explicit qualifiers, and minority viewpoints.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1073302API
curl https://kongchang.com/api/v1/knowledge/claims/1073302MCP
get_claim(id=1073302)