Unverified50% confidenceBenchmarkExact time
In a test of 119 pairs of "near-identical questions", at the 0.81 threshold 62% of modified questions incorrectly hit; even raised to 0.90, 25% still hit; the hit rate for genuine synonymous rewrites dropped from 39% to 4%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/19/2026
First Seen
Valid until: 1/17/2027
Sources
Related Entities
Related Claims
UnverifiedCode replay inherits errors from the extraction stage; in the cluster sufficiency test, gpt-oss-120b's pass rate actually dropped from 56% to 40%.76% similarUnverifiedAt the 0.81 threshold, the proportion of cached answers that "fit the question" is 5.8 to 10.7 percentage points lower than fresh answers; converted to all responses, the overall quality loss is about 2 to 3 percentage points; Haiku 5.5 has the largest gap (10.7 percentage points), with a cached-answer pass rate of only 67.9%.76% similarUnverifiedIn 11 of the 12 models, between 87.5% and 100% of capitulation cases stopped at the exact wrong number provided by the user, meaning capitulation typically involves switching sides entirely.76% similarUnverifiedUnder the held-out evaluator, ASC's median error rate drops by 38%, the median improvement in supported sentences reaches 46%, and this holds in all test settings.76% similarUnverifiedIn a standalone test of the "view incomplete" prompt on 100 questions, the wrong-answer rate was 16% without the prompt and dropped to 5% with it.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065411API
curl https://kongchang.com/api/v1/knowledge/claims/1065411MCP
get_claim(id=1065411)