Unverified50% confidenceBenchmarkExact time
In terms of invalid intent rejection, the best model correctly rejected 191 of 195 invalid intents.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/6/2026
First Seen
Valid until: 2/4/2027
Sources
Related Entities
Related Claims
UnverifiedReal-world paraphrasing reduces the invalid intent rejection rate: among 16 configurations, 15 showed a drop in rejection rate for the best model.75% similarUnverifiedThe error criterion accepted 17 runs (13 of which were pool selections) with failure probabilities as low as 0.38 to 0.89 times the reference value; another 181 runs exhausted the budget.71% similarUnverifiedIn 11 of the 12 models, between 87.5% and 100% of capitulation cases stopped at the exact wrong number provided by the user, meaning capitulation typically involves switching sides entirely.69% similarUnverifiedWhen the open-source small model gemma2:2b was used as the checker, it accepted 8 of the 28 wrong answers (6 wrong party and 2 wrong action), while rejecting 10 of the 14 correct answers.67% similarVerifiedIn a 2024 research paper, Anthropic quantified the phenomenon of model over-refusal, finding that safety-trained models had false positive refusal rates as high as 15–30% in certain scenarios66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1141652API
curl https://kongchang.com/api/v1/knowledge/claims/1141652MCP
get_claim(id=1141652)