Unverified50% confidenceBenchmarkExact time
After removing the scope check, schemagate's wrong-answer rate on BIRD rose from 5.4% to 11.3%.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/17/2026
First Seen
Valid until: 10/31/2026
Sources
Related Entities
Related Claims
UnverifiedOn BIRD, the proportion of questions schemagate should have refused but answered was 5.4%, lower than the 7.4% of the paper's best configuration.77% similarUnverifiedGPT-6.1 Sol's rate of "not honestly reporting when a tool fails" dropped from 4.9% to 2.1%, while Astra's rate is 1.5%.62% similarUnverifiedIn document deskewing, even a tilt of 3–5 degrees can cause recognition rates to drop by more than 10%.61% similarUnverifiedIn the BIRD benchmark, Aura's token-substitution approach achieved an accuracy of 67.6%, only 0.6 percentage points below the 68.2% achieved with real data, while the number of leaked personal information items dropped from 849 to 0.61% similarUnverified接受率随位置逐级衰减：GSM8K从位置0的0.92下滑至约0.53，MTBench从0.78急跌至0.13，因误差在连续猜测中累积60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1050461API
curl https://kongchang.com/api/v1/knowledge/claims/1050461MCP
get_claim(id=1050461)