Unverified50% confidenceBenchmarkExact time
On the Spider dataset, schemagate tied with the same model (Claude Sonnet 4.5) at 93.0; the paper's best result on Spider came from DeepSeek V3.2-Reasoning at 95.9.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/17/2026
First Seen
Valid until: 10/31/2026
Sources
Related Entities
Related Claims
UnverifiedOn Spider, schemagate and the paper's best configuration both achieved an AC-F1 of 93.0, but schemagate's wrong-answer rate was 5.1%, slightly higher than the paper's 4.6%.68% similarUnverifiedIn schemagate's benchmark, on the Spider dataset's 1034 questions the pure mathematical ranking accuracy is 80.3%, and 96.5% after re-ranking; on the BIRD dataset's 1534 questions the pure mathematical ranking is 88.6%, and 96.7% after re-ranking.62% similarUnverifiedFor the schemagate benchmark's paired McNemar test, the p-value is about 1e-49 on Spider and about 1e-36 on BIRD.60% similarUnverifiedClaude 3.5 Sonnet在HumanEval基准测试上的得分超过90%，而2021年最初的Codex模型得分仅为28.8%59% similarUnverified在图片隐藏虫子识别测试中，Sonnet 5的Max模式花费约10分钟并识别结果完全正确，而Gemini顶配模型只找到4只，GPT-5.5 X-High未能检测出57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1051145API
curl https://kongchang.com/api/v1/knowledge/claims/1051145MCP
get_claim(id=1051145)