Unverified50% confidenceSolutionExact time
schemagate's fix includes three changes: raising max_tokens from 120 to 1024; adding a fake model in tests that refuses to answer when max_tokens is below 512; and having the benchmark record whether the model gave an answer on each question.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/16/2026
First Seen
Valid until: 1/14/2027
Sources
Related Entities
Related Claims
Unverifiedschemagate's reranker code calls provider.complete with max_tokens=120, causing the thinking process of Claude 5 series models to exhaust the entire token budget, and the model is truncated before outputting any ranking result.70% similarUnverifiedClaude Code's recovery limit for max_tokens errors is three attempts, with the constant named MAX_OUTPUT_TOKENS_RECOVERY_LIMIT = 3.65% similarUnverifiedThe same parameter name reasoning_effort behaves differently across models from different vendors: in gpt-5.4-mini the median output tokens rose from 18 at none to 254 at high (×14.1); gpt-oss-120b rose from refusal to 1,340 (×4.6); claude-haiku-5.5 rose from 150 to 298 (×2.0); gemini-3.7-flash rose from 453 to 1,013 (×2.2).64% similarUnverifiedIn schemagate's benchmark, on the Spider dataset's 1034 questions the pure mathematical ranking accuracy is 80.3%, and 96.5% after re-ranking; on the BIRD dataset's 1534 questions the pure mathematical ranking is 88.6%, and 96.7% after re-ranking.63% similarUnverifiedIn the task of completing the last Token of an FP8 quantization function, after DeepSeek-V4-Flash-Base adds equal-sign decorations before the code, when the number of equal signs modulo 4 is 0 or 1 the error rate is as high as 71.3%, and when the remainder is 2 or 3 the accuracy returns to 91.5%.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1049516API
curl https://kongchang.com/api/v1/knowledge/claims/1049516MCP
get_claim(id=1049516)