Unverified50% confidenceFactExact time
The schemagate authors note its limitations: only one model was tested, and the prompt wording was chosen on 100 BIRD questions; removing these two factors leaves the score essentially unchanged (90.08).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
UnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.66% similarUnverifiedThree independent Subagents performed adversarial re-verification of the retrospective diary, checking 120 verifiable claims (including 26 commit hashes and more than 40 cited line numbers), resulting in zero fabrications and nine corrections (all involving line number drift and unit conversion issues).65% similarUnverifiedThis test has limitations: only 14 groups, one 2-billion-parameter model, one run; three valid paraphrases were corrected before the run because they changed the legal meaning, and the corrections are in the same commit as the test set; the test set has not yet been run on larger models.65% similarUnverifiedIn memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.65% similarUnverifiedIn few-shot scenarios (e.g., approximately 20 samples per class), multi-class subdivision should be abandoned, all defects should be collapsed into a binary classification problem (defective/non-defective), and specific classification should be left to manual post-processing.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1051149API
curl https://kongchang.com/api/v1/knowledge/claims/1051149MCP
get_claim(id=1051149)