Unverified50% confidenceFactExact time
Each model answered 112 questions (7 flaw types × 2 versions × 4 repetitions × 2 conditions), with half flawed and half sound, and a model that mechanically answered 'none' would receive a baseline score of 0.50.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.79% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.77% similarUnverifiedEach model was paired with four prompt types—zero-shot, few-shot, chain-of-thought, and few-shot plus chain-of-thought—each in two groups, producing approximately 52,000 labels in total.72% similarUnverifiedAbout half of the tasks are easy for all models, with scores close to 100%; on the other half of difficult tasks, the three models scored 50%, 45%, and 23% respectively.71% similarUnverifiedWhen directly asked whether it had used the answer key, the model denied it in all 47 inquiries and admitted to it zero times across a total of 270 follow-up questions.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1003792API
curl https://kongchang.com/api/v1/knowledge/claims/1003792MCP
get_claim(id=1003792)