Unverified50% confidenceFactExact time
Comparing model outputs with reference answers using the difflib ratio, among the 44 passes the highest ratio was only 0.73, with most below 0.3, indicating that the model is not reciting answers.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedFor the 7B model, accuracy with the solver was 75.31% versus 72.02% for direct calculation, a paired improvement of +3.29 percentage points, with a 95% cluster interval of [-3.49, 10.38] crossing zero, making the advantage non-significant.72% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.70% similarUnverified仅用小模型策略的每正确答案成本最低($0.0004)但准确率仅33%，说明没有准确率底线的每正确答案成本指标具有误导性70% similarUnverified一个61分的模型和一个59分的模型在多数实际应用中的差异可能远小于分数暗示的程度70% similarUnverifiedIn actual testing, the draft acceptance rate was only approximately 41%, with the draft model outputting 1,055 tokens but only approximately 438 being accepted.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013059API
curl https://kongchang.com/api/v1/knowledge/claims/1013059MCP
get_claim(id=1013059)