Unverified50% confidenceFactExact time
The author noted that some benchmark tasks use synthetic data, that the benchmarks are exploratory in nature, and that they do not constitute proof of universal superiority over direct training methods.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe questions in the proprietary evaluation are not publicly disclosed, and models cannot improve their scores by memorizing questions from training data, making it a better reflection of true generalization ability.75% similarVerifiedGoodhart's Law states that when a metric becomes a target in itself it is no longer a good metric, and introducing test sets into training data causes a model's exam scores to decouple from its true capabilities74% similarUnverifiedThe sample size and task coverage of personal experiments designed by a single developer are limited, making the conclusions difficult to generalize directly, and they are better suited as a methodological reference rather than a definitive conclusion.74% similarUnverified链式思维、少样本提示、自我一致性等提示词工程技巧无法让模型访问训练截止日期之后的信息，无法保证浮点运算的精确性，也无法处理模型从未见过的外部文件74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/991519API
curl https://kongchang.com/api/v1/knowledge/claims/991519MCP
get_claim(id=991519)