Unverified50% confidenceTradeoffExact time
A reader trained on 20 million tokens further reduced overall LM loss but caused degradation on math tasks, so the author chose 15 million tokens as the balance point.
1
Sources
50%
Confidence
Long-term
Relevance
9/25/2026
First Seen
Sources
Related Entities
Related Claims
Unverified消融实验显示，若继续训练主干通用知识掉10%、数学掉17.8%；若两塔共享权重则通用任务掉26%，其余指标全线崩溃75% similarUnverified发布的checkpoint按块大小16训练，若将采样块大小改为64，HumanEval从76%跌至约20%，GSM8K从高位跌到2.2%67% similarUnverifiedAfter 15 training epochs on the demonstration dataset, the text classification model achieves an accuracy of 99%, with a total of 110,000 model parameters (16,000 for Attention, 33,000 for FFN, and the remainder for Embedding)65% similarUnverifiedThe power-law exponent between test loss and compute is approximately 0.05, meaning that every 10x increase in compute yields approximately a 12% improvement in model performance.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/949989API
curl https://kongchang.com/api/v1/knowledge/claims/949989MCP
get_claim(id=949989)