Expired50% confidenceFactExact time
The training data volume of current mainstream large models has reached tens of trillions, or even sixty trillion, tokens.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Related Claims
UnverifiedIn the training of hundred-billion-parameter large models, inter-node communication overhead can account for 30%–50% of total training time.72% similarUnverifiedLLaMA 1 was trained on 1.4T tokens of data, and LLaMA 2 was trained on 2T tokens of data.71% similarUnverified主流大模型的训练数据通常有6-18个月的滞后期69% similarUnverifiedMid-training (continued pre-training) is a training stage between pre-training and fine-tuning, with data volumes typically reaching hundreds of billions to trillions of tokens.69% similarVerifiedIn 2022, DeepMind published the Chinchilla paper, which corrected the optimal training ratio and indicated that, under a given compute budget, the number of model parameters and the number of training tokens should maintain an approximate ratio of 1:2069% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/53866API
curl https://kongchang.com/api/v1/knowledge/claims/53866MCP
get_claim(id=53866)