Unverified50% confidenceFactExact time
The pretraining of MNN-Pocket-0.3B covers approximately 4.1 trillion high-quality Chinese and English tokens.
1
Sources
50%
Confidence
Long-term
Relevance
11/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe post-training of MNN-Pocket-0.3B consists of three stages: instruction fine-tuning, multi-teacher on-policy distillation, and quantization-aware training (QAT).64% similarUnverifiedA systematic empirical study used nearly 200B tokens for pretraining on decoder-only LLMs ranging from 360M to 3.1B parameters, examining fourteen orthogonal training variants.64% similarUnverified开源模型LLaMA 2使用了2万亿token训练数据63% similarUnverifiedMusk himself spoiled that Grok 4.8 has a total of 2.5 trillion parameters, is developed on a brand-new training architecture, and that pre-training has now concluded smoothly while the team is working full-speed on reinforcement learning tuning.63% similarUnverifiedSmoLLM基础模型训练使用了来自FineWeb-Edu数据集的约25亿token62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1154764API
curl https://kongchang.com/api/v1/knowledge/claims/1154764MCP
get_claim(id=1154764)