Unverified80% confidenceFactTime unknown
2006年Google翻译初始系统使用了一个在数万亿词文本上训练的5-gram语言模型，这很可能是世界上第一次万亿token规模的语言模型训练
1
Sources
80%
Confidence
Long-term
Relevance
6/1/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedIn 2022, the Google Brain team first systematically documented the phenomenon of 'emergent abilities' in large language models in the paper 'Emergent Abilities of Large Language Models'76% similarVerified大语言模型通过在数千亿乃至数万亿个词元（Token）的文本数据上进行自监督预训练76% similarUnverified现代多语言模型如mT5、BLOOM和Llama系列在数百种语言的文本上联合训练，通过共享词汇表和跨语言注意力机制构建跨语言语义地图70% similarVerifiedThe context windows of current mainstream large language models range from tens of thousands to hundreds of thousands of tokens.67% similarUnverifiedBERT（2018年，Google）采用掩码语言模型（MLM）实现双向上下文理解，但无法生成连贯文本66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/22940API
curl https://kongchang.com/api/v1/knowledge/claims/22940MCP
get_claim(id=22940)