Verified80% confidenceFactTime unknown
大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
7
Sources
80%
Confidence
Long-term
Relevance
5/31/2026
First Seen
Sources
AI全栈开发入门：从机器学习到大模型的知识体系梳理
bilibili李宏毅transformer-
Related Entities
Related Claims
Unverified现代多语言大模型在预训练阶段建立语言无关语义表示，通过海量多语言语料联合训练在高维向量空间中构建跨语言语义对齐80% similarVerifiedThe training objective of a Base LLM is to predict the next word, using a paradigm called Autoregressive Language Modeling79% similarUnverified2018年BERT预训练模型问世，通过在海量文本上预训练，模型能够捕捉词语在不同语境下的细微含义差异79% similarUnverified上下文学习最初在大语言模型中被观察到，其能力本质源于预训练数据的多样性79% similarUnverifiedLLM通过在海量文本语料上进行无监督预训练（预测下一个词）来习得语言能力79% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/4241API
curl https://kongchang.com/api/v1/knowledge/claims/4241MCP
get_claim(id=4241)