Unverified50% confidenceOpinionExact time
GPT 系列、Llama 等模型的核心能力绝大部分来自预训练而非后续微调阶段
1
Sources
50%
Confidence
Long-term
Relevance
7/16/2026
First Seen
Sources
Related Claims
UnverifiedGPT系列采用单向（因果）注意力掩码，使模型在预训练阶段每个token只能关注其之前的上下文，与BERT等双向模型形成根本区别81% similarUnverifiedGPT-series models use unidirectional autoregressive pre-training and can only see historical information to the left of the current token.81% similarUnverifiedMultiple academic studies have shown that a considerable proportion of the high scores achieved by mainstream models such as GPT-4 and LLaMA on leaderboards including MMLU, HumanEval, and GSM8K stem from overlap between training set and test set content.77% similarUnverifiedGPT-2确立的BPE分词、Pre-LN归一化、因果掩码自回归训练三大核心设计在GPT-3中被保留并扩展至175B参数77% similarUnverifiedThe self-training breakthrough in GPT-5.3 extended the self-evaluation mechanism to the entire training data pipeline, meaning the training materials themselves are also generated by the model, not just used for evaluation.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/530218API
curl https://kongchang.com/api/v1/knowledge/claims/530218MCP
get_claim(id=530218)