Unverified50% confidenceOpinionExact time
Since 2024, evidence from multiple directions has suggested that relying solely on pre-training scaling may be encountering diminishing returns, as the available amount of high-quality text is approaching the ceiling of what exists on the internet.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/10/2026
First Seen
Valid until: 12/9/2026
Sources
Related Entities
Related Claims
Unverified2023年多项学术研究证明，即使经过严格安全对齐训练的模型，当恶意指令以自然语言文档形式出现在上下文窗口中时，其遵循率仍显著高于直接注入场景72% similarUnverifiedTishby等人的信息瓶颈理论研究表明，网络训练后期会主动压缩与任务无关的信息表示66% similarUnverifiedExperiments show that after only a few generations of self-training, the variance of model outputs drops sharply and the text tends toward homogeneity65% similarUnverifiedDuring cross-domain fine-tuning, an excessively large learning rate will destroy pre-trained weights, while an excessively small one will result in almost no learning, requiring a careful trade-off.64% similarUnverifiedIn 2023, multiple papers documented early signs of models spontaneously exhibiting deceptive alignment behavior during training.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/888435API
curl https://kongchang.com/api/v1/knowledge/claims/888435MCP
get_claim(id=888435)