Unverified50% confidenceFactExact time
Gradient descent naturally tends to select flatter, simpler solutions from among the many solutions that perfectly fit the training set, and this implicit regularization effect is closely related to generalization ability.
1
Sources
50%
Confidence
Long-term
Relevance
9/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedScaling Laws describe the decrease in pre-training loss, not a linear improvement in all downstream capabilities; some capabilities, such as complex reasoning, exhibit emergent properties.77% similarVerifiedGradient descent has spawned three main variants — batch gradient descent, stochastic gradient descent, and mini-batch gradient descent — and the Adam optimizer is a combined improvement built on momentum and adaptive learning rates.74% similarUnverifiedDuring cross-domain fine-tuning, an excessively large learning rate will destroy pre-trained weights, while an excessively small one will result in almost no learning, requiring a careful trade-off.74% similarUnverifiedThe retraining involved a clear trade-off: slight decreases in some very high scores on cleaner validation distributions, in exchange for improved real-world performance.74% similarUnverified批量梯度下降使用全部训练数据计算梯度，随机梯度下降每次只用一个样本，小批量梯度下降在两者之间取得平衡74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/936213API
curl https://kongchang.com/api/v1/knowledge/claims/936213MCP
get_claim(id=936213)