Unverified50% confidenceFactExact time
In standard training, gradient descent with small step sizes tends to converge to the minimum L2-norm solution for linear models.
1
Sources
50%
Confidence
Long-term
Relevance
11/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGradient descent naturally tends to select flatter, simpler solutions from among the many solutions that perfectly fit the training set, and this implicit regularization effect is closely related to generalization ability.77% similarVerifiedSmaller models often show exponential rather than linear degradation in performance on long reasoning chain tasks.77% similarUnverifiedUnder a local quadratic model, a larger batch means fewer but longer update steps, with each step incurring a greater loss on curvature, resulting in a critical batch size that depends on the learning rate, beyond which sample efficiency declines.73% similarVerifiedIf the dataset is small while the model has too many parameters, the model may memorize the training data prematurely and degenerate; this can be mitigated by adding more data, applying dropout regularization, or reducing model size.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1142246API
curl https://kongchang.com/api/v1/knowledge/claims/1142246MCP
get_claim(id=1142246)