Unverified60% confidenceFactExact time
A 2017 study by Keskar et al. showed that large-batch training tends to converge to sharp minima corresponding to worse generalization, while small-batch training tends toward flat minima.
2
Sources
60%
Confidence
Long-term
Relevance
7/13/2026
First Seen
Sources
The Theory of Deep Learning: Why Do Neural Networks Work? A Complete Overview of Theoretical Research
hackernewshackernews7/11/2026
Related Claims
VerifiedFine-tuning typically requires only one percent or less of the original training cost to achieve domain performance close to that of a dedicated model.78% similarUnverifiedIn distributed training, a small number of nodes whose kernel parameters are inconsistent with the rest can drag down the throughput of the entire training job, creating bottleneck nodes.77% similarUnverifiedIf the quality of the guidance source is poor (large δ), extended training will amplify the bias, in which case the guidance weight should be reduced or external guidance should be stopped altogether.75% similarUnverifiedThe cost of a supervised small model is that it requires a certain amount of high-quality annotation, and its generalization ability is limited by the training distribution.74% similarUnverifiedThe Rif and Kabyle variety subsets are extremely small in scale, which is a significant shortcoming for training models that cover all varieties.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/501135API
curl https://kongchang.com/api/v1/knowledge/claims/501135MCP
get_claim(id=501135)