ConceptImplicit Regularization
Implicit Regularization
Implicit regularization refers to the phenomenon whereby optimization algorithms such as gradient descent inherently bias toward low-complexity solutions without any explicit regularization term being added. This property is considered one of the important sources of generalization ability in neural networks.
Timeline (last 90 days)
Sep 20
Gradient descent naturally tends to select flatter, simpler solutions from among the many solutions that perfectly fit the training set, and this implicit regularization effect is closely related to generalization ability.
Unverified50%