Implicit Bias
The property whereby an optimization algorithm implicitly favors a certain class of solutions during training, determining the direction of the solutions to which the algorithm converges and its generalization behavior.
Timeline (last 90 days)
In adversarial linear logistic regression, the gap in implicit bias between adversarial training and standard training persists even when the step size is arbitrarily large.
Gradient descent naturally tends to select flatter, simpler solutions from among the many solutions that perfectly fit the training set, and this implicit regularization effect is closely related to generalization ability.
All Facts (2)
In adversarial linear logistic regression, the gap in implicit bias between adversarial training and standard training persists even when the step size is arbitrarily large.
50%UnverifiedGradient descent naturally tends to select flatter, simpler solutions from among the many solutions that perfectly fit the training set, and this implicit regularization effect is closely related to generalization ability.
50%