Standard Training
A conventional model training approach that directly minimizes the original loss function without introducing adversarial perturbations, in contrast to adversarial training.
Timeline (last 90 days)
A recent paper specifically studies the gradient descent (GD) dynamics of adversarial training (AT) with large step sizes, revealing three key differences from standard training (ST).
In adversarial linear logistic regression, the gap in implicit bias between adversarial training and standard training persists even when the step size is arbitrarily large.
In standard training, gradient descent with small step sizes tends to converge to the minimum L2-norm solution for linear models.
In standard training, gradient descent has a uniform convergence rate guarantee on smooth convex problems, independent of the specific data distribution.
All Facts (4)
A recent paper specifically studies the gradient descent (GD) dynamics of adversarial training (AT) with large step sizes, revealing three key differences from standard training (ST).
50%UnverifiedIn adversarial linear logistic regression, the gap in implicit bias between adversarial training and standard training persists even when the step size is arbitrarily large.
50%UnverifiedIn standard training, gradient descent with small step sizes tends to converge to the minimum L2-norm solution for linear models.
50%UnverifiedIn standard training, gradient descent has a uniform convergence rate guarantee on smooth convex problems, independent of the specific data distribution.
50%