Learning Rate Schedule
A strategy in deep learning training that controls how the learning rate changes as training progresses; common forms include constant, strong decay, and polynomial decay schedules.
Timeline (last 90 days)
When the learning rate is adjusted according to training duration, the locally connected student performs better in the early stage while the convolutional student wins in the later stage, with an analytically predictable crossover point between the two.
The paper deals with a constant stepsize, whereas learning rate schedules commonly used in practice (such as cosine annealing) are not within the current framework.
All Facts (2)
When the learning rate is adjusted according to training duration, the locally connected student performs better in the early stage while the convolutional student wins in the later stage, with an analytically predictable crossover point between the two.
50%UnverifiedThe paper deals with a constant stepsize, whereas learning rate schedules commonly used in practice (such as cosine annealing) are not within the current framework.
50%