Stochastic Gradient Descent
A variant of gradient descent that computes gradients and updates parameters using one sample or a small mini-batch at a time. It is more computationally efficient than full-batch gradient descent and is well suited for training on large-scale datasets.
Timeline (last 90 days)
The online estimation algorithm studied in the paper is a single-pass stochastic gradient algorithm with a constant learning rate γ, in the recursive form θ_t = θ_{t-1} + γ * ψ(Y_t, θ_{t-1}), where ψ is the expectile score function.
Stochastic gradient descent (SGD) uses only one sample at a time, which is fast but produces noisy update directions.
Extending the AMP framework from full-batch gradient descent to stochastic gradient descent (SGD) requires handling the additional sampling randomness and data reuse bias.
All Facts (3)
The online estimation algorithm studied in the paper is a single-pass stochastic gradient algorithm with a constant learning rate γ, in the recursive form θ_t = θ_{t-1} + γ * ψ(Y_t, θ_{t-1}), where ψ is the expectile score function.
50%UnverifiedStochastic gradient descent (SGD) uses only one sample at a time, which is fast but produces noisy update directions.
50%UnverifiedExtending the AMP framework from full-batch gradient descent to stochastic gradient descent (SGD) requires handling the additional sampling randomness and data reuse bias.
50%