Stochastic Gradient Descent with Momentum

An optimization algorithm that introduces a momentum term into stochastic gradient descent. It accelerates convergence and smooths the update direction by accumulating an exponentially weighted average of past gradients. It is one of the most commonly used optimizers in deep learning training.