Shampoo Optimizer

A higher-order optimization algorithm that maintains separate second-order statistics for each dimension of parameter tensors, avoiding explicit computation of the full Hessian matrix. Under large-batch training with the same computational budget, it achieves convergence in fewer iterations compared to standard first-order optimizers.