Loss Spike

A phenomenon in large model training in which the loss value rises sharply within a short period. Loss spikes typically indicate gradient explosion, numerical overflow, data contamination, or severe anomalies in parameter updates. While an important indicator of training health, loss spikes exhibit temporal lag relative to the underlying root cause, complicating timely diagnosis.