ConceptOOM / Out of Memory
Out of Memory (GPU)
A failure in which GPU memory is insufficient to hold model weights, activations or the KV cache, causing a task to be interrupted. During training it is usually caused by an overly large batch or lengthening sequences.
Timeline (last 90 days)
Oct 18
Out-of-memory (OOM) during training is mostly caused by oversized batches, longer sequences, or accumulation of intermediate activations; in the inference stage it is often triggered by rising concurrent request counts and KV Cache growing linearly with context length.
Unverified50%