Concept
Online Softmax
A streaming Softmax computation algorithm that rewrites Softmax into a recursively updatable form, dynamically correcting the existing maximum and normalization factor as each tile is processed, yielding exact results without materializing the full matrix.
Timeline (last 90 days)
Oct 14
Online Softmax is the key algorithm behind efficient attention implementations such as FlashAttention, avoiding materialization of the full [B, H, S, S] matrix through streaming recurrence.
Unverified50%