[KongchangAI]
ConceptActivation Checkpointing / 梯度检查点

Activation Recomputation

A memory optimization technique in deep learning training that trades additional computation for reduced memory usage. During the forward pass, intermediate activations are not retained; instead, they are recomputed as needed during the backward pass.

Timeline (last 90 days)

Nov 18

Gradient checkpointing (activation recomputation) does not save all intermediate activations during the forward pass, retaining them only at a few checkpoints, and recomputes the missing activations from the nearest checkpoint during backpropagation.

Unverified50%

All Facts (1)

Source Articles