[KongchangAI]
ConceptGradient Norm Penalty

Gradient Norm Penalty

A regularization technique for robust training that suppresses harmful updates caused by noisy samples or tokens by penalizing excessively large gradient norms; it typically requires additional backpropagation computation.

Timeline (last 90 days)

Nov 10

PORT's CDF weighting design constitutes an efficient forward-pass surrogate for gradient-norm penalization, requiring no additional backpropagation computation.

Unverified50%

All Facts (1)

Source Articles