[KongchangAI]
Concept一阶Taylor评估 / Taylor重要性评估

First-Order Taylor Importance Estimation

A dimension importance measure used in structured pruning. By taking a first-order Taylor expansion of the loss function, it approximates the effect of removing a dimension as the product of the gradient and the activation value.

Timeline (last 90 days)

Nov 3

First-order Taylor importance estimation approximates the impact on loss of removing a given attention dimension by the product of "gradient x activation value"; the smaller the absolute value of the product, the weaker that dimension's contribution.

Unverified50%

All Facts (1)

Source Articles