Concept一阶Taylor评估 / Taylor重要性评估
First-Order Taylor Importance Estimation
A dimension importance measure used in structured pruning. By taking a first-order Taylor expansion of the loss function, it approximates the effect of removing a dimension as the product of the gradient and the activation value.
Timeline (last 90 days)
Nov 3
First-order Taylor importance estimation approximates the impact on loss of removing a given attention dimension by the product of "gradient x activation value"; the smaller the absolute value of the product, the weaker that dimension's contribution.
Unverified50%