Influence Function
A technique in statistics for estimating the degree to which an individual training sample affects model parameters or predictions. Its core idea is to estimate the change in model parameters when a sample is removed; in deep learning it is commonly used to identify mislabeled samples, for data selection and for model debugging.
Timeline (last 90 days)
Point-wise sample selection strategies suffer from sensitivity and redundancy problems, which easily lead to overfitting of the acquisition function.
Influence functions are a technique in statistics for estimating the degree of influence of a single training sample on model parameters or predictions, whose core idea is: if a certain sample were removed and the model retrained, how much would the model parameters change.
In the deep learning context, exactly computing influence functions requires solving the product of the inverse Hessian matrix and the gradient vector, which is extremely costly, so in practice conjugate gradient methods or Hessian-vector products are commonly used for approximation.
Koh & Liang introduced influence functions into deep learning in 2017, using them to identify mislabeled samples and understand model behavior.
All Facts (4)
Point-wise sample selection strategies suffer from sensitivity and redundancy problems, which easily lead to overfitting of the acquisition function.
50%UnverifiedInfluence functions are a technique in statistics for estimating the degree of influence of a single training sample on model parameters or predictions, whose core idea is: if a certain sample were removed and the model retrained, how much would the model parameters change.
50%UnverifiedIn the deep learning context, exactly computing influence functions requires solving the product of the inverse Hessian matrix and the gradient vector, which is extremely costly, so in practice conjugate gradient methods or Hessian-vector products are commonly used for approximation.
50%UnverifiedKoh & Liang introduced influence functions into deep learning in 2017, using them to identify mislabeled samples and understand model behavior.
50%