Tikhonov Regularization
A classical regularization method for solving ill-posed problems. It constrains the solution space by adding a penalty on the parameter norm (such as L2 regularization) to the objective function, making the solution more stable and more unique.
Timeline (last 90 days)
The paper theoretically proves that SPIN implicitly adopts an inverse Hessian approximation, providing a Tikhonov regularization effect that can damp the noise in single-sample Hessian estimates in OCL, thereby mitigating forgetting.
Tikhonov regularization is a classical method for solving ill-posed problems, constraining the solution space by adding a parameter norm penalty (such as L2 regularization) to the objective function, which in second-order optimization is equivalent to adding a positive definite term (such as λI) to the Hessian matrix.
All Facts (2)
The paper theoretically proves that SPIN implicitly adopts an inverse Hessian approximation, providing a Tikhonov regularization effect that can damp the noise in single-sample Hessian estimates in OCL, thereby mitigating forgetting.
50%UnverifiedTikhonov regularization is a classical method for solving ill-posed problems, constraining the solution space by adding a parameter norm penalty (such as L2 regularization) to the objective function, which in second-order optimization is equivalent to adding a positive definite term (such as λI) to the Hessian matrix.
50%