mini-KFAC
An optimization method that approximates the block structure of the Fisher information matrix using Kronecker-factored decomposition. It captures second-order curvature information among intra-layer parameters, providing more accurate parameter importance estimates than diagonal EWC approaches.
Timeline (last 90 days)
In scenarios where regularization is needed, mini-KFAC EWC can achieve lower word error rates and a more stable optimization process compared to standard diagonal EWC.
mini-KFAC is a further simplification of full KFAC that controls memory and computational overhead by reducing the number of samples used for statistical estimation per layer, striking a balance between accuracy and efficiency.
All Facts (2)
In scenarios where regularization is needed, mini-KFAC EWC can achieve lower word error rates and a more stable optimization process compared to standard diagonal EWC.
50%Unverifiedmini-KFAC is a further simplification of full KFAC that controls memory and computational overhead by reducing the number of samples used for statistical estimation per layer, striking a balance between accuracy and efficiency.
50%