NTK
The Neural Tangent Kernel (NTK) is a theoretical tool for analyzing the training dynamics of neural networks, based on the kernel method whereby infinitely wide networks can be approximated as linear models near initialization.
Timeline (last 90 days)
NTK-based linearization holds only for small weight updates, and the update magnitude of LoRA adapter matrices is not necessarily small enough, so linearization may fail to capture LoRA's true mechanism.
Neural Tangent Kernel (NTK) theory was proposed by Jacot et al. in 2018, revealing a profound connection between wide neural networks and kernel methods.
All Facts (2)
NTK-based linearization holds only for small weight updates, and the update magnitude of LoRA adapter matrices is not necessarily small enough, so linearization may fail to capture LoRA's true mechanism.
50%UnverifiedNeural Tangent Kernel (NTK) theory was proposed by Jacot et al. in 2018, revealing a profound connection between wide neural networks and kernel methods.
50%