ConceptNTK-aware scaling / NTK-aware RoPE
NTK Scaling
A RoPE extension method that scales rotation frequencies to support context windows beyond the training length, adjusting the scaling ratio of components at different frequencies based on neural tangent kernel (NTK) theory.
Timeline (last 90 days)
Oct 12
NTK scaling cannot slow down the fastest frequency pair, because θ₀ = base⁰ = 1, which stays the same no matter how the base is adjusted.
Unverified50%