Context window extension techniques such as NTK-aware scaling of RoPE, ALiBi, and Ring Attention have pushed context length from the early 4K to 128K or even millions of tokens, but face the limitations of quadratic growth in computational cost and long-range attention decay.