The context window of modern large models has expanded from the early 4K tokens to 128K or even longer, and the computational complexity of the attention mechanism grows quadratically with context length.