Unverified50% confidenceFactExact time
In linear attention, kernel functions are used to decompose softmax attention into an associable form, reducing inference computational complexity from quadratic in sequence length to linear.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe computational complexity of standard self-attention is quadratic with respect to sequence length, and achieving a context of one million tokens requires introducing techniques such as sparse attention, sliding window attention, and KV Cache compression.78% similarVerifiedOptimization techniques such as Flash Attention and sparse attention can compress memory complexity to near-linear.78% similarUnverifiedLinear attention replaces exp(q^T k) in softmax with a kernel function φ(q)^T φ(k), reducing overall complexity from O(n²d) to O(nd²).77% similarVerifiedSelf-attention weights are normalized through the Softmax function, and the sum of the weights is always 1.77% similarVerifiedSliding window attention reduces the complexity of attention computation from O(n²) to O(n·w), linearizing GPU memory overhead for long texts.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1073419API
curl https://kongchang.com/api/v1/knowledge/claims/1073419MCP
get_claim(id=1073419)