Unverified50% confidenceFactExact time
Standard softmax attention (Vaswani et al., 2017) has O(n^2) time and memory complexity in the sequence length n.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe paper derives that under the softmax retrieval framework, the evidence score margin required to maintain a fixed accuracy must grow at a rate of Ω(√log N), where N is the number of effective distractors rather than the context length.69% similarVerifiedThe self-attention mechanism causes computational complexity to grow quadratically with sequence length, so expanding the context from 1,000 to 4,000 tokens increases the computation by 16 times.68% similarVerifiedThe root cause of attention dilution lies in Softmax normalization: the more tokens in a sequence, the effective attention weight allocated to each position decreases approximately inversely proportional to the sequence length.67% similarUnverifiedThe Min-Max normalization formula is (x - min) / (max - min), which linearly maps data to [0,1] but is extremely sensitive to outliers.67% similarUnverifiedThe decomposition strategy of spatiotemporal attention reduces computational complexity from O((T×H×W)²) to O(T×(H×W)²+(H×W)×T²).65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037964API
curl https://kongchang.com/api/v1/knowledge/claims/1037964MCP
get_claim(id=1037964)