Unverified50% confidenceFactExact time
Entries with large attention weights do not necessarily contribute most to the output after matrix multiplication; weight and output contribution are two distinct quantities.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe paper argues that treating the attention matrix as a 'bag of values' from which to pick the largest entries is a mathematically incorrect intuition, because the attention matrix is a structured operator participating in matrix multiplication.78% similarVerified自注意力机制的计算复杂度随序列长度呈近似二次方增长，序列长度为n时注意力矩阵计算量正比于n²76% similarVerifiedContext length and computational load grow in an approximately quadratic relationship, influenced by the attention mechanism.73% similarUnverifiedThe arithmetic intensity of the MLP layer is B×T, the arithmetic intensity of the attention layer is S×T/(S+T), and during generation the arithmetic intensity of the attention layer is approximately 1 and is completely independent of batch size.70% similarUnverifiedThe computational cost of standard self-attention grows as O(n²) with sequence length, limiting the scale of context windows70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990370API
curl https://kongchang.com/api/v1/knowledge/claims/990370MCP
get_claim(id=990370)