Unverified50% confidenceOpinionExact time
The paper argues that treating the attention matrix as a 'bag of values' from which to pick the largest entries is a mathematically incorrect intuition, because the attention matrix is a structured operator participating in matrix multiplication.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEntries with large attention weights do not necessarily contribute most to the output after matrix multiplication; weight and output contribution are two distinct quantities.78% similarUnverifiedIn the standard attention mechanism, the Query matrix and Key matrix perform matrix multiplication to obtain an n×n attention score matrix, with a computational complexity of O(n²·d).76% similarVerified自注意力机制的计算复杂度随序列长度呈近似二次方增长，序列长度为n时注意力矩阵计算量正比于n²70% similarUnverified选择性SSM本质上等价于一种特殊的半可分矩阵（semiseparable matrix）形式的注意力70% similarUnverified标准 Scaled Dot-Product Attention 中 Q 与 K 做矩阵乘法得到 n×n 注意力分数矩阵，计算量为 O(n²·d)69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990369API
curl https://kongchang.com/api/v1/knowledge/claims/990369MCP
get_claim(id=990369)