Unverified50% confidenceFactExact time
Frequency-collapse attention replaces the conventional Q/K dot product with an inner product computed after bandpass filtering at learned specific frequencies.
1
Sources
50%
Confidence
Long-term
Relevance
10/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedStandard attention's computation grows quadratically with sequence length, whereas the low-rank bottleneck decomposes the transformation into two steps: first compressing to r dimensions and then expanding.68% similarUnverifiedThe root cause of attention dilution lies in Softmax normalization: the more tokens in a sequence, the effective attention weight allocated to each position decreases approximately inversely proportional to the sequence length.65% similarUnverified自回归模型生成长序列时注意力机制计算复杂度随序列长度呈二次增长，且误差会累积导致音色漂移65% similarUnverifiedThe KV Cache footprint of linear attention is independent of sequence length, whereas the cache of traditional attention grows linearly with sequence length.65% similarUnverifiedQuantization error has a non-uniform distribution characteristic; the Q/K/V matrices of the attention mechanism and the last few layers are more sensitive to quantization.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/971773API
curl https://kongchang.com/api/v1/knowledge/claims/971773MCP
get_claim(id=971773)