Unverified50% confidenceFactExact time
选择性SSM本质上等价于一种特殊的半可分矩阵（semiseparable matrix）形式的注意力
1
Sources
50%
Confidence
Long-term
Relevance
8/12/2026
First Seen
Sources
Related Claims
UnverifiedIn the standard attention mechanism, the Query matrix and Key matrix perform matrix multiplication to obtain an n×n attention score matrix, with a computational complexity of O(n²·d).72% similarUnverifiedThe Multi-Query Attention technique allows multiple attention heads to share a single set of Key and Value projection matrices, compressing KV cache memory requirements to 1/H of the original, at the cost of a slight reduction in model expressiveness.70% similarVerifiedThe self-attention mechanism allows the model to simultaneously see the information of all other tokens in the input sequence when processing each token, and the computational complexity of the attention matrix is O(n²)68% similarUnverified标准 Scaled Dot-Product Attention 中 Q 与 K 做矩阵乘法得到 n×n 注意力分数矩阵，计算量为 O(n²·d)67% similarUnverified通过Hessian矩阵或Fisher信息矩阵进行逐层敏感性分析可识别模型中的量化瓶颈层，注意力层Q/K投影通常为高敏感层，FFN层大部分权重可安全压缩至FP8甚至INT866% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/735122API
curl https://kongchang.com/api/v1/knowledge/claims/735122MCP
get_claim(id=735122)