Unverified50% confidenceFactExact time
In multi-query attention (MQA), Hkv = 1, and the cache is even smaller than with GQA.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedMLA can reduce the KV Cache VRAM usage to 5%–13% of traditional multi-head attention.67% similarUnverifiedMASA's score quantifies how much the Frobenius error between the matrix product ÂV and the full attention output AV can be reduced by adding a given unit, taking into account both attention weights and value vector magnitudes.66% similarUnverifiedGLM-5.2 采用 IndexShare 技术，每四个稀疏注意力层复用同一个索引器，在 1M 上下文长度下将每 token 的 FLOPs 降低了 2.9 倍63% similarUnverifiedUnder fixed compute, single-head attention scores 0.9 points lower than the optimal setting, and quality actually decreases when there are too many heads.63% similarUnverifiedOn the validation-set average NLL metric, Continuum Attention reduces by 0.0200 nats/byte compared to ordinary softmax attention.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1057308API
curl https://kongchang.com/api/v1/knowledge/claims/1057308MCP
get_claim(id=1057308)