Unverified50% confidenceFactExact time
In MQA (Multi-Query Attention), all query heads share a single set of KVs, resulting in a smaller cache but with relatively greater quality degradation.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGQA 通过让多个 Query 头共享同一组 Key-Value 头，在保持模型表达力的同时降低了 KV Cache 的显存占用65% similarVerifiedLLaMA 3 8B adopts Grouped Query Attention (GQA), reducing the number of Key and Value attention heads to 1/4 of the number of Query heads.64% similarUnverifiedPrompt缓存必须从前缀开始严格匹配，一旦中间某个token变化，其后所有token的注意力计算都会改变，已缓存的KV对随之失效61% similarUnverifiedRRF does not require normalization of raw scores from different retrievers and relies only on rank information, but when multiple retrievers systematically make the same type of error, RRF only amplifies the mistakes60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/915870API
curl https://kongchang.com/api/v1/knowledge/claims/915870MCP
get_claim(id=915870)