Unverified50% confidenceFactExact time
GQA was proposed by Google in 2023 as a compromise between MHA and MQA; the 70B version of LLaMA 2 uses 8-group GQA, reducing the KV cache to 1/4 of that of MHA.
1
Sources
50%
Confidence
Long-term
Relevance
8/29/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGQA由Google在2023年提出，将多个查询头共享同一组Key-Value头，可降低KV Cache内存消耗50%-75%74% similarUnverifiedGQA由Google在2023年提出，将多个查询头共享同一组Key-Value头，已被LLaMA 3、Mistral等主流模型采用71% similarVerifiedGQA (Grouped Query Attention) is an efficient variant of standard multi-head attention, proposed by Google in 2023, and has been adopted by mainstream models such as LLaMA 3 and Mistral.66% similarUnverifiedGQA was first proposed by Google during the training of PaLM 2 and subsequently adopted by open-source models such as Llama 2/3 and Mistral.58% similarUnverifiedMoE架构最早由Google在2017年提出，后被Mistral的Mixtral系列和阿里的Qwen系列等广泛采用57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/819742API
curl https://kongchang.com/api/v1/knowledge/claims/819742MCP
get_claim(id=819742)