Verified75% confidenceFactExact time
GQA (Grouped Query Attention) is an efficient variant of standard multi-head attention, proposed by Google in 2023, and has been adopted by mainstream models such as LLaMA 3 and Mistral.
5
Sources
75%
Confidence
Long-term
Relevance
7/6/2026
First Seen
Sources
Needle: Fine-Tuning a 26M Ultralight Tool-Calling Model Locally — Full Walkthrough
bilibili薛定猫AI7/3/2026
Related Claims
UnverifiedGQA由Google在2023年提出，将多个查询头共享同一组Key-Value头，已被LLaMA 3、Mistral等主流模型采用81% similarUnverifiedGQA is an attention optimization scheme proposed by Google in 2023 that groups query heads so each group shares a set of KV projections, typically reducing KV cache VRAM by 4–8×.76% similarUnverifiedGQA was first proposed by Google during the training of PaLM 2 and subsequently adopted by open-source models such as Llama 2/3 and Mistral.69% similarUnverifiedGQA由Google在2023年提出，将多个查询头共享同一组Key-Value头，可降低KV Cache内存消耗50%-75%67% similarUnverifiedGQA was proposed by Google in 2023 as a compromise between MHA and MQA; the 70B version of LLaMA 2 uses 8-group GQA, reducing the KV cache to 1/4 of that of MHA.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/114485API
curl https://kongchang.com/api/v1/knowledge/claims/114485MCP
get_claim(id=114485)