Unverified60% confidenceFactExact time
GQA has been adopted by Llama 2 70B and the Llama 3 series, while MQA has been adopted by the Falcon series; both reduce the number of cached KV vectors by having multiple query heads share the same set of KV heads.
2
Sources
60%
Confidence
Long-term
Relevance
8/12/2026
First Seen
Sources
Related Claims
UnverifiedGQA（分组查询注意力）等技术可以有效压缩KV Cache，Qwen3系列也采用了类似优化67% similarUnverifiedconda从4.6版本起引入基于libmamba的新求解器后端，将复杂环境依赖解析速度提升数十倍64% similarUnverifiedLLaMA-2 70B采用80层、64个注意力头的架构，并使用GQA（8个KV头共享64个Q头）63% similarUnverifiedSGLang's high-frequency release cadence is aimed at rapidly tracking new version releases of mainstream model families such as Llama, Qwen, and Gemma, as well as new quantization technologies.62% similarVerifiedLoRA and QLoRA are the currently mainstream parameter-efficient fine-tuning methods, allowing fine-tuning to be completed on consumer-grade GPUs.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/735611API
curl https://kongchang.com/api/v1/knowledge/claims/735611MCP
get_claim(id=735611)