Grouped-Query Attention
A variant of attention mechanisms in which query heads are grouped to share key and value heads, reducing KV cache size and balancing inference efficiency against model quality.
Timeline (last 90 days)
Setting kv=1 in GQA gives each layer only one key-value vector, compressing the KV cache to be almost independent of sequence length, thereby avoiding crowding out L3 space.
Grouped-Query Attention (GQA) allows multiple attention heads to share grouped K and V, striking a balance between memory usage and model expressiveness.
All Facts (3)
Gemma系列采用了分组查询注意力（GQA）和RoPE位置编码等Transformer架构改进技术
80%UnverifiedSetting kv=1 in GQA gives each layer only one key-value vector, compressing the KV cache to be almost independent of sequence length, thereby avoiding crowding out L3 space.
50%UnverifiedGrouped-Query Attention (GQA) allows multiple attention heads to share grouped K and V, striking a balance between memory usage and model expressiveness.
50%