Unverified50% confidenceFactExact time
Gemma 2 uses attention logit soft-capping rather than QK-Norm.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGemma系列采用了分组查询注意力（GQA）和RoPE位置编码等Transformer架构改进技术60% similarUnverifiedThe vision and OCR capabilities of the Gemma 4 14B and 12B models have serious deficiencies and cannot reliably handle text extraction from invoices or screenshots.59% similarUnverifiedQwen3.5 introduces a Gating mechanism to perform secondary filtering on attention outputs, which can eliminate the Attention Sink phenomenon and enhance long-text extrapolation capability.52% similarUnverified标准Softmax注意力在极长序列上存在注意力湮没（Attention Sink）问题，催生了Sliding Window Attention、RoPE外推、ALiBi等改进方案51% similarUnverifiedDeBERTa-v3-small 引入解耦注意力机制，在小参数量下保持更强的文本理解能力51% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/912658API
curl https://kongchang.com/api/v1/knowledge/claims/912658MCP
get_claim(id=912658)