Unverified50% confidenceFactExact time
In the attention mechanism, the dot product of queries and keys is very sensitive to the scale of input features, and large differences in feature scales among physical quantities may cause an imbalanced distribution of attention weights.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe use of attention weights as a basis for interpretability is controversial, as high attention weights do not always correspond to high causal influence.78% similarUnverifiedDifferent attention heads in multi-head attention tend to capture different types of semantic relationships, with some focusing on syntactic dependencies and others on coreference relations.77% similarUnverifiedEvaluation mechanisms concentrate research attention highly on a small number of dominant datasets, creating a Matthew effect.77% similarUnverifiedThe calculation of attribution relationships is typically based on gradients or attention weights.77% similarUnverified提示词中靠前的词汇对生成结果影响更大，因为交叉注意力机制对序列开头的token赋予略高的注意力分数77% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1126651API
curl https://kongchang.com/api/v1/knowledge/claims/1126651MCP
get_claim(id=1126651)