Unverified50% confidenceFactExact time
Head-level score modulation means assigning independent modulation parameters or functions to each attention head in multi-head attention, applying different transformations to the attention scores rather than having all heads share a single structural injection scheme.
1
Sources
50%
Confidence
Long-term
Relevance
10/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe "self" in the self-attention mechanism refers to tokens at different positions within the sequence attending to each other, not the model attending to its own parameters or state.76% similarUnverifiedCompared with adding a uniform edge bias to the scores, head-level score modulation allows different heads to attend to different types of edge relationships or subgraph patterns, enhancing the model's ability to express heterogeneous structural information and being naturally compatible with the block-wise computation flow of standard attention kernels.76% similarUnverifiedThe same attention head can perform entirely different sub-functions across different tasks.74% similarVerifiedThe core operation of Self-Attention is to compute the dot-product similarity between query vectors and key vectors, then use this as weights to perform a weighted sum of the value vectors.72% similarUnverified注意力层作为信息路由器在不同序列位置间聚合上下文，其键值是从提示中其他token动态派生的短期记忆表示72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1079034API
curl https://kongchang.com/api/v1/knowledge/claims/1079034MCP
get_claim(id=1079034)