Unverified50% confidenceConceptExact time
Attention mass refers to the proportion of the total attention weights that the model allocates to specific positions (such as the evidence span) when predicting, and can be computed by reading the attention maps of each layer and aggregating them onto the evidence tokens.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedHead-level score modulation means assigning independent modulation parameters or functions to each attention head in multi-head attention, applying different transformations to the attention scores rather than having all heads share a single structural injection scheme.72% similarVerifiedThe core operation of Self-Attention is to compute the dot-product similarity between query vectors and key vectors, then use this as weights to perform a weighted sum of the value vectors.72% similarUnverifiedAttention visualization shows the distribution of attention weights between tokens; activation probes train linear classifiers on hidden layers to test whether information is decodable, which is correlational evidence; feature attribution measures the contribution of each input dimension to the output.70% similarUnverifiedExisting mainstream sparse attention methods filter interactions by retaining entries with larger values or high-quality regions in the attention matrix.70% similarUnverifiedCommon attention mechanism modules for model improvement include the SE attention mechanism, CBAM channel and spatial attention, and ECA efficient channel attention.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134068API
curl https://kongchang.com/api/v1/knowledge/claims/1134068MCP
get_claim(id=1134068)