Unverified50% confidenceFactExact time
Research has found that there are highly similar feature representations across different layers of a Transformer, and many attention heads contribute negligibly to final performance.
1
Sources
50%
Confidence
Long-term
Relevance
9/5/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe self-attention mechanism of the Transformer can capture dependencies between any two positions in a sequence, and regardless of how far apart the tokens are, the computational cost of calculating attention weights is the same78% similarUnverifiedBecause the Transformer self-attention mechanism exhibits positional bias, earlier input content has a stronger anchoring effect on subsequent generation, which is the underlying reason for the context anchoring effect of system prompts77% similarUnverified各向异性现象的根源在于Transformer的自注意力机制和残差连接结构，使高频词的梯度信号主导嵌入空间整体方向75% similarUnverifiedTransformer注意力层存在位置偏置，模型对序列起始位置(primacy bias)和末尾位置(recency bias)的关注天然高于中间段落74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/860347API
curl https://kongchang.com/api/v1/knowledge/claims/860347MCP
get_claim(id=860347)