Unverified50% confidenceFactExact time
Transformer注意力层存在位置偏置，模型对序列起始位置(primacy bias)和末尾位置(recency bias)的关注天然高于中间段落
1
Sources
50%
Confidence
Long-term
Relevance
7/18/2026
First Seen
Sources
Related Claims
UnverifiedBecause the Transformer self-attention mechanism exhibits positional bias, earlier input content has a stronger anchoring effect on subsequent generation, which is the underlying reason for the context anchoring effect of system prompts82% similarVerifiedThe self-attention mechanism of the Transformer can capture dependencies between any two positions in a sequence, and regardless of how far apart the tokens are, the computational cost of calculating attention weights is the same76% similarUnverifiedThe causal attention mask of Transformer ensures that predictions at each position depend only on historical information by setting the upper triangular portion of the attention matrix to negative infinity.76% similarVerifiedThe Lost in the Middle effect originates from an inherent architectural bias in the Transformer attention mechanism where tokens in the middle positions receive lower attention weights, and no solution currently exists to completely eliminate it in all scenarios73% similarUnverifiedIn the Transformer architecture, the Self-Attention layer normalizes relevance scores across all positions using the Softmax function; when the sequence length expands from a few thousand tokens to tens of thousands, the attention weight allocated to each position is greatly diluted.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/554407API
curl https://kongchang.com/api/v1/knowledge/claims/554407MCP
get_claim(id=554407)