Unverified50% confidenceFactExact time
Additive sinusoidal positional encoding gives different attention scores for the same offset but different absolute positions, because after expanding the dot product, cross terms related to the absolute positions remain.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
Unverified相对位置编码（如 RoPE、ALiBi）会对距离较远的 token 对施加注意力衰减偏置，序列边界位置拥有更多邻近token的注意力积累优势72% similarUnverifiedPositional encoding is used to inject word order information into the model, because the attention mechanism itself does not distinguish positional order.72% similarUnverifiedALiBi 不引入显式位置编码，而是在注意力分数上叠加与相对距离成线性比例的负偏置项70% similarUnverified自注意力机制的计算路径长度与序列长度无关，任意两个位置之间的信息传递都是直接的，解决了长序列建模中的梯度消失问题70% similarUnverifiedThe self-attention in an encoder is bidirectional, allowing each position to simultaneously see context from both left and right sides69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1005581API
curl https://kongchang.com/api/v1/knowledge/claims/1005581MCP
get_claim(id=1005581)