Unverified50% confidenceFactExact time
RoPE injects absolute position into the query and key, but the attention score between any two tokens depends only on the distance between them (m-n), with the absolute positions canceling out in the dot product.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedRoPE acts on the Query and Key vectors in attention computation via rotational transformations, making the attention score between two tokens depend only on their relative positions.78% similarVerifiedRoPE (Rotary Position Embedding) injects positional information into attention computation in the form of a rotation matrix, preserving relative positional relationships across arbitrary sequence lengths.65% similarUnverifiedExtrapolation and interpolation methods for RoPE (Rotary Position Embedding) can extend a model's positional awareness range without retraining.63% similarUnverifiedRotary Position Embedding (RoPE) was proposed by Su Jianlin in 2021; the position-dependent term in its attention scores depends only on relative position.62% similarUnverifiedThe self-attention mechanism captures dependencies of arbitrary distance in a single forward pass by dynamically computing association weights between an element and all other elements in the sequence, overcoming the long-range dependency difficulties caused by vanishing gradients in RNNs/LSTMs.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1005579API
curl https://kongchang.com/api/v1/knowledge/claims/1005579MCP
get_claim(id=1005579)