Unverified50% confidenceFactExact time
Many current LLMs use rotary position embeddings (RoPE), rotating the query and key vectors by position so that attention depends on relative distance.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedRoPE (Rotary Position Embedding) injects positional information into attention computation in the form of a rotation matrix, preserving relative positional relationships across arbitrary sequence lengths.76% similarUnverifiedRotary Position Embedding (RoPE) was proposed by Su Jianlin in 2021; the position-dependent term in its attention scores depends only on relative position.74% similarUnverifiedRoPE injects absolute position into the query and key, but the attention score between any two tokens depends only on the distance between them (m-n), with the absolute positions canceling out in the dot product.70% similarUnverifiedExtrapolation and interpolation methods for RoPE (Rotary Position Embedding) can extend a model's positional awareness range without retraining.70% similarVerifiedLong-context optimization techniques include sliding window attention, sparse attention, and RoPE rotary positional encoding extension schemes.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1066283API
curl https://kongchang.com/api/v1/knowledge/claims/1066283MCP
get_claim(id=1066283)