[KongchangAI]
Conceptpositional aliasing

Positional Aliasing

One of three failure mechanisms underlying long-context degradation. In extremely long sequences, the rotational angle differences in positional encodings such as RoPE tend toward periodic overlap, causing the model's perception of distinct positions to become ambiguous. This provides an encoding-level explanation for the "lost in the middle" phenomenon.

Timeline (last 90 days)

Sep 22

The paper links long-context degradation to three mechanisms: score aliasing, positional aliasing, and softmax dilution.

Unverified50%

All Facts (1)

Source Articles