Conceptscore aliasing
Score Aliasing
One of three failure mechanisms underlying long-context degradation. It refers to the condition in which attention scores of distractor tokens become indistinguishable from those of genuine evidence tokens, causing the model to fail at correctly identifying critical information.
Timeline (last 90 days)
Sep 22
The paper links long-context degradation to three mechanisms: score aliasing, positional aliasing, and softmax dilution.
Unverified50%