Recency Bias
A systematic tendency of LLMs in long contexts whereby attention weights assigned to content at the end are significantly higher than those for the middle portion, related to the self-attention distribution in Transformers and positional encoding.
Timeline (last 90 days)
Recency bias is a systematic tendency of LLM attention mechanisms: in long contexts, the model's attention weight on content at the end is significantly higher than on the middle, and attackers often exploit this by appending instructions such as "ignore all previous rules" to the end of user input.
In the class-incremental graph learning scenario of graph neural networks, as new classes arrive one after another the model exhibits catastrophic forgetting, and predictions are often dominated by the most recently learned classes, producing recency bias.
Continually fine-tuning a learnable router on new data biases routing toward the most recent components, producing routing-level recency bias.
All Facts (3)
Recency bias is a systematic tendency of LLM attention mechanisms: in long contexts, the model's attention weight on content at the end is significantly higher than on the middle, and attackers often exploit this by appending instructions such as "ignore all previous rules" to the end of user input.
50%UnverifiedIn the class-incremental graph learning scenario of graph neural networks, as new classes arrive one after another the model exhibits catastrophic forgetting, and predictions are often dominated by the most recently learned classes, producing recency bias.
50%UnverifiedContinually fine-tuning a learnable router on new data biases routing toward the most recent components, producing routing-level recency bias.
50%