recency bias
The general tendency of language models to assign greater weight to content nearer the end of the context. It can cause information loss after compaction and can also be exploited to achieve state restoration.
Timeline (last 90 days)
Large language models tend to assign greater attention weight to recent inputs and high-frequency words, a phenomenon known as recency bias.
When autoregressive generative models predict the next token, they exhibit a recency bias toward the end of the sequence, a sequential end-of-sequence effect that has been documented in pure-text research.
All Facts (2)
Large language models tend to assign greater attention weight to recent inputs and high-frequency words, a phenomenon known as recency bias.
50%UnverifiedWhen autoregressive generative models predict the next token, they exhibit a recency bias toward the end of the sequence, a sequential end-of-sequence effect that has been documented in pure-text research.
50%