Unverified50% confidenceFactExact time
The paper abstracts the context degradation phenomenon as a problem of extreme-value interference in attention.
1
Sources
50%
Confidence
Long-term
Relevance
9/22/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe researchers propose the attention dilution effect hypothesis to explain why batching reduces overcorrection.73% similarUnverifiedThe attention score of key evidence has an upper bound, while the maximum score among all effective distractors grows as the number of distractors increases.69% similarUnverifiedThe paper points out that the root cause of long-context degradation lies not entirely in context length, but in the number of effective distractors it contains.66% similarUnverifiedThe root cause of attention dilution lies in Softmax normalization: the more tokens in a sequence, the effective attention weight allocated to each position decreases approximately inversely proportional to the sequence length.66% similarUnverifiedWhen prompts are too long, large models experience an 'attention explosion' phenomenon, where excessively long inputs cause effective information to be lost.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/940094API
curl https://kongchang.com/api/v1/knowledge/claims/940094MCP
get_claim(id=940094)