Unverified50% confidenceFactExact time
Current mainstream Transformer architecture models (such as the GPT series, Claude, Gemini, etc.) have an attention decay problem when processing long contexts, with memory of earlier content becoming increasingly unstable as conversations grow longer
1
Sources
50%
Confidence
Long-term
Relevance
10/3/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe attention mechanism of the Transformer architecture significantly reduces its focus on early content when processing very long sequences, causing the model to forget earlier settings77% similarUnverified当前主流LLM采用Transformer架构，随着上下文长度增加会出现对早期信息的注意力衰减现象75% similarUnverifiedLLM在长序列生成中因注意力稀释而遗忘之前约束的现象被称为「长上下文遗忘（Lost in the Middle）」，是当前Transformer架构的固有局限73% similarUnverifiedCurrent frontier models face attention decay when handling long-range dependencies, with the ability to maintain consistency of early-established settings declining significantly as context length increases.73% similarUnverified当前大模型并没有真正的持久化记忆机制，随着对话轮次增加，早期需求描述在注意力机制中的权重会逐渐衰减72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/969436API
curl https://kongchang.com/api/v1/knowledge/claims/969436MCP
get_claim(id=969436)