Unverified50% confidenceFactExact time
Transformer模型存在位置偏差，对中间位置信息的检索能力尤其薄弱，即lost in the middle问题
1
Sources
50%
Confidence
Long-term
Relevance
8/10/2026
First Seen
Sources
Related Claims
UnverifiedTransformer models exhibit a "Lost in the Middle" phenomenon, where attention to content located in the middle of the context and far from the current generation position systematically decreases.80% similarUnverified斯坦福大学2023年的研究将Transformer在处理超长上下文时对中间位置信息遗忘率显著高于首尾位置的现象命名为'Lost in the Middle'问题78% similarUnverifiedIn Transformer models, the first and last few layers are more sensitive to quantization, the middle layers have higher redundancy, and attention layers and feed-forward network layers differ in their tolerance to quantization.75% similarUnverifiedTransformer 中任意两个位置之间的信息传递路径长度恒为1，从根本上克服了 RNN 在长序列上因梯度消失导致的遗忘问题75% similarUnverifiedTransformers lack the inductive biases that CNNs naturally possess, such as locality and translation invariance, so DETR must rely entirely on data to learn spatial relationships, resulting in an extremely large demand for data volume.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/723142API
curl https://kongchang.com/api/v1/knowledge/claims/723142MCP
get_claim(id=723142)