Lost in the Middle
A performance phenomenon in long-context language models: when key information is located in the middle of the context, the model's ability to retrieve and use it is significantly lower than when the information is at the beginning or end, following a U-shaped curve of degradation.
Core Facts
Timeline (last 90 days)
The larger the context window, the more information the model can remember in a single call, but it also brings rising inference costs and the problem of intermediate information being ignored.
Simply expanding the context window does not equate to a linear improvement in comprehension ability; there is a phenomenon of forgetting in the middle.
In 2023, Stanford University and other institutions published research finding that large language models pay noticeably more attention to the beginning and end of input content than to the middle when processing long documents, exhibiting a 'U-shaped curve' effect.
Placing key information at the beginning and end of the input can mitigate the impact of the 'lost in the middle' effect on results.
All Facts (4)
The larger the context window, the more information the model can remember in a single call, but it also brings rising inference costs and the problem of intermediate information being ignored.
65%UnverifiedSimply expanding the context window does not equate to a linear improvement in comprehension ability; there is a phenomenon of forgetting in the middle.
50%UnverifiedIn 2023, Stanford University and other institutions published research finding that large language models pay noticeably more attention to the beginning and end of input content than to the middle when processing long documents, exhibiting a 'U-shaped curve' effect.
50%UnverifiedPlacing key information at the beginning and end of the input can mitigate the impact of the 'lost in the middle' effect on results.
50%