Lost-in-the-middle

In long-context settings, a phenomenon in which key information located in the middle of a prompt is significantly less likely to be retrieved and used by a model than information at the beginning or end. It was identified by Stanford and other institutions through multi-document question-answering experiments, with causes including positional encoding decay, training data distribution bias, and attention dilution.