Found in the Middle
A research paper published in ACL Findings 2024 that addresses the lost-in-the-middle problem in large language models. The paper attributes the phenomenon to positional bias in attention mechanisms and proposes calibration methods to improve the utilization of information located at middle positions in RAG tasks.
Timeline (last 90 days)
The paper "Found in the Middle" (Hsieh et al., ACL Findings 2024) attributes the lost-in-the-middle effect to the attention mechanism: tokens at the beginning and end of the input receive more attention regardless of their actual relevance.
The calibration method proposed in "Found in the Middle" weakens attention bias and improves RAG results by up to 10 percentage points over existing methods.
All Facts (2)
The paper "Found in the Middle" (Hsieh et al., ACL Findings 2024) attributes the lost-in-the-middle effect to the attention mechanism: tokens at the beginning and end of the input receive more attention regardless of their actual relevance.
50%UnverifiedThe calibration method proposed in "Found in the Middle" weakens attention bias and improves RAG results by up to 10 percentage points over existing methods.
50%