Retrieval Task
A class of tasks in information retrieval that requires a model to find the documents most relevant to a query from a large corpus; such tasks are highly sensitive to subtle differences in embedding vectors, and noise introduced by quantization or dimensionality reduction readily degrades performance.
Timeline (last 90 days)
Retrieval tasks degrade much faster than other task categories.
In retrieval tasks, when the context is extended with neutral filler text, the retrieval accuracy of five open-source models (0.5B to 3B parameters) is almost unaffected, staying between 0.97 and 1.00.
In retrieval tasks, when the filler is masked, accuracy loss at 32K tokens is as high as 45 percentage points; when the same filler remains visible, the loss is zero.
The paper is currently validated only on retrieval tasks and does not yet cover generation tasks or zero-shot classification.
All Facts (4)
Retrieval tasks degrade much faster than other task categories.
50%UnverifiedIn retrieval tasks, when the context is extended with neutral filler text, the retrieval accuracy of five open-source models (0.5B to 3B parameters) is almost unaffected, staying between 0.97 and 1.00.
50%UnverifiedIn retrieval tasks, when the filler is masked, accuracy loss at 32K tokens is as high as 45 percentage points; when the same filler remains visible, the loss is zero.
50%UnverifiedThe paper is currently validated only on retrieval tasks and does not yet cover generation tasks or zero-shot classification.
50%