Sieve and Sage
A research framework published on arXiv that addresses the abstention decision problem in retrieval-augmented language models (RALMs). It decouples retrieval failures into unanswerable and distracted states, and implements a two-stage pipeline consisting of a lightweight Sieve component and a heavyweight Sage component to improve the reliability of RAG systems.
Timeline (last 90 days)
Sieve and Sage is a framework published on arXiv for addressing the question of when retrieval-augmented language models (RALMs) should abstain.
The Sieve and Sage framework decouples retrieval failure into two states: Unanswerable and Distracted.
The Unanswerable state refers to the situation where the required evidence simply does not exist in the retrieval results, and the correct action is to abstain directly.
The Distracted state refers to the situation where relevant evidence exists but is mixed with conflicting, negated, or adversarial information.
Compared to single-stage baselines, the Sieve and Sage system achieves an accuracy improvement of up to 69.4 percentage points.
The Macro-F1 metric of Sieve and Sage improves by up to 55.2 percentage points.
Sieve and Sage achieves an inference speedup of up to 1.99×.
The research team evaluated the framework in two types of scenarios: general-domain and high-stakes expert-domain.
All Facts (8)
Sieve and Sage is a framework published on arXiv for addressing the question of when retrieval-augmented language models (RALMs) should abstain.
50%UnverifiedThe Sieve and Sage framework decouples retrieval failure into two states: Unanswerable and Distracted.
50%UnverifiedThe Unanswerable state refers to the situation where the required evidence simply does not exist in the retrieval results, and the correct action is to abstain directly.
50%UnverifiedThe Distracted state refers to the situation where relevant evidence exists but is mixed with conflicting, negated, or adversarial information.
50%UnverifiedCompared to single-stage baselines, the Sieve and Sage system achieves an accuracy improvement of up to 69.4 percentage points.
50%UnverifiedThe Macro-F1 metric of Sieve and Sage improves by up to 55.2 percentage points.
50%UnverifiedSieve and Sage achieves an inference speedup of up to 1.99×.
50%UnverifiedThe research team evaluated the framework in two types of scenarios: general-domain and high-stakes expert-domain.
50%