SAFE-QE
A training-free, retriever-agnostic query expansion method that identifies and down-weights hallucinated expansion terms generated by LLMs through a subsequence association filtering mechanism. It models expansion term selection as a constrained coverage maximization problem and introduces the HETR metric to quantify query-side faithfulness.
Timeline (last 90 days)
SAFE-QE is a training-free and retriever-agnostic query expansion method, whose full name is Subsequence-Association Filtered, breadth-aware Query Expansion.
SAFE-QE scores each candidate expansion term based on its cross-context reproducibility conditioned on the corpus: if a term repeatedly appears in randomized query contexts yet finds no support in the pseudo-relevant document set, it is likely the product of model-driven hallucinated associations and is down-weighted.
SAFE-QE models the selection of expansion terms as a constrained coverage maximization problem over the pseudo-relevant set, with the objective of retaining a set of expansion terms that is sufficiently broad, consistent, and stable under resampling.
The study introduces a new query-side faithfulness metric, HETR (Hallucinated Expansion Term Rate), which counts the number of injected expansion terms that have no support in any relevant document.
SAFE-QE was evaluated on two benchmark retrieval collections, TREC and BEIR.
On the nDCG@10 ranking metric, SAFE-QE shows no statistically significant difference from generative expansion and corpus-guided expansion.
SAFE-QE injects only 4.3 unsupported terms per query, compared to 27.8 for Query2doc and 57.5 for HyDE.
The HyDE method injects approximately 13 times more hallucinated terms per query than SAFE-QE.
On learned sparse retrievers, all expansion methods exhibited performance regression, corroborating concerns that query expansion may harm strong retrievers, while SAFE-QE still maintained a low hallucination rate in this scenario.
The core conclusion of the study is that gains in faithfulness need not come at the cost of ranking quality, and hallucination filtering can be achieved at almost no cost.
All Facts (10)
SAFE-QE is a training-free and retriever-agnostic query expansion method, whose full name is Subsequence-Association Filtered, breadth-aware Query Expansion.
50%UnverifiedSAFE-QE scores each candidate expansion term based on its cross-context reproducibility conditioned on the corpus: if a term repeatedly appears in randomized query contexts yet finds no support in the pseudo-relevant document set, it is likely the product of model-driven hallucinated associations and is down-weighted.
50%UnverifiedSAFE-QE models the selection of expansion terms as a constrained coverage maximization problem over the pseudo-relevant set, with the objective of retaining a set of expansion terms that is sufficiently broad, consistent, and stable under resampling.
50%UnverifiedThe study introduces a new query-side faithfulness metric, HETR (Hallucinated Expansion Term Rate), which counts the number of injected expansion terms that have no support in any relevant document.
50%UnverifiedSAFE-QE was evaluated on two benchmark retrieval collections, TREC and BEIR.
50%UnverifiedOn the nDCG@10 ranking metric, SAFE-QE shows no statistically significant difference from generative expansion and corpus-guided expansion.
50%UnverifiedSAFE-QE injects only 4.3 unsupported terms per query, compared to 27.8 for Query2doc and 57.5 for HyDE.
50%UnverifiedThe HyDE method injects approximately 13 times more hallucinated terms per query than SAFE-QE.
50%UnverifiedOn learned sparse retrievers, all expansion methods exhibited performance regression, corroborating concerns that query expansion may harm strong retrievers, while SAFE-QE still maintained a low hallucination rate in this scenario.
50%UnverifiedThe core conclusion of the study is that gains in faithfulness need not come at the cost of ranking quality, and hallucination filtering can be achieved at almost no cost.
50%