SAFE-QE: Taming Hallucinated Terms in LLM Query Expansion via Subsequence-Association Filtering

The Hidden Risk of LLM-Based Query Expansion: Hallucinated Terms Polluting Retrieval

Large language models (LLMs) have become the go-to source for generating query expansion terms in information retrieval. They can produce semantically relevant expansion vocabulary from a raw query, helping retrieval systems cover a broader semantic space. But this approach carries a long-underestimated problem — hallucination.

When generating expansion terms, LLMs inject entities that don't exist or words unrelated to the topic. These terms not only add ranking noise but can actively hurt retrieval performance when paired with already strong retrievers. In other words, the gains from query expansion may be cancelled out by the negative effects of hallucinated terms.

Existing mitigation strategies — such as corpus grounding, mutual verification, and preference alignment — mostly treat hallucinated terms as a generic "noise" category and rely on consistency-based heuristics to filter them out. This approach has two clear weaknesses: it lacks a mechanistic explanation for why a term appears, and it offers no principled criterion for deciding how many expansion terms to keep.

Background: Query Expansion is a classic information retrieval technique that supplements the user's original query with related vocabulary to improve recall. Traditional approaches fall into two main categories: manual expansion using thesauri or synonym dictionaries, and automatic expansion via Pseudo-Relevance Feedback (PRF) — which retrieves top-ranked documents using the original query and extracts high-frequency terms to feed back into the query. LLMs open up a third path: having the model directly generate semantically related expansion terms or pseudo-documents from the query. Representative methods include Query2doc (generating short passages) and HyDE (generating hypothetical documents). The strength of these approaches lies in capturing deep semantic relationships, but since the model has no knowledge of the actual corpus contents, the generated vocabulary may have no corresponding documents in the corpus — creating so-called "hallucinated expansion terms" that are particularly harmful in closed-domain or specialized retrieval settings.

The Core Idea Behind SAFE-QE: Starting from the Hallucination Mechanism

This paper proposes SAFE-QE (Subsequence-Association Filtered, breadth-aware Query Expansion). Its two defining characteristics are training-free and retriever agnostic, meaning it requires no additional training overhead and can be paired with any retrieval system.

Identifying Hallucinations via Cross-Context Reproducibility

The theoretical foundation of SAFE-QE is a "subsequence-association" account of hallucination. The method scores each candidate expansion term based on its cross-context reproducibility conditioned on the corpus.

The specific mechanism works as follows: if a term repeatedly appears across randomized query contexts but finds no support in the pseudo-relevant document set, it is likely a product of dominant hallucination associations within the model. Such terms are flagged and down-weighted. This criterion offers a more mechanistically grounded explanation than simple consistency voting — it distinguishes between "terms the model habitually produces" and "terms genuinely relevant to the corpus."

Background: The Pseudo-Relevant Set is a key concept in the PRF framework, referring to the collection of top-ranked documents retrieved using the original query — typically the top 5–20 documents. These documents are assumed to be "probably relevant" to the query and can therefore serve as corpus anchors for verifying the authenticity of expansion terms. SAFE-QE uses this set as a grounding reference: if a candidate expansion term appears sufficiently in the pseudo-relevant documents, it is considered corpus-supported; if it is only produced habitually by the model due to co-occurrence patterns in its training data but disconnected from the current corpus, it is classified as a hallucinated term. The core insight of "subsequence association" is that hallucinated terms are often triggered by fixed patterns from certain local token sequences in the query, independent of specific context — by randomly shuffling query context and observing which terms are still stably generated, these "pattern-driven" hallucinations can be precisely located.

Framing Term Selection as Constrained Coverage Maximization

The research also draws on a characterization of language generation as a trade-off between consistency (avoiding hallucination) and breadth (coverage).

From this perspective, SAFE-QE frames expansion term selection as a "constrained coverage maximization" problem over the pseudo-relevant set. The objective is to retain a set of expansion terms that is sufficiently broad, remains consistent, and is stable under resampling. This transforms "how many terms to keep" from an arbitrary hyperparameter into an optimizable objective.

A New Metric — HETR: Quantifying Query-Level Faithfulness

To evaluate performance, the paper introduces a new query-level faithfulness metric: the Hallucinated Expansion Term Rate (HETR). It counts the number of injected expansion terms that have no support from any relevant document.

This metric fills a gap: previous evaluations of query expansion focused almost entirely on final ranking quality (e.g., nDCG) while ignoring how many "unsupported" terms were injected along the way. HETR brings faithfulness directly to the query level, making the hallucination problem measurable and comparable.

Experimental Results: Higher Faithfulness at Zero Ranking Cost

The study was evaluated on two benchmark retrieval collections — TREC and BEIR — with compelling results.

On nDCG@10, the core ranking metric, SAFE-QE shows no statistically significant difference compared to generative expansion and corpus-guided expansion methods — meaning no loss in ranking quality. But in terms of unsupported terms injected, the gap is striking:

• SAFE-QE: only 4.3 unsupported terms per query
• Query2doc: 27.8
• HyDE: 57.5

This means popular methods like HyDE inject roughly 13× more hallucinated terms per query than SAFE-QE. Additionally, all expansion methods showed performance regression on a strong learned sparse retriever — which confirms the earlier concern that expansion can hurt strong retrievers. SAFE-QE maintained a low hallucination rate even in this scenario.

The core finding can be summarized in one sentence: gains in faithfulness do not require sacrificing ranking quality. SAFE-QE demonstrates — through a training-free, retriever-agnostic framework — that hallucination filtering can be achieved at virtually no cost.

Background: TREC (Text REtrieval Conference) is an information retrieval evaluation conference hosted by the National Institute of Standards and Technology (NIST). Its benchmark datasets have long been considered authoritative testbeds for retrieval systems, spanning news, web pages, dialogue, and other domains. BEIR (Benchmarking IR) is a zero-shot retrieval evaluation suite covering 18 heterogeneous datasets across biomedical, financial, legal, and other domains, specifically designed to assess the generalization ability of retrieval methods. nDCG@10 (Normalized Discounted Cumulative Gain at 10) is the mainstream metric for measuring ranking quality, considering both relevance and rank position, with a maximum score of 1.0. Learned sparse retrievers such as SPLADE output sparse weight vectors directly in the vocabulary space through training, combining the interpretability of keyword matching with the expressive power of semantic matching. Because they already handle vocabulary variation well, additional query expansion tends to introduce noise and cause performance regression.

Implications for Retrieval System Design

The value of this work extends beyond a single method. It offers a new lens for thinking about LLM-based query expansion: rather than cleaning up hallucinations post-hoc with heuristic rules, it's better to understand why hallucinations occur from a generative mechanism standpoint, then apply coverage maximization principles to decide which terms to keep.

For engineering teams building RAG systems or search engines, SAFE-QE's training-free and retriever-agnostic properties mean a low barrier to adoption. The HETR metric also gives teams a new evaluation dimension — beyond optimizing recall and ranking, it's worth tracking how much noise is being injected during expansion. As LLMs become more deeply embedded in retrieval pipelines, this kind of mechanistic understanding and quantifiable control of hallucination is becoming increasingly important.