Multilingual Dense Retriever
A dense retrieval method that uses a shared encoder to map text in all languages into a single vector space, supporting zero-shot cross-lingual retrieval. Because a shared index is used, representations from different languages may crowd together, which can introduce cross-lingual interference and retrieval bias.
Timeline (last 90 days)
When the same question is posed in different languages, the documents retrieved by a multilingual dense retriever differ enormously.
When each language is evaluated separately, the multilingual dense retriever performs well, and the rankings of candidate documents are consistent across languages.
The source of the irrelevant documents that interfere with retrieval results varies by model: some models are more easily disturbed by irrelevant documents in the query's own language, while others are more easily disturbed by English documents.
Whether language-specific offset directions are stable, whether they can transfer across models, and how the crowding effect evolves as the number of languages in the index increases remain open questions.
All Facts (4)
When the same question is posed in different languages, the documents retrieved by a multilingual dense retriever differ enormously.
50%UnverifiedWhen each language is evaluated separately, the multilingual dense retriever performs well, and the rankings of candidate documents are consistent across languages.
50%UnverifiedThe source of the irrelevant documents that interfere with retrieval results varies by model: some models are more easily disturbed by irrelevant documents in the query's own language, while others are more easily disturbed by English documents.
50%UnverifiedWhether language-specific offset directions are stable, whether they can transfer across models, and how the crowding effect evolves as the number of languages in the index increases remain open questions.
50%