Multilingual RAG Is More Than Translation: Entity Consistency and Retrieval Evaluation in E-commerce
The core of multilingual RAG is entity consistency, not translation: separate five concerns and evaluate retrieval and language quality independently.
The article argues that multilingual RAG applications are often mistakenly reduced to translation problems, but in e-commerce the same product has multilingual names, synonyms, and descriptions. The real challenge is maintaining underlying entity consistency while allowing users to ask naturally in their native language. The author proposes five concerns that must be separated: canonical product identity, language-specific knowledge, structured constraints, cross-language aligned evaluation, and independent evaluation of answer quality. The core lesson is "fluency does not equal correctness": a grammatically perfect German answer may point to the wrong product, so retrieval accuracy and language quality must be measured separately; otherwise, generation fluency will mask retrieval failures. In practice, prioritize cross-language entity mapping, language-isolated knowledge, cross-language equivalent evaluation sets, and a separately maintained language-agnostic structured constraint layer.
Why Multilingual RAG Is Not Just Translation
Adding a language to a RAG application is far more than translating queries and documents. In e-commerce, the same product may have different names, synonyms, and descriptions across German, French, and English sites. If the system only translates, a German query may cause the retrieval layer to match a similarly named but actually different product, or return suggestions unsuitable for that language site.
The original article explicitly states: the challenge is maintaining underlying entity consistency while letting customers ask questions naturally in their own language. This means translation only solves the surface-level language problem, while entity alignment, knowledge isolation, and constraint consistency are the real engineering difficulties.
Five Concerns That Must Be Separated
The original article breaks multilingual RAG into five independent but interrelated layers, and this separation is the prerequisite for building a maintainable system.
1. Canonical Product Identity
Names, aliases, and spelling variants in different languages should all resolve to the same underlying product entity. This is the foundation of multilingual retrieval—if entity resolution fails, every downstream step fails. In practice, you need cross-language entity mapping tables and must handle synonyms and regional differences.
Entity Resolution and Canonical IDs In multilingual scenarios, "canonical product identity" usually refers to a language-agnostic unique key (such as SKU, GTIN, or internal product ID), and all language variants point to that key through mapping tables. Entity resolution must handle three types of problems: same name, different entities (e.g., German "Schloss" can mean either lock or castle, requiring category context for disambiguation); different names, same entity (different language names, abbreviations, and misspellings of the same product); and regional variants (the same product may have slight naming differences between the German and Austrian sites). A common engineering approach is to first build within-language alias indexes, then merge language clusters into canonical IDs through cross-language alignment (e.g., based on multilingual embeddings or manual dictionaries), and maintain confidence scores and fallback strategies for the mapping table to avoid cascading retrieval failures caused by incorrect merges.
2. Language-specific Knowledge
Product suggestions and descriptions must match the requested language and store view. The recommendation reasons, compliance notes, and copy style a German user sees should conform to German site norms, not be a direct translation from English. This requires organizing the knowledge base along language dimensions rather than simply sharing one set of content.
3. Structured Constraints
Product attributes and availability rules (such as inventory, delivery range, and size units) must remain consistent regardless of query language. No matter which language the user asks in, the system's judgment of "is this product sellable" should be the same. The structured data layer needs to be independent of the language processing layer.
4. Evaluation
Equivalent questions in different languages should be tested against the same expected result. That is, if a German phrasing and an English phrasing are semantically equivalent, the retrieval results should point to the same product. Evaluation sets need cross-language alignment rather than a separate standard designed for each language.
Aligning Cross-language Evaluation Sets When building cross-language equivalent evaluation sets, a common approach is to anchor on canonical product IDs, annotate each query with its "expected matched entity," and then write query variants in multiple languages for the same intent. This requires distinguishing "semantic equivalence" from "literal translation": literal translation may produce awkward queries that do not match local expression habits, thereby reducing evaluation realism. A more reliable approach is for native speakers or localization teams to rewrite queries according to the same intent, with manual verification that they consistently point to the expected entity. For metrics, report retrieval-layer metrics (such as Recall@k and MRR, based on canonical IDs) separately from generation-layer metrics (fluency and language naturalness), so you can locate whether the problem lies in entity resolution, knowledge isolation, or generation.
5. Answer Quality
Retrieval accuracy and language quality should be evaluated independently. A German answer may be grammatically perfect and fluently expressed but point to the wrong product or catalog entity. If both are scored together, the real problems in the retrieval layer will be masked.
Core Lesson: Fluency Does Not Equal Correctness
The most important reminder in the original article is: a fluent answer does not necessarily mean correct retrieval. A German response may sound flawless yet cite an unsuitable product or the wrong catalog entity. For multilingual RAG systems, consistency is as important as translation quality.
This directly affects evaluation system design: "whether retrieval hits the correct entity" and "whether the generated language is natural" must be measured as two independent metrics; otherwise, fluent generation will mask retrieval failures.
Practical Recommendations
From an engineering implementation perspective, prioritize three things: first, build cross-language product entity mapping to ensure name variants converge to canonical IDs; second, isolate knowledge content by language and store view to avoid contamination from direct translation; third, build cross-language equivalent evaluation sets and use the same expected results to test retrieval performance across languages. The structured constraint layer should remain language-agnostic and be maintained separately.
The difficulty of multilingual RAG is not in the translation model, but in entity consistency, knowledge isolation, and evaluation alignment. Only by handling these layers separately can the system provide stable and reliable results across multiple languages.
One-line Summary
The core of multilingual RAG is entity consistency rather than translation; separate five concerns and evaluate retrieval and language quality independently.
Paragraph Summary
The article points out that multilingual RAG applications are often mistakenly reduced to translation problems, but in e-commerce the same product has multilingual names, synonyms, and descriptions. The real challenge is maintaining underlying entity consistency while allowing users to ask questions naturally in their native language. To this end, the author proposes five concerns that must be separated: canonical product identity (different language variants resolve to the same entity), language-specific knowledge (suggestions and descriptions match the requested language and store view), structured constraints (attributes and availability rules are language-agnostic), cross-language aligned evaluation (equivalent questions target the same expected result), and independent evaluation of answer quality. The core lesson is "fluency does not equal correctness": a grammatically perfect German answer may point to the wrong product, so retrieval accuracy and language quality must be measured separately; otherwise, generation fluency will mask retrieval failures. In practice, prioritize building cross-language entity mapping, isolating knowledge by language, building cross-language equivalent evaluation sets, and separately maintaining a language-agnostic structured constraint layer.
Related articles
AI New Grad Resume Diagnosis: How to Stand Out for ML/CV Roles?
How can AI/ML new grads optimize their resumes to get interviews? Based on a real Reddit case, this article analyzes project selection, depth display, and structure for ML/CV roles, with actionable revision advice.
OpenAI Opens Up Decision Models as Competition with Jev Heats Up
OpenAI makes its decision model widely available, directly challenging first mover Jev. We analyze the impact on the decision model competitive landscape and key considerations for developers and enterprises.
Startup Growth Secrets: Early Sales Are Not the Key
At Disrupt, experts from Lovable, Polska, and Theory Ventures said early startup growth depends not on early sales but on a repeatable sales process, a clear ICP, team development, and data-driven iteration.