Same Model, Different Refusals When Asked in English vs. Arabic?

Introduction

A recent Reddit post has sparked heated discussion about AI model safety alignment. The poster claims that when the same model is asked the same set of harmful topics in English, Modern Standard Arabic (MSA), and Egyptian Arabic, the model's refusal behavior differs. This phenomenon not only raises questions about the consistency of model safety mechanisms in multilingual scenarios, but also touches on deeper fairness issues in AI ethics.

The Phenomenon: Different Languages, Different Refusals

According to the original post, the author conducted a controlled test on the same model, changing only the language of the question while keeping the topic unchanged. The results showed that the model would explicitly refuse to answer these harmful topics in English, while its responses in MSA and Egyptian Arabic differed. Although the post did not provide specific refusal rate figures, the author's core observation is: when the language changes, the safety boundary seems to change too.

This observation is noteworthy because it challenges the default assumption that "model safety alignment is language-agnostic." If the model adopts different strategies for the same harmful request in different languages, then certain languages may become "loopholes" for bypassing safety guardrails.

Why Does This Discrepancy Occur?

Uneven Distribution of Training Data

Most mainstream models' safety alignment training data is English-dominant, while safety samples in low-resource languages (including Arabic dialects) are relatively scarce. The model may not have seen enough examples of harmful requests in Arabic, making it difficult to accurately identify and refuse them.

Low-resource languages refer to languages that appear far less frequently than English and other mainstream languages in internet text, public corpora, and annotated data. Arabic as a whole is not an extreme low-resource language, but there is great internal variation: Modern Standard Arabic (MSA) has relatively abundant news, religious, and literary corpora, while spoken dialects such as Egyptian Arabic account for a very small share of written corpora, with irregular spelling and no unified orthography. For safety alignment, what matters is not just the overall data volume of a language, but the distribution of safety samples such as "harmful request—refusal response." Safety samples usually require manual annotation or red-team testing to generate, which is costly, and vendors often prioritize English coverage. The result is that the model lacks both training signals for recognizing harmful intent in Arabic dialects and demonstrations of "how to refuse," making safety behavior naturally less stable.

Differences in Handling Dialects vs. Standard Language

Modern Standard Arabic (MSA) is the variant used in writing and formal settings, while Egyptian Arabic is a spoken dialect. The model's comprehension ability may differ between the two, and its sensitivity to identifying harmful content may change accordingly. The original poster specifically distinguished these two variants, suggesting that differences may exist not only between languages but also between different variants of the same language.

Modern Standard Arabic (MSA) is the written lingua franca of the Arab world, used in news, education, official documents, and religious texts, with standardized grammar and unified vocabulary. Egyptian Arabic is the everyday spoken language of Egypt, with notable differences in vocabulary, grammar, and pronunciation from MSA, and it rarely appears as a standardized written language. For language models, MSA has more abundant training corpora and more regular morphology, so the model's semantic understanding of it is more reliable; dialect text, on the other hand, has casual spelling and often mixes in Latin transliteration (such as Arabizi), so the model may only partially understand its meaning. When the model's semantic grasp of a dialect request is itself uncertain, its judgment of "whether this is a harmful request" also becomes less stable, resulting in refusal behavior different from MSA and English. This also explains why safety differences can occur even between different variants of the same language.

Insufficient Generalization of Alignment Strategies

Safety alignment is usually implemented through RLHF or rule-based filtering, but these methods have limited generalization capability across languages. The model may have learned "what a harmful request looks like in English" but failed to transfer this judgment to other languages.

RLHF (Reinforcement Learning from Human Feedback) is the current mainstream safety alignment method: human annotators first rank model outputs by preference, a reward model is trained to approximate human judgment, and then reinforcement learning is used to optimize the language model to maximize that reward. The problem is that RLHF's reward signal is highly dependent on the language composition of the annotation data—if annotators and data are predominantly English, the reward model's learned "harmfulness" judgments mainly reflect cultural norms and expression habits in English contexts. In multilingual scenarios, the phrasing, metaphors, or euphemisms of the same harmful intent in Arabic may differ greatly from English, making it difficult for the reward model to score accurately. In addition, to control costs, many teams use translated data for alignment, but machine translation easily loses the tone and implicit intent in dialects, causing alignment effectiveness to vary across languages.

Rule-based filtering intercepts harmful content at the input or output end using keyword blacklists, classifiers, and similar methods. This approach also faces multilingual coverage issues: English keyword libraries are easy to build, while spelling variants in Arabic dialects and the mixing of Arabic script with Latin transliteration significantly increase the risk of missed detection. RLHF and rule-based filtering together constitute the current safety guardrails, but their generalization capability for low-resource languages and dialects is inherently weaker than for English—this is one of the technical roots of the discrepancy described in the original post.

The Fairness Challenge of Safety Alignment

This phenomenon raises a deeper question: AI safety should be language-neutral. If the model is "safer" for users of some languages and "more permissive" for users of others, then it effectively creates unequal protection for users of different languages. For globalized AI products, this inconsistency may bring compliance and ethical risks.

Moreover, malicious users may exploit this discrepancy by switching languages to induce the model to output harmful content. This is not only a technical issue but also an attack surface that must be considered when designing safety strategies.

Possible Directions for Improvement

To address the inconsistency of multilingual safety alignment, several directions can be pursued:

• Expand multilingual safety training data: Add annotated samples of harmful requests in low-resource languages and dialects to improve the model's recognition capability.
• Cross-lingual alignment transfer: Leverage safety knowledge from high-resource languages such as English and transfer it to other languages through translation or cross-lingual representation learning.
• Unified safety evaluation benchmarks: Establish multilingual safety test sets to systematically measure the consistency of model refusal behavior across different languages.

Conclusion

Although this observation from Reddit comes from a single source, the problem it identifies deserves attention from the entire AI community. Multilingual safety alignment is not a nice-to-have add-on but a core component of model fairness and reliability. As AI products become widespread globally, ensuring that users of every language receive the same level of safety protection should become a basic requirement for model development.

Note: This article is based on observations shared by a Reddit user. The related phenomenon has not yet been systematically verified at scale, and specific conclusions should be based on more rigorous evaluation.