The Hidden Bias in LLM Routing: Why Non-Standard English Users Get Downgraded Service

When LLM Routing Meets Language Variation: A Overlooked Fairness Problem

Large language model services are increasingly adopting a "tiered routing" strategy: for each user query, the system first uses a low-cost complexity estimator to gauge difficulty, then routes simple queries to smaller, lower-capability models and harder ones to more powerful large models. This approach balances cost and performance and appears both reasonable and efficient.

But a study published on arXiv (arXiv:2609.17542) exposes a hidden flaw in this mechanism: complexity-based routing is not register neutral. The research shows that text written in non-standard English registers — such as African American English (AAE) or English written by second-language learners — is systematically routed to lower-capability model tiers, while semantically equivalent standard English queries receive service from stronger models.

Tiered routing takes several forms in industry. Common approaches include rule-based length/perplexity thresholds, small classifiers that predict query complexity scores, and cascade routing where a weaker model attempts an answer first before deciding whether to escalate. African American English (AAE) is one of the most well-studied English varieties in the United States, with a complete internal grammar system and tens of millions of speakers, yet it has long been underrepresented in NLP training data or discarded as "noise." L2 English covers hundreds of millions of non-native speakers worldwide whose writing tends to be more concise and use shorter sentences, even though their cognitive complexity is no different from native speakers. Both user groups are inherently disadvantaged in AI systems, giving the routing fairness problem significant social implications.

The Root of the Bias: The "Simple" Signal of Input Length

The study's most valuable contribution is precisely identifying the source of the bias. The authors show that the culprit is not the overall complexity estimation mechanism, but a specific and widely used routing signal — input length.

Non-standard registers tend to omit function words such as articles and copulas. This makes sentences expressing the same meaning appear shorter on the surface, causing them to seem "simpler" in length-based complexity estimation. The system then concludes that the query doesn't need a strong model and routes it to a lower tier. The study emphasizes that other complexity signals do not carry this bias — the problem lies squarely with the single dimension of length.

In other words, differences in linguistic expression are mechanically misread as differences in question difficulty. A complex question posed by a second-language learner may be demoted to a "simple" question simply because of its concise grammatical structure.

Function words is a linguistic term for a class of words that carry little lexical meaning themselves and primarily serve to organize grammatical structure, including articles (a/the), prepositions (in/of), copulas (is/are), and conjunctions (and/but). Content words — nouns, verbs, adjectives — are their counterpart, directly carrying conceptual meaning. AAE has a grammatical rule of copula deletion, for example rendering "She is tired" as "She tired." This is not a grammatical error but a legitimate rule of that register. Similarly, L2 learners frequently omit function words during acquisition. Such omissions do not affect core semantics but significantly reduce character and token counts. For routers that use input length as a proxy signal, this directly triggers the automatic judgment chain of "short → simple → route to weaker model," with the system entirely unaware of what is happening.

Large-Scale Validation: From Real Learner Corpora to Controlled Experiments

To validate this phenomenon, the study employs a two-pronged evidence chain. On one hand, the authors demonstrate this routing disparity across 37,704 real learner sentence pairs; on the other, they reproduce the same bias in a controlled parallel corpus under strict semantic equivalence conditions.

This combination of real-world data and controlled experimentation strengthens the credibility of the findings: the bias is not incidental data noise but a structural property of the routing mechanism. When the same meaning is expressed in different registers, routing decisions consistently tilt against users of non-standard registers.

The Deeper Problem: Model Bias Is Worse Than Routing Bias

The study further measures the quality consequences of this routing disparity across a device–edge–cloud model capability ladder, arriving at a conclusion that is surprising yet thought-provoking.

The real harm does not primarily come from the routing decision itself. The authors find that every model tier — including the most advanced cloud-based models — achieves significantly lower accuracy on non-standard register queries. In other words, even when a non-standard register query is correctly routed to the most powerful model, the answer quality it receives is still worse. By contrast, the marginal quality loss attributable to the routing decision itself is not significant on this benchmark.

This finding elevates the problem to a higher level: model bias is a pervasive underlying issue, and complexity routing acts as an amplifier. The routing mechanism does not create bias, but it exposes the users who are already served worst to even weaker models, further compounding the inequity they face.

The fundamental reason models perform worse on non-standard registers lies in the composition of pretraining data. Major large language models are trained predominantly on web text, books, and news, sources that are naturally dominated by standard written English, leaving AAE and L2 English severely underrepresented. This causes higher error rates for non-standard register inputs across semantic understanding, slot filling, and factual question answering — a problem known in academia as "register bias" or "dialect bias." Notably, even instruction tuning or reinforcement learning from human feedback (RLHF) cannot fully eliminate this underlying bias, since alignment data is likewise dominated by standard registers. Therefore, fixing routing strategies alone without simultaneously improving model coverage of non-standard registers treats only the symptoms, not the disease.

Implications for AI Service Design

This research poses a sharp fairness challenge to currently popular model routing architectures. As more AI services adopt tiered routing for cost control, designers need to recognize that:

• Surface-level signals can embed bias. "Cheap" signals like input length are tempting for their efficiency, but they are highly correlated with linguistic background and may inadvertently misinterpret language variation as lower capability requirements.
• Fairness requires a two-pronged approach. Optimizing routing fairness alone is insufficient, because the underlying models themselves have systemic deficiencies in handling non-standard registers.
• Compounded exposure for vulnerable groups. L2 speakers and dialect speakers are often already the groups least well served by AI systems; an unexamined routing mechanism makes their situation worse.

The study's core warning is clear and forceful: complexity routing "compounds models' exposure of already worst-served users." While pursuing cost efficiency, AI service providers must reassess the choice of routing signals and incorporate register fairness into evaluation benchmarks — otherwise the gains from technical optimization may come at the cost of degraded service for marginalized communities.