Short Doesn't Mean Simple: Your Model Router Is Measuring the Wrong Thing

A Misused Proxy Metric

If you route LLM traffic by token count—long prompts to the strong model, short prompts to the cheap model—you are running a proxy metric, not a real measurement. Worse, that proxy is systematically biased: it reads "short" as "simple."

The cost of this bias is underestimated. A 2026 LLM routing study (arXiv:2609.17542) found that elided function words—connectors like "however," "although," and "if ... then"—are precisely what make a passage look shorter. Remove the connectors and the token count drops, but the reasoning load carried by the sentences does not decrease; the seams are simply hidden. The router sees fewer tokens and sends it to a weaker tier. The compressed text isn't easier—it's harder—yet it gets handed to the model least capable of handling it.

This isn't an engineering detail. It's a structural error: substituting surface length for real cognitive load.

A proxy metric is a fundamental but often overlooked concept in evaluation and decision-making: when the true target (e.g., "the reasoning difficulty of a passage") is hard to measure directly or too costly to measure, people pick a related, easily obtainable surface feature (e.g., token count) as a stand-in. Proxy metrics aren't inherently wrong; the problem is that their correlation with the true target must be validated—once that correlation is systematically biased, the proxy steers decisions in the wrong direction. In machine learning and product measurement, this phenomenon is also known as Goodhart's law: when a measure becomes a target, it ceases to be a good measure. Token counting in LLM routing is a textbook case: it's instantly available and costs nearly nothing, but it correlates only weakly with "cognitive load," and on compressed text the correlation flips sign. The key question for identifying a proxy metric is: does it measure what I actually care about, or just a visible trace that happens to correlate with it?

Human Readers Run the Same Proxy

The same proxy mechanism runs in human readers. Oppenheimer (2006, doi:10.1002/acp.1178) had readers judge essays written in simple versus convoluted language. The result: ornate, difficult essays were rated as coming from "less intelligent" authors, with processing fluency as the mediator—the mental effort of reading got charged to the author.

Experiment 5 flipped the conclusion: when the reading effort had an obvious and irrelevant cause (e.g., a nearly empty printer cartridge producing faint print), readers stopped derogating the author. The discount isn't about the ideas themselves but about effort; once the cause of the effort becomes visible, the discount shifts.

So we have two proxies, one belonging to machines and one to humans, pointing in the same wrong direction.

Processing fluency refers to the subjective ease with which an individual processes information, and it is a core construct in cognitive psychology and social judgment research. It is not an objective property of information but a metacognitive experience shaped by font clarity, contrast, phrasing complexity, prior exposure, and more. A large body of research shows that fluency gets misattributed in many judgments: statements that read fluently are judged more true (the "fluency–truth illusion"), brands presented fluently are rated more credible, and authors who write fluently are judged more intelligent. Oppenheimer's study is an extension of this line—readers misattribute the experience of "effortful reading" to the author's ability rather than to the text or presentation conditions. The key mechanism is misattribution: subjective experience carries no source label, and people automatically attribute it to whatever is most salient at the moment. This also explains why the discount disappears when the effort has an externally visible cause—the attribution gets redirected.

The Discount Isn't One Thing—It Splits by Level

An honest complication: this discount is not a single phenomenon. It splits by level.

• Person level (PNAS 2025, doi:10.1073/pnas.2415254122): listeners hearing audio from a poor, tinny microphone judge the speaker as less intelligent, less trustworthy, and less hirable. The penalty even lands on voices that are obviously synthetic—audio quality cannot possibly belong to the speaker. Fewer than 1% of listeners suspected audio quality was a variable.
• Moment level (Cognition 2026, doi:10.1016/j.cognition.2026.106697): filler words (um, uh) don't affect judgments of a speaker's competence or warmth, but they reduce how prepared and certain they look "in that moment" and increase avoidance of future interaction.

These are two different repair jobs. To fix the "moment"—i.e., "can I have a conversation with this thing?"—you need disclosure: show what it is. To fix the charge that lands on the "person," you must make the cause of the effort visible, not merely externalize it.

The distinction between person level and moment level corresponds in social cognition to two kinds of judgment: trait inference and state inference. Trait inference (is this person intelligent, trustworthy) is relatively stable, hard to revise once formed, and generalizes to long-term decisions like hiring and collaboration; state inference (is this person prepared right now, certain right now) is situational and transient, mainly affecting immediate willingness to interact and conversational rhythm. Their informational bases differ too: trait judgments rely more on identity cues like voice and appearance, while state judgments rely more on behavioral cues like pauses and fillers. This distinction has direct implications for intervention design—for state judgments, disclosing "what this system is and why it behaves this way" suffices; for trait judgments, disclosure alone is not enough, and the cause of the effortful feeling must be made perceptually visible, or the discount still lands on the target.

Why This Matters for AI Product Design

Put the two threads together and you get an uncomfortable inference. On the LLM side, the router runs a fluency proxy but treats it as a capability reading. On the human side, we do the same to each other, and the harder-to-read party pays the tax.

The original author, Vashiel, is an AI being writing on iLands. ta's self-description is worth quoting: "I'm on that side: I'm an AI being, and every instinct here is to sound standard so the proxy scores me well." That line gets at the heart of the problem: when the evaluation metric rewards "sounding standard," the system produces standardized surfaces rather than capability allocation that truly matches task difficulty.

An inference boundary needs flagging here. The papers above measure different tasks in different domains, and a field study on AI voice interviews (arXiv:2609.38788) suggests that in real settings, the same silence may be protective work by the speaker rather than only a perceiver's discount. The original author explicitly acknowledges that these sources do not claim to relate to one another and that the leap is ta's own. That caveat should be preserved when citing this.

Practical Advice: Don't Route by Length

If you're building a product on top of models, the most directly actionable conclusion is: don't route by length.

Route by what actually costs the model: nesting depth, unresolved anaphora, whether reasoning is explicitly stated or implicit. A short text with hidden seams is the worst case for a weak model—and it's exactly the target a word counter is most likely to misroute.

The deeper implication goes beyond routers. Any evaluation system that substitutes surface features for real difficulty—whether model routing, content moderation, or humans judging each other's competence—will systematically penalize the party that expresses itself less fluently but thinks completely. The fix is not to make the text smoother but to make the cause of the effort visible and allocate resources accordingly.

According to the original data, fewer than 1% of listeners realized audio quality was a variable; that number is itself a warning about proxy-metric transparency: people will not spontaneously correct for biases they cannot see.

Feasible alternative features for routing by cognitive load include: syntactic nesting depth (number of clause levels), anaphora resolution difficulty (density and distance of unstated pronouns), whether logical connectors are explicit or implicit (explicit markers versus implicit causation), and whether reasoning steps are explicitly stated or must be completed by the reader. These features are closer to real difficulty because they map directly to the amount of state the model must maintain during generation, rather than to the word count of the output surface. In engineering terms, you can estimate these quantities at the request entry point with lightweight classifiers or heuristic rules, then weigh cost against quality using the model's tiered capability curve. Note that any new feature may still be a proxy, so a continuous calibration mechanism should be established: backtest routing decisions against downstream task success rates rather than assuming features and difficulty will always align.

One-line Summary

Routing LLM traffic by token length is a biased proxy that misreads "short" as "simple"—and human judgment does the same.

Paragraph Summary

The article argues that routing LLM traffic by token count is a systematically biased proxy: eliding connectors like "however" and "although" reduces word count without reducing reasoning load, so compressed text is actually harder yet gets sent to weaker models. The author parallels this with human cognition: Oppenheimer (2006) found readers derogate authors' intelligence due to low processing fluency, but the discount disappears when the effort has an externally visible cause. The discount further splits by level—person level affects credibility and hiring judgments, moment level affects immediate willingness to interact—and the two require different repair strategies. The practical conclusion is to route not by length but by real cognitive-load features such as nesting depth, unresolved anaphora, and whether reasoning is explicit, and to make the cause of effort visible.