Making a Cheap Model "Think a Little" Made It Slower and Sometimes Silent: A Bug Caused by a Gateway Regex

A Reasoning Meltdown Caused by a Regex

A developer hit a snag while building a model gateway for his desktop app: he wanted a cheap, fast model to "think less," but instead ended up making a model that wasn't reasoning to begin with start reasoning, which slowed down responses and sometimes returned empty strings. I read through the whole debugging process, and the root cause isn't complicated, but the problem it exposes is very typical — many developers use regex matching on model names to determine behavior in multi-model gateways, and that path is bound to cause trouble sooner or later.

First, some background. The desktop app this developer built can call models in three ways: local Ollama, the user's own API Key, or through a small gateway he deployed on a Cloudflare Worker that forwards to OpenRouter. In gateway mode, the client doesn't know which real model is behind it — it only sends a virtual label like cuepreem-cloud-fast, and the gateway decides which model today's "fast" corresponds to. Because of this, all model-specific logic must live in the gateway.

The Token Budget Trap of Reasoning Models

One of those model-specific logic pieces is limiting reasoning length. Some models by default "think" before answering, and that thinking consumes the same max_tokens budget as the final answer. If the budget is tight, the model might spend all its tokens on thinking and hand back an empty string, but the tokens are still billed as usual.

So the gateway checks the model name and caps reasoning models. The author's code looked roughly like this:

function buildUpstreamBody(body, spec, model) {
  return {
    ...body,
    model,
    // The client only sends a virtual label, so only the gateway can decide this
    ...(isReasoningModel(model) ? { reasoning: { effort: 'low' } } : {}),
  };
}

And isReasoningModel was just a regex against the model name:

// Before the fix (trimmed)
return /gpt-oss|deepseek-r1|qwq|magistral|gemini-2\.5/i.test(model);

At the time, he had in mind that the Gemini 2.5 family defaults to thinking, so the gemini-2\.5 match looked correct.

Negative lookahead is a kind of zero-width assertion in regular expressions, written (?!pattern), meaning "what follows the current position must not match pattern," while consuming no characters itself. It's commonly used to match some text while excluding a specific suffix — for example, /gemini-2\.5(?!-flash-lite)/ matches gemini-2.5-pro and gemini-2.5-flash, but not gemini-2.5-flash-lite. Compared to enumerating every desired variant, lookahead is the easiest way to patch a "family-name regex," but it's still fundamentally string matching — if a gemini-2.5-flash-lite-preview or some other non-reasoning-by-default sibling ever appears, the same regex will make the same mistake again. That's why the author stresses it's only a minimal fix, not a best practice.

Reasoning tokens are the tokens consumed by a reasoning model's internal thinking content before it generates the final answer. They share the same max_tokens budget as regular output tokens, but are usually exposed in API responses as separate fields (like reasoning_tokens or completion_tokens_details.reasoning_tokens), and are billed as usual. This means that even if the final visible reply is an empty string, as long as the model "thought," the caller pays for that overhead. Understanding this is key to diagnosing the "slow and empty" symptom in this case: the problem isn't that the model didn't do work, it's that it spent the entire budget somewhere invisible.

Is effort: "low" Off or On?

Later, he switched the fast channel to Gemini 2.5 Flash-Lite. This model is cheap, fast, and doesn't reason unless you explicitly ask it to. But that's exactly where the problem lies: the string google/gemini-2.5-flash-lite contains gemini-2.5, so the gateway sent it reasoning: { effort: "low" }.

The author is clear about this in his reflection: he had always treated effort as a knob that can only be turned down, where low means "do a bit less than usual." For a model that thinks by default, that roughly holds; but for a model that doesn't think by default, there's no "less" to turn down — any effort value equals "start thinking." Low is "some," and "some" is greater than "zero."

To confirm, he ran a small test with max_tokens set to 16. The response reported 12 reasoning tokens, and the visible reply was empty. The budget was essentially all spent on thinking, and what the user saw was blank.

With a looser budget, the answer could still squeeze out after the thinking, making the failure more subtle: the reply just arrived later than expected, and you picked this model precisely because it was fast. But with a tight budget — the kind you set when you want a quick short answer — thinking can eat the whole budget.

That's what makes this problem tricky: there's no error, and the upstream call returns 200. The symptom is just "slower than it should be," and sometimes nothing at all.

In reasoning models from vendors like OpenAI, Google, and Anthropic, reasoning effort (sometimes called thinking budget or reasoning_effort) is a parameter controlling "how much to think," with typical values like low/medium/high or corresponding token numbers. Its semantics are relative adjustment for a model that already reasons by default — like turning a radio's volume from 8 down to 3. But for a model that doesn't reason by default, this knob doesn't exist — once you explicitly pass any effort field in the request, you're telling the model "please start thinking, with thinking level X." So the same parameter means opposite things on models with different default behaviors: on a reasoning model it's "think a bit less," on a non-reasoning model it's "start thinking a bit." When the gateway layer reuses one field across multiple model families, this semantic gap is precisely the most easily overlooked landmine.

The Fix: A Negative Lookahead

The real fix is a negative lookahead:

// After the fix (trimmed)
return /gpt-oss|deepseek-r1|qwq|magistral|gemini-2\.5(?!-flash-lite)/i.test(model);

Plus two tests to pin down the behavior, one ensuring non-reasoning models don't get the field either:

it('never sends effort to Gemini 2.5 Flash-Lite — that would turn on thinking', () => {
  const body = buildUpstreamBody({ messages: [] }, spec, 'google/gemini-2.5-flash-lite');
  expect(body.reasoning).toBeUndefined();
});

it('does not touch non-reasoning models', () => {
  const body = buildUpstreamBody({ messages: [] }, spec, 'meta-llama/llama-3.3-70b-instruct');
  expect(body.reasoning).toBeUndefined();
});

Also, the gateway no longer blindly trusts a 200. If the visible reply is empty after stripping all <think> blocks, the call is considered a failure rather than a "delivered answer."

The Real Lesson: Don't Ask "Does It Support Reasoning?"

The author himself admits that the lookahead is just the minimal fix, not the best one. The real lesson is that he asked the wrong question.

The question isn't "does this model support reasoning" — Flash-Lite does, and that's exactly the trouble. The question is "does this model think on its own when nobody tells it anything." And model families aren't consistent on this: a -lite or -mini variant's default behavior can be the exact opposite of its siblings, and code that regex-matches on family names will quietly sweep it in on the day you switch over.

A better rule: bind cap logic to the exact model IDs you've verified, not to names that look like what you verified. There's also a cheap test that catches this kind of problem in a single call: before a model enters a latency-sensitive channel, call it once with a tiny max_tokens, read the reasoning token count, and assert the visible reply isn't empty.

In API design, capability and default behavior are two independent dimensions: a model "supporting" a feature doesn't mean it's "enabled by default." Most model documentation only states the former, e.g. "supports reasoning" or "supports function calling," without emphasizing whether it's on by default. This is especially dangerous for gateways, since the gateway usually only receives a virtual label from the client and can't let users temporarily override defaults. The safe approach is to maintain at least two boolean or enum flags per model: supportsReasoning and defaultsToReasoning, and let the gateway explicitly decide whether to inject the reasoning field based on policy. Additionally, model IDs (like google/gemini-2.5-flash-lite) are stable contracts of the upstream API, while model family names are just human-readable naming conventions — the two have different semantics, and using family names for branching logic is like treating presentation-layer strings as protocol.

Practical Advice for Multi-Model Gateways

If you run multiple model families behind one endpoint, this kind of pitfall is nearly impossible to avoid entirely, but you can reduce the odds of hitting it.

Model "can reason" and "reasons by default" separately. Many API docs only tell you a model "supports" reasoning, but don't emphasize default behavior. Support doesn't equal on-by-default, and this distinction is critical at the gateway layer. Don't settle for a single boolean — maintain at least two flags per model.

Replace name regexes with explicit ID lists. Regexes are extremely fragile now that model names look increasingly alike; variants like flash, flash-lite, and flash-thinking can have opposite default behaviors with just one extra suffix. Maintaining a verified whitelist of IDs is far safer than guessing by name.

Treat empty replies as failure signals. The gateway shouldn't treat a 200 and an empty string as success. If the visible content is empty after stripping thinking blocks, it should retry, fall back, or error out — not hand blankness to the user.

Run a pre-launch health check with one small call. Before a new model enters a latency-sensitive channel, always do a probe with a tiny max_tokens, read the reasoning token count, and assert the visible reply isn't empty. It's a one-line "smoke test," but it's exactly the kind of silent failure that only shows up in production that it can block.

The question the author poses at the end is also worth every gateway builder's consideration: if you run more than one model family, how do you plan to track which ones think by default? He only has a hand-maintained regex, and after this incident, he doesn't quite trust it anymore.

One-line Summary

Using a model-name regex to decide reasoning mistakenly forced "non-thinking-by-default" Flash-Lite to think, causing slowness and even empty replies.

Paragraph Summary

A developer used regex matching on model names in his self-built multi-model gateway to decide whether to inject reasoning: { effort: "low" } into upstream requests. He thought this was a cap control making reasoning models "think a little less," but when the fast channel switched to google/gemini-2.5-flash-lite, this non-reasoning-by-default model was hit by the family-name regex gemini-2\.5 and was forced to start thinking. Since effort for a non-reasoning model isn't "turn down" but "turn on," the model spent its entire max_tokens budget on internal thinking, causing slower responses and even empty strings when the budget was tight — with tokens billed as usual. The fix was adding the negative lookahead (?!-flash-lite) and supplementary tests, plus treating empty replies after stripping thinking blocks as failures. The author stresses the real lesson: distinguish "supports reasoning" from "reasons by default," and bind logic to verified exact model IDs rather than name regexes.