LLM Gateway Selection Checklist: Evaluate a Gateway in 10 Minutes

When running Agent tools like Claude Code and Codex, calling the official APIs directly often leads to absurdly high bills. Switching to a third-party LLM Gateway has become a common choice for many developers, but gateway quality varies wildly. How do you quickly filter out the unreliable ones? After evaluating a large number of gateways over the course of a year, one developer distilled a checklist you can run through in 10 minutes.

The core attitude behind this checklist: assume every vendor is exaggerating until the datasheet proves otherwise.

Protocol Coverage: Don't Be Fooled by "OpenAI Compatible"

"OpenAI compatible" is now just table stakes and tells you nothing. What you actually need to check is whether the gateway provides Anthropic's native /v1/messages endpoint.

The reasons are straightforward:

• Calling Claude through the OpenAI format loses prompt cache and thinking capabilities
• For an Agent that re-reads the code repository every turn, losing caching means costs more than double
• The auth headers differ too — Anthropic native uses x-api-key, while OpenAI uses Authorization: Bearer. Those mysterious 401 errors usually originate here

One more thing to confirm: the Responses API on most gateways only supports the GPT family. Don't assume it can handle Claude.

Prompt cache is a server-side caching mechanism Anthropic provides for Claude: when a request prefix (especially system prompts, tool definitions, and long documents) is identical to a previous request, that portion isn't recomputed — it hits the cache directly and is billed at a unit price far below normal input. For Agent-style applications, every conversation turn re-injects the entire code repository or long context into the request. Without caching, you're essentially paying again to process the same content every time, and costs multiply.

Thinking (extended thinking) refers to Claude generating an internal reasoning segment before producing its final answer, used for complex planning and multi-step tasks. These reasoning tokens also incur charges and affect the output structure. OpenAI's Chat Completions protocol has no corresponding field or semantics, so when forwarding through that format, thinking-related parameters and streaming events are often dropped or downgraded. When choosing a gateway, confirming whether it passes through these native Anthropic capabilities matters far more than checking whether it's "OpenAI compatible."

The Pricing Table Must Have a Cache Read Column

If a pricing table only lists input and output prices, that alone is a signal. Agent workloads are overwhelmingly cache reads. A vendor that doesn't reflect cache pricing in its table almost certainly hasn't optimized for caching. Based on the developer's hands-on experience, cache read prices are typically a fraction of normal input prices, and ignoring that column will seriously distort cost estimates.

Cache read and cache write are two distinct billing items. Cache write happens the first time content is stored into the cache and usually costs slightly more than regular input; cache read happens on subsequent cache hits and usually costs about one-tenth of regular input. So in any Agent cost model, the biggest expense usually concentrates in the "cache read" column, not in regular input or output.

A pricing table that only lists input/output leaves users thinking cache hits are free, or unable to determine whether the vendor has actually implemented caching. Conversely, listing input, output, and cache read together lets you both compute real costs and indirectly verify whether the gateway supports cache pass-through. A vendor missing the cache column usually means it forwards requests as-is with no cache optimization, and long-term costs will be significantly higher than the advertised numbers.

Discounts Are Per-Model, and They Change Constantly

"Up to 10% off" (or "up to 90% off") as a single number is meaningless. What you actually need to check:

• Is the per-model pricing table live, or just a marketing screenshot?
• Are discounts calculated in real time at the rate in effect when the request happens, or is it a gimmick?
• Does it display upstream official pricing alongside its own prices so you can verify independently?

If a vendor is unwilling to put upstream pricing next to its own, that behavior itself is the answer.

Credit Terms: Expiration Means You're Lending Them Money Interest-Free

• Do credits expire? Then you're effectively helping them with cash flow
• Is a subscription required? Then you're locked in
• Pay-as-you-go, never expires, top up anytime — this is the baseline, not a selling point

Uptime Should Be Viewed Per-Model

A global "99.9%" masks the real performance of the specific model you're actually using. What you need is a range, and you should look at the worst number, not the best.

SLA (Service Level Agreement) is a formal commitment to availability and response time, usually including metrics like "monthly availability no lower than X%" and corresponding compensation terms. A single global 99.9% figure is often a weighted average blending all models, all regions, and all time periods, which masks the fact that the specific model you call frequently wobbles.

Publishing SLA ranges per model (rather than a single global number) means the vendor is willing to disclose the real performance of different upstream models. During evaluation, focus on the worst number in the range, since it corresponds to the failure scenarios you're most likely to encounter. Also distinguish between "gateway availability" and "upstream model availability" — the former is the gateway's responsibility, while for the latter the gateway can at best fail fast or switch. If a vendor only provides a pretty average number and refuses to break it down, it usually means the long-tail models' availability isn't pretty.

Migration Cost

Changing environment variables is fine. But if it requires installing an SDK, rewriting client code, or forcing you through a signup flow, then unless other aspects are exceptionally strong, just push it to the bottom of the list.

What He Deliberately Left Off the Checklist

A detail worth noting (in the author's own words): he deliberately excluded the advertised discount figures from the checklist. "Up to 90% off" is a marketing construct — it's the maximum on the best day, on the cheapest model. He only looks at the pricing tables for the two or three models he actually runs, and compares them against the corresponding upstream official pricing.

Another issue he only recently started paying attention to: what happens to in-flight requests when upstream degrades? Some gateways fail fast, some silently retry at a higher price tier, and some return a model you didn't ask for. No gateway proactively advertises this — it only surfaces the first time you're on call and notice the bill is off.

Upstream degradation refers to when the underlying model service a gateway depends on experiences rate limiting, timeouts, or partial unavailability, forcing the gateway to decide on its own how to handle in-flight requests. The three common strategies each have costs:

• Fail fast: return an error immediately and let the caller retry, transparent behavior but it interrupts the Agent's continuous execution;
• Silent retry: the gateway retries automatically, possibly falling back to a more expensive or slower tier, so bills rise without the caller knowing;
• Model substitution: return a model that isn't the one you requested, making quality changes hard to detect.

Almost no gateway proactively advertises this behavior. It only surfaces the first time reconciliation looks off, making it the hardest-to-verify and highest long-term-impact blind spot in selection. Keeping a backup key for direct official API access is currently the most common hedge.

The Author's Final Choice and Reservations

The developer ultimately settled on TeamoRouter, because it passed all six checks above: it supports four protocol formats (Anthropic native, OpenAI compatible, Responses API, Gemini native), has a live per-model pricing table with input/output/cache columns, credits never expire, no subscription required, publishes SLA ranges per model rather than one global number, and configuration only requires two environment variables.

He considers the strongest quality signal to be this: TeamoRouter's official documentation explicitly advises against calling Claude through the OpenAI-compatible format, because it's more expensive and performs worse. A platform that proactively publishes a warning steering you away from the simpler integration path is more persuasive than any discount promotion.

Of course, there are reservations: pricing is denominated in USD, so budgeting in other currencies carries exchange-rate risk; entity registration information isn't centrally displayed, so enterprises need to ask before procurement; and he has no low-cost way to test failover behavior, which is why he still keeps a direct API key as a backup path.

Failover Testing: An Unsolved Problem

The author poses a concrete question: how do you test failover behavior during upstream degradation without spending money to set up a load-testing environment? This is both item 7 on his checklist and the one thing he currently has no good way to verify. If you have a different weighting of priorities, or evaluation dimensions he missed, it's worth discussing together.

(Original disclosure: the article contains promotional links.)