Qwen Code with Custom OpenAI-Compatible APIs: A Practical Guide

Title

Qwen Code with Custom OpenAI-Compatible APIs: A Practical Guide

Meta Description

Tested Qwen Code with custom OpenAI-compatible APIs: quick start with three environment variables, modelProviders configuration details, the 41,000-character first-request cost, auto-memory doubling requests, and the long hang caused by two layers of retries multiplying—plus how to avoid it.

Article Content

Qwen Code is a terminal coding agent from the Tongyi team. It was originally forked from Gemini CLI but switched to the OpenAI Chat Completions protocol, so it can connect to vLLM, Ollama's /v1 endpoint, LiteLLM, or any hosted gateway. That gives developers a lot of freedom—but the default configuration has quite a few under-the-radar details. One developer used a local mock server (capable of simulating 401, 429, and 503) to test every network request made by Qwen Code 0.25.2. Below are the test results and the configuration pitfalls worth noting.

Quick Setup: Three Environment Variables

The lightest approach is three environment variables: OPENAI_API_KEY, OPENAI_BASE_URL, and OPENAI_MODEL, then simply run qwen -p "...". No config file is needed—Qwen Code will choose the OpenAI-compatible path and send requests with Bearer authentication, stream: true, and a tool list.

The tests surfaced two easy-to-hit pitfalls.

The base URL must keep /v1. Qwen Code uses the official OpenAI Node SDK, which appends /chat/completions to whatever address you provide. If OPENAI_BASE_URL=https://host (without /v1), the request goes to /chat/completions, and most gateways will return a 404.

max_tokens defaults to 32000. Every request made with only environment variables carries "max_tokens": 32000. Some servers reject values above the model's output limit, while others reserve quota based on that amount. If your model has a lower output limit, you need to set it manually.

Behavior on an incorrect key is clean: a single request, a [API Error: 401 Invalid API key] error, exit code 1, no retry.

The OpenAI Chat Completions protocol is the de facto standard for LLM inference access today, and the major self-hosted frameworks all implement it: vLLM for high-performance production inference, Ollama for lightweight local deployment, and LiteLLM as a unified multi-vendor proxy with key management. What they share is exposing a /v1/chat/completions endpoint and accepting Bearer authentication, so any client built on this protocol can switch backends seamlessly. The reason Qwen Code rewrote its network layer after forking from Gemini CLI was precisely to abandon Gemini's proprietary protocol and gain this swapability. Understanding this is key: all the paths, authentication, and retry behavior described below come from the official OpenAI Node SDK's default implementation, not conventions invented by Qwen Code.

Environment variable precedence is also worth noting. Qwen Code supports both generic variables like OPENAI_API_KEY and gateway addresses like OPENAI_BASE_URL; together they form what the docs call a temporary "runtime model." This approach is good for quick validation but doesn't support multiple providers coexisting, timeouts, retry counts, or other fine-grained control, which is why the official docs treat settings.json as the everyday solution. When both exist, you need to be clear about which side ultimately determines request parameters—otherwise it's easy to change a setting and see no effect.

Recommended for Daily Use: modelProviders in settings.json

Environment variables create what the docs call a "runtime model." For daily use, configuring a provider entry in ~/.qwen/settings.json is recommended: the structure is clearer and easier to manage. Key fields include id, name, envKey (the name of the environment variable, not the key itself), baseUrl, and a nested generationConfig (with timeout, maxRetries, contextWindowSize, and samplingParams). You also need to set security.auth.selectedType to openai and point the top-level model.name at the same model ID.

The key under modelProviders must be a real protocol name (openai, anthropic, gemini, vertex-ai). Inventing a key like "apiclaw" gets skipped with a warning unless you map it in a top-level providerProtocol block.

The key referenced by envKey can live in three places: a project-level .env file (works), ~/.qwen/.env (works), or neither (Qwen Code stops before sending, reporting a missing API key). If you use a project .env, be sure to add it to .gitignore—the coding agent runs right in your repo, and the risk of accidentally committing a key is real.

Once the config takes effect, the request body becomes temperature: 0.2 and max_tokens: 4096, and the 32000 default disappears.

Settings that silently fail

The tests also revealed a subtle behavior: a top-level model.generationConfig.samplingParams.top_p in the same file is never sent. This matches the documented rule—once a modelProviders entry is selected, its generationConfig wholesale replaces the top-level config, so top-level settings of the same name are ignored for that model. samplingParams, customHeaders, and extra_body are all whole-object replacements rather than merges. If a setting seems not to be taking effect, first check whether it's at the wrong level.

Whole-object replacement of generationConfig is the core semantic of Qwen Code's configuration model. It bundles sampling parameters (temperature, top_p, max_tokens), timeouts, retries, and context window into a single unit that takes effect as a whole when a provider entry is selected, without merging with same-named fields at the top level or in other providers. This is the opposite of how most tools do "nearest-key overrides," so when migrating a parameter from the top level into a provider, it's easy to assume you've set it when you haven't. The way to troubleshoot is to print the final request body and check, rather than repeatedly editing the config file.

envKey and key management are designed to avoid writing keys into JSON. envKey stores only the variable name; the actual value is injected by a .env file or the shell environment. A project-level .env gets read by the coding agent working inside the repo—once mistakenly committed, the key leaks along with the code; putting it in ~/.qwen/.env lets you reuse it across projects without committing it. In multi-environment setups, you can also override it per session with shell variables, avoiding long-lived credentials scattered on disk.

The Real Cost of a Single "Say hi"

Running qwen -p "Say hi" non-interactively sends two requests, not one.

The first is the answer request. The system prompt is 30,553 characters, plus a 10,963-character block listing available skills and memory paths, plus six tool definitions (glob, grep_search, read_file, skill, tool_call, tool_search). That means roughly 41,500 characters before you even type your prompt—about 10k input tokens at a rough four characters per token.

The second request is sent after the answer is printed. Its system prompt begins with "You are now acting as the managed memory extraction subagent," replays the conversation, and provides write_file and edit tools to update memory files under ~/.qwen/. This is the managed auto-memory feature, enabled by default.

For token-billed plans, this baseline has a noticeable impact on short prompts; for plans billed per request or rate-limited per minute, the memory round doubles a one-shot run. You can disable it with "memory": {"enableManagedAutoMemory": false, "enableManagedAutoDream": false}, after which qwen -p "Say hi" sends only one request.

Retries: Two Layers Multiplying

This is the part most worth configuring before deploying to CI or a slow endpoint.

Fast-recovering 429s are handled well. When the mock returned 429 with Retry-After: 2 twice, Qwen Code waited about 2 seconds each time, and the third request succeeded.

A persistently down server is another matter. With defaults, when the mock always returns 503: the OpenAI SDK retries three times per attempt with backoff of roughly 0.4s, 0.9s, 1.8s, so requests arrive in groups of four; Qwen Code's own retry loop repeats these groups with growing intervals (about 2, 3, 7, 14, 30, 30 seconds), totaling 28 requests over about 107 seconds across seven groups; then the CLI prints Retrying in 60s (attempt 1/10) and starts a new round. In the test, it was manually stopped at 200 seconds and 44 requests, still retrying.

Setting maxRetries to 0 in generationConfig removes the SDK layer. The same down endpoint then received only 7 requests (about 74 seconds), and the CLI ended with [API Error: 503 upstream busy] and exit code 1. For scripts and CI jobs, set maxRetries to 0 or 1 and wrap your own timeout outside, so a vendor failure fails within a minute instead of hanging.

Exponential backoff and Retry-After are the foundation for understanding retry amplification. The OpenAI SDK retries on 429s, 5xx, and network errors by default, with exponentially growing intervals plus jitter, while respecting the server's Retry-After header. Qwen Code wraps another retry loop of its own around the SDK to handle session-level failures. The two layers count independently, so the actual number of requests is roughly their product: the SDK's 3 attempts times the CLI's 7 rounds, times any multiple requests per round, can rack up dozens of requests within minutes. For per-request-billed or strictly rate-limited endpoints, this both wastes quota and may trigger bans. Lowering maxRetries only disables the SDK layer; the outer loop keeps going, so in CI always also set a process-level timeout as a fallback.

One Reassuring Safeguard

The mock was set to repeatedly request the same tool call with unchanged arguments. Qwen Code stopped after three requests: Loop detection halted the run (global_tool_call_duplicate...). This is an always-on safeguard, especially useful for small or cheap models that sometimes get stuck in repetitive steps.

Loop detection targets a common failure mode in agents: the model repeatedly calls the same tool with identical arguments, consuming tokens without making progress. Qwen Code aborts after three identical consecutive calls with global_tool_call_duplicate, an always-on safeguard requiring no configuration. It's especially valuable for weaker or more heavily quantized models, which are more prone to repetitive steps. Note that legitimate batch-style repeated calls may also be misjudged; in that case, adjust the prompt or split the task rather than relying on disabling detection to work around it.

Configuration Checklist

• OPENAI_BASE_URL (or baseUrl) ends with /v1, not /chat/completions
• Use modelProviders with envKey, and put the key in ~/.qwen/.env or a git-ignored project .env
• Set samplingParams.max_tokens to the model's true output limit, not the 32000 default
• Put all settings for a provider model in its own generationConfig; top-level config is ignored for that model
• Disable managed auto-memory for per-request billing or one-shot -p tasks
• In CI, set maxRetries low and add an outer timeout

One note: the original author is a developer of the OpenAI-compatible gateway APIClaw, and the examples point to their service, so weigh the opinions accordingly; however, the behaviors described apply to any compatible endpoint.

One-line Summary

Tested configuration pitfalls and request costs when connecting Qwen Code to OpenAI-compatible endpoints.

Paragraph Summary

Using a local mock server capable of simulating 401, 429, and 503, the author tested Qwen Code 0.25.2's request-by-request behavior against OpenAI-compatible endpoints. Findings include: the base URL must keep /v1, or the SDK-appended path yields 404s on most gateways; the default max_tokens is 32000, easily exceeding model limits or reserving quota; a provider entry's generationConfig wholesale replaces top-level config, silently disabling top-level settings; one qwen -p invocation sends two requests with a system prompt of about 41,500 characters, with managed auto-memory on by default; the SDK's and CLI's two retry layers multiply, so a down endpoint can generate 40+ requests within minutes. The article gives a configuration checklist and notes that loop detection is an always-on safeguard.