Integrating Custom OpenAI-Compatible APIs with Mastra: URL Configuration, the Silent 5-Step Limit, and the Dual-Layer Retry Trap

Mastra is a gradually gaining attention agent and workflow framework in the TypeScript ecosystem. Most official examples use model strings like "openai/gpt-4o", relying on the built-in provider registry. But once you want to point the same agent at your own vLLM, LiteLLM, Ollama /v1 endpoint, or any OpenAI-compatible gateway, this convenient path breaks down.

Mastra solves this with a small configuration object, without needing extra provider packages. However, author Aman Kumar, after testing against a local mock service (capable of returning tool calls, 401, 429 with Retry-After, 503) that records all request bodies, on @mastra/core 1.78.0 + Node 20.19.2, discovered several default behaviors worth knowing in advance for anyone connecting to custom endpoints. Let's break down the findings.

Minimal Working Configuration: model as an object

The simplest approach is to replace the model string with an object containing id, url, and apiKey:

const agent = new Agent({
  id: "support",
  name: "support",
  instructions: "You answer in two sentences.",
  model: {
    id: "apiclaw/YOUR_MODEL_ID",
    url: "https://apiclaw.biz/v1",
    apiKey: process.env.APICLAW_API_KEY,
  },
});

There's also a { providerId, modelId, url, apiKey } form, but tests show the request sent is identical. The request goes to /v1/chat/completions, with user agent ai-sdk/openai-compatible—under the hood it's Vercel AI SDK's OpenAI-compatible provider. There's no GET /v1/models probing, no extra parameters.

Four Pitfalls with id and url

Only the part after the first slash matters

The part of id before the first / is stripped; only what follows is sent as the model field. So id: "apiclaw/deepseek/deepseek-chat" actually sends "model": "deepseek/deepseek-chat". This is good for gateways where the model ID itself contains slashes, but it means you can't omit the prefix. If you don't provide a url, Mastra will look up this prefix in the registry and throw No API URL found for apiclaw/YOUR_MODEL_ID if not found.

OpenAI-compatible endpoints became a universal standard because the OpenAI Chat Completions API has been widely replicated: vLLM, LiteLLM, Ollama, LocalAI, etc., all use /v1/chat/completions as a unified entry point, with field names and semantics aligned with OpenAI. Mastra's underlying Vercel AI SDK provides the @ai-sdk/openai-compatible provider, specifically for connecting to such unofficial implementations. It relies only on the most basic fields like model, messages, tools, and does not probe model lists or force vendor-specific parameters. This is exactly why Mastra's model object can directly point to a custom gateway: it essentially constructs an OpenAI-compatible provider instance.

url must include /v1

Mastra simply appends /chat/completions to the address you provide; it does not add /v1 itself. If you write url: "http://127.0.0.1:8766", the request goes to /chat/completions, which most services will 404. A trailing slash (.../v1/) is harmless; it still ends up at /v1/chat/completions.

apiKey does not fall back to environment variables

If you omit apiKey and only set APICLAW_API_KEY in the environment, the request won't even include an Authorization header, and the server returns 401. Custom providers require you to read the environment variable yourself and pass it in.

api: "responses" switches the endpoint

With this flag, the agent instead POSTs to /v1/responses, using an input array instead of messages. Many OpenAI-compatible services only implement Chat Completions, so unless the service documentation explicitly supports the Responses API, don't enable it. The author's mock returned 404 for this path, and the agent threw AI_APICallError: Unknown path /v1/responses. Custom headers work as expected: headers: { "X-Team": "docs" } appears on every request.

Tool Loop Silently Stops After 5 Steps

The tool calling part is the most time-consuming to debug. A normal tool call takes two steps: the first request includes tools and "tool_choice": "auto", the model returns a tool call, Mastra executes execute and sends a second request with four messages: system, user, assistant tool call, and tool result.

When the mock keeps requesting tools (common with weaker models that can't determine when to stop), the agent makes five requests, executes the tool five times, then returns normally—but text is an empty string and finishReason is "tool-calls". No error, no log. If your code only reads res.text, the user sees a blank answer.

The author suggests two fixes:

// Adjust step budget per call
const res = await agent.generate(prompt, { maxSteps: 3 });

// Treat runs ending with tool calls as incomplete
if (res.finishReason === "tool-calls" || !res.text) {
  // Retry with a stronger model, or return an explicit error
}

With maxSteps: 3, you get exactly three requests and three tool executions. Note that each step resends the full conversation; the fifth request already carries ten messages, which is where token costs start to rise. For per-request billing plans, one problem consumes five quotas.

The Agent tool loop works like this: after the model returns tool_calls, the Agent executes the corresponding tools, appends the results as role: "tool" messages to the conversation, and requests the model again, until the model stops requesting tools or the step limit is reached. Since each round resends the full conversation history, the message count grows linearly with steps, and token consumption rises quickly. maxSteps limits exactly the number of loop iterations. If the model is weak and always thinks it needs to call more tools, the loop truncates at the limit, and the final response contains no user-facing text, only tool call records.

Two Layers of Retry, and They Multiply

This is the most surprising result. Point the agent at a model that always returns 503, and the configuration and request counts are:

• Default configuration: 3 requests, fails in 6 seconds
• maxRetries: 0 on agent: still 3 requests, 6 seconds
• modelSettings: { maxRetries: 0 }: still 3 requests, 6 seconds
• modelSettings: { maxRetries: 5 }: 18 requests, 99 seconds
• errorProcessorDefaults: false on agent: 1 request, 0.1 seconds
• errorProcessors: [] on generate call: 1 request, 0 seconds

The first layer is AI SDK's model call retry, controlled by modelSettings.maxRetries (or maxRetries on an item in a model array). In Mastra it defaults to 0, uses exponential backoff (1, 2, 4, 8, 16 seconds in logs), and respects Retry-After: 2 on 429s.

The second layer is Mastra's default error processor. Regardless of maxRetries, it retries failed generations twice more, with about 3-second intervals. With both layers enabled, the outer three attempts each run six inner attempts, amplifying a single upstream failure into 18 requests and 99 seconds. 401s fail immediately without retry.

The author ultimately chose to keep only one layer and configure explicitly:

const agent = new Agent({
  // ...
  errorProcessorDefaults: false, // Disable outer retry
});

await agent.generate(prompt, { modelSettings: { maxRetries: 2 } }); // 3 attempts, respects Retry-After

Alternatively, keep defaults and set maxRetries to 0. The key is not to raise both layers simultaneously; if you also wrap your own retry around generate(), that's a third layer.

Exponential backoff is a common client retry strategy: the wait time after each failure grows multiplicatively, usually with random jitter to avoid a thundering herd of simultaneous retries. Mastra's inner layer, controlled by AI SDK, shows intervals of 1, 2, 4, 8, 16 seconds in logs; on 429 it prioritizes the server's Retry-After header. Mastra's outer error processor uses a fixed ~3-second interval for two additional attempts. When both layers stack, the total request count equals inner attempts times outer attempts, and wait times accumulate accordingly, so you must explicitly keep only one retry layer.

Failover with a model array

model also accepts an array, switching to the next item when one fails:

model: [
  { model: { id: "apiclaw/YOUR_PRIMARY_MODEL", url, apiKey }, maxRetries: 1 },
  { model: { id: "apiclaw/YOUR_BACKUP_MODEL", url, apiKey }, maxRetries: 0 },
],

The mechanism itself works, but with the default error processor still enabled, the primary model first consumes six requests (two inner attempts times three outer), and only then does the backup model receive its first request, making the whole call take 9 seconds. If you rely on failover, be sure to disable outer retries so switching happens after one or two failures.

Streaming Responses Lack usage

agent.stream() text streaming works fine, but the request lacks stream_options, so servers following OpenAI behavior won't send usage chunks, and await stream.usage returns empty token counts. To record tokens, bypass Mastra's object configuration and use the AI SDK provider to construct a model instance, then pass it to Mastra:

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const apiclaw = createOpenAICompatible({
  name: "apiclaw",
  baseURL: "https://apiclaw.biz/v1",
  apiKey: process.env.APICLAW_API_KEY,
  includeUsage: true,
});

const agent = new Agent({ id: "a", name: "a", instructions: "...", model: apiclaw("YOUR_MODEL_ID") });

This way the request includes "stream_options": {"include_usage": true}, and usage can report input, output, and total tokens. Both generate() paths report usage.

Integration Checklist

Here's a checklist distilled from the above test findings, ready for direct reference:

1. Use { id: "label/YOUR_MODEL_ID", url, apiKey }; only what's after the first slash is sent as the model name.
2. The url must include /v1, and apiKey must be explicitly passed.
3. Unless the server implements /v1/responses, do not set api.
4. Check finishReason after tool runs, and set maxSteps deliberately.
5. Choose only one retry layer: errorProcessorDefaults: false with modelSettings.maxRetries, or keep defaults but set maxRetries to 0.
6. For streaming with token stats, use createOpenAICompatible with includeUsage: true.

These findings were verified on Mastra 1.78.0; defaults may change across versions, so it's worth re-running against a mock service when upgrading.

One-line Summary

A practical, pitfall-avoiding guide to integrating custom OpenAI-compatible endpoints with Mastra.

Paragraph Summary

Aman Kumar systematically tested custom OpenAI-compatible endpoints on Mastra 1.78.0 and found several default behaviors that easily cause pitfalls: the id in the model object only uses what's after the first slash, url must include /v1, apiKey doesn't fall back to environment variables; tool loops silently stop after five steps and return empty text; AI SDK and Mastra's two retry layers multiply, turning a single 503 into 18 requests and 99 seconds; model array failover under default error processors exhausts six requests on the primary model before switching; streaming responses lack stream_options by default, resulting in empty usage. The author provides configuration fixes for each and summarizes them into an integration checklist.