Batch APIs from Three Major Vendors at Half Price: Save 50% on Overnight LLM Tasks

Title

Batch APIs from Three Major Vendors at Half Price: Save 50% on Overnight LLM Tasks

Meta Description

OpenAI, Anthropic, and Google Gemini's Batch APIs all offer a 50% discount, ideal for overnight summarization, labeling, embedding, and other non-interactive tasks. This article uses real billing data to show a $1,620 monthly saving calculation and provides a checklist of scenarios suited and unsuited for batching.

Article Content

Most LLM application code assumes a user is staring at a loading spinner: requests are sent synchronously, tasks are split across 50 parallel workers, and the bill is settled at full price. But a huge amount of work actually runs at 3 a.m., with no one waiting for the result. OpenAI, Anthropic, and Google Gemini all offer a 50% discount for these tasks—you just need to tell them you're willing to wait.

Vendor Batch API Pricing

First, two concepts: a token is roughly ¾ of a word, models are billed per million tokens, and input (what you send) is cheaper than output (what the model generates); the Batch API is a separate endpoint where you can upload a large number of requests at once, leave, and download the answers later.

According to the original article (verified October 15, 2026), the batch discounts from the three major vendors are as follows:

• OpenAI: 50% cost discount compared to the synchronous API. Each batch completes within 24 hours, usually faster. Up to 50,000 requests or 200MB per batch, supporting chat, responses, embeddings, moderation, and images.
• Anthropic: All usage (input, output, and everything) is 50% lower than standard API pricing. Up to 100,000 requests or 256MB; officially "most batches complete within 1 hour," with a hard expiration time of 24 hours, and expired requests are not billed.
• Google Gemini: 50% of standard cost, target turnaround time of 24 hours, "much faster in most cases." Tasks expire after 48 hours.

All three vendors offer exactly the same leverage: give up the guarantee of second-level responsiveness in exchange for half the price. That's the whole deal.

Asynchronous batch processing vs. synchronous calls differ not only in price but also in programming model. A synchronous API is request-response: the client makes an HTTP call and blocks waiting, the server returns a result within seconds, suitable for interactive scenarios requiring immediate feedback. Asynchronous batch processing, on the other hand, writes requests to a file or task queue and submits them; the server schedules them in the background, and the client polls for status via a task ID or receives a callback, then downloads the result file when complete. This means error handling is completely different—synchronous calls can be retried immediately, while batch tasks must handle partial success, partial expiration, out-of-order results, and so on. Additionally, batch endpoints typically have independent rate limits and quotas, calculated separately from the synchronous API's requests-per-minute limits, so large batch tasks won't crowd out online interactive traffic.

Prompt caching is another independent cost optimization mechanism: when multiple requests share the same prefix (such as system prompts, long documents, few-shot examples), the server caches that portion of the computation, and subsequent input tokens that hit the cache are billed at a discounted rate, typically 10%–50% of the original price. It operates on a different dimension from batch discounts—caching reduces the input cost of a single request, while batching reduces the overall call price. Anthropic explicitly states the two can be stacked, but cache hit rates within batch tasks cannot be guaranteed, because the batch scheduler may spread requests across different time windows and different machines, and the cached prefix may not be reusable. Therefore, when budgeting costs, you should treat the batch discount as a certainty and consider cache discounts as extra upside.

Dollar Bill Comparison: $1,620 Monthly Savings

A realistic background task: every night, summarize the day's 20,000 support tickets for an internal dashboard. Each ticket is about 1,500 input tokens and 200 output tokens. Using GPT-5.6 as an example, the official price is $2 per million input tokens and $12 per million output tokens.

The original code calculation shows: synchronous calls cost $3,240 per month, batch calls cost $1,620 per month. Same summaries, still on the dashboard before anyone starts work, the only code change is where requests are sent and when results are collected.

Note the impact of parallelism on this number: zero. Fifty workers compress the task from three hours to twenty minutes, but you still pay full price per token. Concurrency buys speed, batching buys price. At 3 a.m., you don't need speed.

Which Tasks Are Suitable for Waiting

A quick test: if the answer only arrives tomorrow morning, will anyone notice? If not, it belongs in a batch task. Typical candidates include:

• Labeling—tagging tickets, classifying comments, flagging spam for training sets.
• Archival summarization—last year's call records, backlogged documents.
• Embeddings—converting text into numerical vectors for search. Re-embedding a knowledge base is a textbook batch task (OpenAI directly supports batch embeddings).
• Nightly reports and evaluations—running test prompts with a new model version.

A reasonable objection is: "24 hours is too unpredictable for overnight tasks." In practice, vendors say most batches return much earlier, and you can always fall back to the synchronous API for requests not returned by the deadline. Only the stragglers pay full price.

Embeddings are the process of mapping text into high-dimensional numerical vectors, where semantically similar texts are closer together in vector space, commonly used for semantic search, clustering, deduplication, and recommendation. Embedding tasks are naturally suited for batching: a single knowledge base rebuild may involve millions of documents, each processed independently and without dependencies, and the results only need to be written to a vector database for subsequent queries, with no real-time requirement. OpenAI's batch endpoint directly supports embedding models, meaning a full library re-embedding can be completed overnight at half price. Note that after an embedding model version upgrade, the vector space changes, and old and new vectors cannot be mixed, so re-embedding often requires a full run—which is exactly one of the scenarios where batch discounts are most valuable.

Scenarios Where Waiting Loses

When the next step depends on the previous answer, batching is the wrong choice.

• Any interactive scenario: chat, autocomplete, customer service bots. The user is the loading spinner.
• Agent chains: an Agent that plans, calls tools, reads results, and then decides is a sequence, not a stack. Ten dependent steps, each "usually under an hour," add up to an afternoon; at a 24-hour worst case, that's ten days. Half price at every step, yet completely useless overall.
• Small batches: saving a few cents on 50 requests a day, while paying the code cost of polling loops, result parsers, and expired request error handling. Not worth it.

The rule is: batch the wide parts of a task (thousands of independent items), keep the narrow parts (dependent steps, waiting humans) synchronous.

What Batch APIs Can't Solve

Batching won't make bad prompts cheaper—a 1,500-token ticket that could be trimmed to 600 is still wasteful at half price. Results may be returned in arbitrary order, and you need to match them using your own IDs. Anthropic notes that batch discounts and prompt caching discounts can be stacked, but cache hits within batches are "best effort," so don't budget based on them.

The original article also candidly admits a gap: the gateway it operates resells the same models at 10–25% below the vendor's official price—GPT-5.6 at $1.70/$10.20 versus the official $2/$12, bringing the above task to $2,754 per month. But that gateway has no batch endpoint and doesn't support embeddings. Gateway discounts and vendor batch discounts are different levers: gateway discounts apply to synchronous interactive traffic; the 50% batch discount applies only to work that can wait, and currently can only be obtained directly from the vendors. For 3 a.m. tasks, vendor batching wins.

Open your billing page and find that task running while everyone sleeps. Then ask it: does it really need a second-level answer?

Gateway (LLM Gateway) is a proxy layer between applications and model vendors that unifies API formats, key management, and billing across multiple vendors, and provides routing, retries, rate limiting, and logging capabilities. Some gateways obtain costs below the official list price through bulk purchasing or subsidies, then resell to users at a discount, creating the 10%–25% price gap mentioned in the article. But gateway discounts come from procurement bargaining, unrelated to the vendor's official batch endpoint—the vendor's 50% batch discount is a pricing strategy for the specific asynchronous batch processing call pattern, and if a gateway doesn't connect to that endpoint, it cannot pass this discount on to users. The applicable conditions of the two discounts are mutually exclusive: gateway discounts cover synchronous traffic, batch discounts cover only asynchronous tasks that can wait.

One-line Summary

All three major vendors' Batch APIs are 50% off; overnight tasks that can wait should be batched first to save half the cost.

Paragraph Summary

The article points out that OpenAI, Anthropic, and Google Gemini's Batch APIs all offer a 50% price discount, at the cost of giving up the guarantee of second-level responsiveness. Taking a task that summarizes 20,000 support tickets every night as an example, synchronous calls cost $3,240 per month, while batch calls cost only $1,620, and the code only needs to change where requests are sent and where results are collected. Concurrency buys speed, batching buys price, and tasks running at 3 a.m. don't need speed. Suitable for batching: labeling, archival summarization, embeddings, and nightly evaluations; unsuitable: interactive scenarios, Agent chains with dependent calls, and small batch tasks. Batch discounts can be stacked with prompt caching, but cache hits are best-effort and should not be included in budgets. Vendor batch discounts can currently only be obtained directly from vendors; gateway discounts do not cover batch endpoints.