Batch API
A discounted interface offered by major AI vendors for offline, non-real-time tasks. Requests are submitted as files and results are returned asynchronously, typically at half the standard price. It is suited to overnight regression runs and full pre-release evaluations.
Timeline (last 90 days)
Parallelism (number of workers) does not affect the token price; concurrency buys speed, batching buys price.
Gateway discounts and vendor batch discounts are different levers: gateway discounts apply to synchronous interactive traffic; the 50% batch discount applies only to asynchronous tasks that can wait, and currently can only be obtained directly from the vendor.
All Facts (2)
Parallelism (number of workers) does not affect the token price; concurrency buys speed, batching buys price.
50%UnverifiedGateway discounts and vendor batch discounts are different levers: gateway discounts apply to synchronous interactive traffic; the 50% batch discount applies only to asynchronous tasks that can wait, and currently can only be obtained directly from the vendor.
50%