Batch API

An asynchronous inference interface offered by large-model providers. Users package and submit large numbers of requests; the provider completes processing within a committed time limit and bills at roughly a 50% discount from standard pricing. It is suited to latency-insensitive tasks.