Batch APIs: How Much You Save by Agreeing to Wait

The biggest discount available on an AI bill is not a better model or a shorter prompt. It is agreeing to wait. OpenAI, Anthropic and Google all run batch endpoints: you submit a file of requests instead of calling one at a time, results come back within a stated window rather than in a second, and both input and output are billed at a discount. There is no negotiation, no minimum spend, and no contract. You give up latency and you get a lower rate.

It is also the discount teams most often leave unclaimed, because claiming it means noticing that a large share of what you send in real time never needed to be real time.

Check the rate yourself before you count on it

Start with the honest part. Our archive records three numbers per model every day: standard input, standard output, and the cache-read rate where a provider publishes one. It does not record batch rates, so we do not price batch anywhere on this site and no figure here is a batch figure. The discount is roughly half on the providers that publish one, but that is their published number and not our observation, so treat it as something to verify on the provider's own pricing page rather than as a fact you read here.

That is worth a paragraph rather than a footnote because it is the rule the whole site runs on: we publish rates we captured and arithmetic on them, and nothing else. When you see a figure here you can check it against the archive. When you see a claim like this one, you should check it against the provider. The two are not the same kind of statement and we would rather say so than blur them.

Which of your calls could have waited

The useful question is not whether batch is cheaper. It is which share of your traffic a human is actually waiting on. Go through your workload one job at a time and ask whether anybody is watching a spinner when it runs. Usually a surprising amount of it fails that test: nightly report generation, evaluation and regression suites, labeling and classification backfills, moderation sweeps, enrichment of records that arrived overnight, summarizing yesterday's tickets, embedding a corpus you just imported, and the entire first pass of any migration.

What generally cannot wait is anything in front of a person: a chat reply, an autocomplete, a search result, a form being filled in. Everything else is negotiable, and the negotiable share of a mature system is often the larger half, because scheduled and background work grows quietly while the interactive path stays roughly fixed.

Put a number on your own deferrable share

Do this in two passes. First, split your monthly token volume into the part a person waits for and the part they do not. Your usage export already has the shape of this if the two run under different API keys, different models, or at different hours; if they do not, that separation is worth building before you optimize anything, because you cannot route what you cannot see. The bill auditor will give you the per-model split from a real export in a few seconds, and it runs in your browser, so the export never leaves your machine.

Second, price the two halves separately. Put your real-time volume into the cost calculator and read the total. Then do the same for the deferrable volume and apply whatever discount the provider currently publishes to that half only. The difference between the two totals is the money on the table, and it is usually large enough to change which model you can afford for the interactive path, which is the second-order win nobody expects: batching the background work often pays for a better model in front of the customer.

Four things that catch people out

The stated turnaround is a ceiling, not a promise, and not a guarantee either. Jobs usually return much faster than the window, but you have to build as though they will take the whole of it, which means a queue, a completion check and somewhere to put results that arrive while nobody is looking. If your pipeline assumes a synchronous response, the engineering to change that is the real cost of this discount.

Batching does not reduce tokens. A wasteful prompt batched is a wasteful prompt at a discount, and the discount applies to the waste too. Trimming the prompt first and batching second compounds; doing only the second leaves the first saving on the table forever. The prompt optimization guide covers the trimming half.

How batch interacts with prompt caching is provider-specific and worth checking rather than assuming, because the two discounts apply to different parts of the same request and the rules for combining them differ. If your batched work has a large repeated prefix, that combination is exactly the case where the answer matters most.

And failures still cost. A request that runs and returns something unusable is billed like any other, so a batch of fifty thousand items with a broken prompt is an expensive way to find a bug. Validate on a handful synchronously, then submit the file.