How Prompt Caching Cuts LLM API Bills by Up to 80%

Most production LLM requests share a large, identical prefix: the system prompt, tool definitions, few-shot examples, or a set of reference documents. Prompt caching lets the provider process that prefix once, store the computed state briefly, and serve subsequent requests from cache at a steep discount. Anthropic bills cache reads at 10% of the normal input rate. Every OpenAI model we price reads cached input at 10% of its standard rate, the same discount. DeepSeek is deeper still, charging 2% to 3% of its already low input rate on a cache hit. If your bill is a coding agent rather than an application you wrote, the caching is automatic and the useful question is how much of it your own runs actually earned. Our Claude skill cost tool measures that from your transcripts.

What the discount is worth at scale

The math gets dramatic at scale. Imagine a support bot with a 3,000 token system prompt handling 500,000 requests a month on Claude Sonnet. Without caching, that prefix alone is 1.5 billion input tokens, $3,000 a month. With a 90% cache hit rate the same traffic costs $570.00, cutting the input bill by 81%. Nothing about the product changed; you simply stopped paying full price to reprocess identical text.

Work out your ceiling before you get excited

The discount applies only to the part of your prompt that repeats. Everything that changes on each call is billed at full rate no matter what you do, so the most caching can ever save you is set by one ratio: the fixed block divided by the total input. Work that out first, on one real request, before planning anything around it.

This is where enthusiasm usually meets arithmetic. A chatbot with a 3,000 token system prompt and a 200 token question is almost all fixed, and caching is transformative. Document extraction is the opposite, and it surprises people. There the document is what changes on every call and the instructions are what repeat, so the cache can only ever reach the instruction block, and its share shrinks as documents get longer. On a five-page scan the reachable share is around two percent. Same feature, same provider, and the saving is a rounding error rather than a transformation.

The four things that quietly stop it working

A single changed character. Caching matches on an exact prefix, so the block must be byte-identical between calls. The classic bug is a timestamp, a session id or a customer name interpolated into the system prompt: it looks like fixed instructions, and it invalidates the cache on every request. If your hit rate is mysteriously zero, look for something in the prefix that is not actually constant.

Order. Caching works on prefixes, so stable content has to lead and variable content has to trail. A prompt that interleaves fixed instructions with retrieved documents caches nothing beyond the first variable byte, however much fixed material sits after it.

Minimum sizes and short lifetimes. Providers set a floor below which a prefix is not cached at all, and caches expire after a period of inactivity measured in minutes. Steady traffic keeps a cache warm; bursty or low-volume traffic pays the write cost and then lets it lapse. Since cache writes can cost more than ordinary input, caching a prefix you do not reuse promptly is a small loss rather than a saving.

Your tooling may not expose it. A great deal of production automation is built in n8n, Make and Zapier, and those frequently give you no way to mark a cacheable prefix at all. The discount is real and you still cannot reach it. Check that your stack can ask for it before you budget on it.

And on 8 of the models we price, there is no discount to reach

Of the 40 models in our archive, 32 publish a cache-read rate and 8 publish none. On that second group caching is not a lever at any volume, and a plan that assumes it is will miss. That is worth checking on the rate card before you pick a model on the strength of a discount it does not offer.

Use the caching slider in the advanced calculator to model your own hit rate against your own prompt shape, rather than a headline number from someone else's architecture.