What Is Prompt Caching, and When Does It Save Money

Prompt caching is a billing and latency feature that lets a provider store the processed form of a repeated chunk of your prompt and reuse it instead of processing it again. Concretely: if every request your support bot makes begins with the same 8,000 token system prompt and knowledge base, prompt caching means you pay full price for those tokens roughly once rather than on all thousand daily requests. What is prompt caching worth in practice depends entirely on one property of your prompt, and a lot of implementations get no benefit at all without anyone noticing.

What is prompt caching doing: it caches a prefix, not a prompt

The word "caching" suggests a lookup table of questions and answers. That is not what this is. It caches the model's internal processing of the *beginning* of your input, in order, up to the point where your input starts differing from the cached copy.

OpenAI's documentation is blunt about the consequence: "Cache hits are only possible for exact prefix matches within a prompt." One changed character near the start and the cache contributes nothing from that point on.

This makes prompt layout a cost decision. The structure that caches well looks like this:

[ system instructions        ]  stable, cacheable
[ tool definitions           ]  stable, cacheable
[ knowledge base / examples  ]  stable, cacheable
--------------------------------- cache boundary
[ conversation history       ]  grows
[ this user's question       ]  always new

And the structure that quietly caches nothing looks like this:

[ Today is 2026-08-12 14:07  ]  changes every request
[ system instructions        ]  now uncacheable
[ knowledge base             ]  now uncacheable

One timestamp at the top invalidates everything after it. This is the single most common way teams get zero benefit while believing caching is on.

What it actually costs

Anthropic publishes explicit multipliers against the base input price, which makes the economics easy to reason about. Per its prompt caching documentation, 5 minute cache write tokens are 1.25 times the base input price, 1 hour cache write tokens are 2 times, and cache read tokens are 0.1 times.

That gives you three prices for the same tokens.

The asymmetry is the whole story. A write costs 25% more than not caching. A read costs 90% less. So caching is enormously profitable when reads happen and mildly wasteful when they do not.

Worked example: a support bot

Take a bot with an 8,000 token stable prefix, a 200 token user question, and 1,000 requests spread across a working day.

Without caching: 8,200 input tokens per request, 1,000 requests, so 8.2 million input tokens billed at 1.0x.

With caching: one write of 8,000 tokens at 1.25x costs the equivalent of 10,000 tokens. The remaining 999 requests read those 8,000 tokens at 0.1x, costing the equivalent of 800 tokens each, or 799,200. The 200 token question is never cacheable, adding 200,000 across all requests.

Total: about 1.01 million token-equivalents against 8.2 million. Roughly an 88% reduction in input cost, for a change that touches prompt ordering and nothing else.

The latency improvement usually matters as much as the money, because the cached prefix does not need reprocessing on every call.

The two cases where it silently does nothing

Your prefix is not actually stable. Anything that varies at the front of the prompt breaks the match: a timestamp, the user's name, a session id, or retrieved context injected before the system instructions rather than after them. Retrieval-augmented setups are especially prone to this, because chunk order often varies between requests even when the chunks themselves are identical. The fix is ordering, not configuration: put everything stable first and everything variable last.

Your prefix is below the minimum. Caching does not engage on short prompts at all. OpenAI's docs state caching is available for prefixes containing at least 1,024 tokens. Anthropic's minimums vary by model, listed in its documentation as 1,024 tokens for models including Claude Sonnet 5 and Claude Opus 4.8, 512 for Claude Opus 5, and higher figures for some others. Below the threshold nothing errors, you simply pay the normal rate.

There is a third case worth watching: writing a cache that is never read. A prompt used once per hour against a 5 minute TTL will write every time and read never, costing 25% more than doing nothing. Either the traffic sustains the cache or it does not.

How long it survives

Retention is short and differs between providers, which matters for anything with bursty traffic.

Anthropic documents a default 5 minute TTL with an optional 1 hour TTL at higher write cost, measured from the start of the request that writes or reads the entry, meaning each hit refreshes the clock. OpenAI states that for GPT-5.6 and later a cached prefix "remains eligible for reuse for at least 30 minutes", with earlier models generally staying active for 5 to 10 minutes of inactivity.

The practical rule: sustained traffic keeps a cache warm indefinitely, because reads refresh it. Sporadic traffic pays write costs repeatedly. If your workload is a handful of requests scattered through the day, caching is not your cost lever, and the broader options in reducing AI API costs will do more.

What invalidates a cache

Anthropic documents the invalidation order as a hierarchy: tools, then system, then messages, where a change at one level invalidates that level and everything after it. Changing a tool definition therefore drops the system and message caches too, which is why adding a single tool can make costs jump without any prompt text changing.

This is also the argument for treating your system prompt as a versioned, stable artefact rather than something edited casually. Every edit is a cache reset, and the discipline involved is the same one behind versioning your prompts. It helps to be clear on what separates a system prompt from a user prompt before you decide what belongs in the cacheable region.

Frequently asked questions

Does prompt caching change the model's answers?

No. It reuses the processed representation of identical input, so the model sees the same prompt it would have seen otherwise. It affects price and latency, not output.

Is prompt caching the same as caching the response?

No, and the distinction matters. Response caching returns a previously generated answer for a repeated question, skipping the model entirely. Prompt caching still runs the model on every request and only reuses the processed prefix, so answers stay fresh.

Why did enabling prompt caching not reduce my bill?

Almost always because the prefix is not byte-identical across requests, or is shorter than the provider's minimum. Check for anything variable near the start of the prompt, particularly timestamps, user identifiers and retrieved context, since a single differing token disables everything after it.

Should I use the longer cache lifetime?

Only if your traffic gaps regularly exceed the default window, since the longer TTL costs 2 times base input on writes against 1.25 times. It pays when a warm cache would otherwise expire between bursts, and it wastes money when the default was sufficient.

Does caching mean my prompts are stored?

Providers cache the processed prefix for a limited period to serve your own subsequent requests, which is a different question from whether inputs are retained or used for training. That is governed by the terms rather than by the caching feature, and is worth confirming separately using the checks in how to check if an AI tool trains on your data. For the underlying reason repeated tokens cost anything at all, see how AI models work.