Why Long Context Costs More Than You Expect

Long context costs more because most frontier models now bill it at a higher rate per token once a request crosses a size threshold, not just because the request contains more tokens. Those are two different multipliers and they stack. A request that is four times longer than another can cost eight times as much, and the arithmetic surprises people the first time it shows up on a bill.

This is a pricing structure worth understanding before you design an agent around a million token window.

The two multipliers

The first multiplier is obvious. Twice the tokens, twice the input cost, at the same rate.

The second is the tiered rate. Vendors publish a threshold, and requests above it are priced differently. GPT-6 Astra, released 3 September 2026, lists $10 per million input tokens and $50 per million output tokens at standard rates, and $20 and $75 at long context rates, with the model reference putting the boundary at 272,000 input tokens. GPT-5.6 Sol runs the same structure at $4 and $20 standard, $8 and $30 long, per the pricing page.

Note what tips into the higher tier: crossing the threshold changes the rate for the request, not only for the tokens above the line. Check this against your own invoice rather than taking it on trust, because vendors do differ here and the difference is large.

Worked through, for a single Astra request:

The step from 250,000 to 280,000 tokens is a 12% increase in content and a 124% increase in cost. There is a cliff at the threshold, and agents that grow their context turn by turn walk straight off it without any code change.

Why long context costs more to serve

It is not arbitrary. Serving a long request is genuinely more expensive per token, for two reasons that compound.

Attention cost grows faster than linearly with sequence length. Every new token attends to every previous token, so doubling the sequence more than doubles the compute for that step. Modern implementations shave a lot off this, but the shape survives.

The KV cache grows linearly with context and lives in GPU memory. A single 500,000 token conversation can occupy the memory the provider would otherwise use to batch several short requests together. Batching is where inference economics come from, so a long request does not just use more compute, it blocks the technique that makes compute cheap. That is the real driver, and it is why the premium is a step rather than a smooth curve: it kicks in around the point where a request stops fitting the standard serving profile.

This is a different mechanism from the one that makes bigger models cost more, which we covered in why bigger AI models cost more per token. Model size and context length are separate levers on your bill, and you can pull one without the other.

The three ways to stay under the line

Cache the stable part. Prompt caching prices repeated prefix tokens far below fresh ones. Astra lists cached input at $1 per million against $10 standard, a 90% reduction on the part of your prompt that does not change between turns. If your agent replays a long system prompt, a schema, or a fixed document set on every call, caching is the single highest-leverage change available. Prompt caching explains what qualifies as a cacheable prefix, which is stricter than most people expect.

Summarise on a schedule, not on failure. The instinct is to let context grow until something breaks. Better is to compact deliberately at a fixed point well below the threshold, replacing the raw transcript with a structured summary of decisions and open items. You lose fidelity you were unlikely to use and you keep the whole workload in the cheap tier.

Retrieve instead of stuffing. A 1,050,000 token window makes it tempting to paste the entire knowledge base into every request. Retrieval exists precisely to avoid paying for tokens the model does not need for this question. A bigger window changed how much you can stuff in, not whether stuffing is a good idea, which is the argument in does a bigger context window mean better answers.

When paying the premium is correct

Sometimes it is. A single long request that replaces four short ones plus your own orchestration code is often cheaper in total, and always cheaper in engineering time. Tasks where the model needs to hold genuinely interdependent material at once, a full contract, a whole repository module, a long transcript being cross-referenced against itself, are exactly what the long tier is for.

The test is whether the material is interdependent or merely adjacent. Interdependent material justifies the premium. Adjacent material is a retrieval problem you are paying to avoid solving.

FAQ

Does the long context rate apply to output tokens too?

Usually yes, at a smaller multiple. Astra lists 1.5x on output against 2x on input. Output tokens are already the expensive side of the meter, so a 1.5x multiple on a long generation can outweigh the input premium.

Can I tell in advance which tier a request will land in?

Yes, by counting tokens before you send. Every major provider ships a tokenizer you can run locally, and the count is deterministic for a given model. Counting before you send is the whole technique.

Do cached tokens count toward the threshold?

Treat them as counting unless your provider documents otherwise. Caching reduces the price per token, not the size of the request, so a cached prefix can still push you into the long tier at the cheaper cached rate.

Is a smaller model with a shorter window cheaper overall?

Often, for narrow tasks. A smaller model running a classification step at a fraction of the rate can handle the filtering, leaving the expensive model a much shorter final prompt. Two models frequently beat one on cost without hurting quality.

How do I compare tiered pricing across vendors?

Model your own request size distribution first, then price it, rather than comparing headline rates. The method is in how to compare AI API pricing across providers, and it matters more now that headline rates describe fewer and fewer real requests. For the wider release context, see our guide on reading AI release news without letting it churn your roadmap.