What Is a Context Window? Tokens, Limits, Costs

A context window is the maximum amount of text a model can hold in view at one time, counted in tokens, covering everything at once: your instructions, the conversation so far, any files you attached, and the answer it is currently writing. When you go past it, something has to be dropped, and the model stops being able to see whatever got dropped.

That is the whole concept. The interesting part is what it means in practice, because the headline number on a model card is a much weaker promise than most people assume.

Tokens, in one paragraph

Models do not read characters or words. They read tokens, which are chunks of text roughly the size of a common word or a fragment of a longer one. The usual working estimate for English is about four characters per token, or about 750 words per 1,000 tokens. Code runs denser than prose because punctuation and indentation eat tokens, so a file of code costs more than a page of writing the same length on screen.

What happens when you go over

Three different things, depending on the tool, and knowing which one you are dealing with explains most confusing behaviour.

• Hard error. The request is rejected outright. Annoying but honest, because you know exactly what happened.

• Silent truncation. The oldest part of the conversation is cut. The model keeps answering fluently while having quietly forgotten your first instruction.

• Summarisation. The tool compresses earlier turns into a shorter summary and continues. Better than truncation, but details get lost in the compression and you are not told which ones.

The second and third are the ones that cause real damage, because nothing looks wrong. A model that has forgotten your requirement does not say so. It produces confident output based on what it can still see.

A worked example: a 40 file project

Say you are working on a small web app and you want the model to see the whole thing. Here is roughly where the tokens go.

Against a 200,000 token window that looks comfortable. The trap is that this is the state at the start. Twenty turns later the conversation section alone has tripled, and every failed attempt, every error message you pasted, and every file the model re-printed is still sitting in there consuming space.

A bigger window is not a better memory

This is the part that surprises people. Fitting inside the window is necessary, not sufficient. Models attend unevenly across a long input, and material buried in the middle of a very long context gets used less reliably than material near the beginning or the end. This gradual decline in reliability as a prompt grows, well before the window actually fills up, is what researchers call context rot.

The practical consequence: a requirement you stated once, forty messages ago, is technically still in the window and effectively invisible. It has not been forgotten in the sense of being deleted. It is just competing with 60,000 tokens of noise for the model attention.

This is why restating your constraint in the message where it matters works so much better than trusting that you said it earlier. It is also why the single most effective fix for a conversation that has gone bad is to start a fresh one and paste in only what matters.

There is now benchmark data behind that caution. RULER-style long-context evaluations consistently find that a model's effective, reliably usable context is often only 50 to 70 percent of the advertised window, with recall and reasoning over material scattered across the full context degrading well before the token limit is reached. The number on a model card is a ceiling, not a guarantee of full recall.

What it costs

Context is billed. Every token you send is charged on every single turn, not once, because the whole conversation is re-sent each time. That detail is where surprise bills come from.

Some arithmetic at current prices. OpenAI cut the price of its cheaper GPT-5.6 tiers on July 30, 2026, taking the lowest tier to $0.20 per million input tokens, as reported by CNBC. At that rate, a 68,000 token context costs about 1.4 cents to send once. Do it across a 40 turn working session and the input side alone is roughly 55 cents, before a single word of output. On a flagship tier at $5 per million, the same session is closer to $14.

Pricing on the long-context side specifically has also moved. Anthropic dropped its long-context pricing surcharge in 2026: Fable 5, Opus 4.8 and later, and Sonnet 4.6 now bill the full 1 million token window at standard per-token rates instead of the premium tier that used to apply past 200,000 tokens. If you priced out a long-context workflow on an older rate card, it is worth re-checking, since that extra-context tax no longer applies to those models.

Neither number is frightening on its own. Both are worth knowing before you leave a coding agent running in a loop overnight. We covered what the July price change means more broadly in OpenAI cutting GPT-5.6 Luna by 80 percent.

How to actually work within it

1. Start a new conversation per task instead of running one endless thread. Fresh context beats a long one almost every time.

2. Send the files that matter, not the whole repository. Relevance beats volume.

3. Restate the constraint that matters in the message where it matters.

4. When the model starts repeating fixed mistakes or forgetting rules, treat that as a context problem rather than a model problem, and reset.

5. Paste errors once. Re-pasting the same stack trace four times triples your context and adds nothing.

That kind of forgetting is a context-budget problem, not a personality quirk. See why an AI chatbot forgets earlier instructions for how it shows up day to day, and how to tell a dropped instruction from one that is just losing the competition for attention.

Tools that fetch information on demand, rather than stuffing everything in up front, are the structural answer to this problem. That is a large part of why the Model Context Protocol exists.

Context windows are measured in tokens, not words or characters. If that unit itself is unfamiliar, what a token actually is is worth reading first.

Frequently asked questions

How many words is a 128,000 token context window?

Roughly 96,000 words of English prose, using the 750 words per 1,000 tokens estimate. For code, expect meaningfully less, because code is denser in tokens per visible character.

Does the context window include the answer?

Yes. Input and output share the budget, so a very long input can leave the model too little room to finish a long answer. Some providers set a separate cap on output length on top of that.

Why does the AI forget what I told it earlier?

Either the instruction was truncated out of the window, or it is still in there but buried far enough back that it is not competing well for attention. Restating it in your current message resolves both cases.

Is a model with a bigger context window always better?

No. A bigger window removes a hard limit, it does not improve how carefully the model uses what is inside it, and it costs more per turn to fill. Well selected context in a small window usually beats everything in a big one.

Does the context window explain why AI writes broken code?

It explains a good share of it. Code that does not fit the rest of the project is often code written by a model that could not see the rest of the project. There are several other causes worth knowing, and they have different fixes. Being specific in the prompt covers most of the rest, which is the subject of our prompting guide for AI app builders.

Related: how tool calling works in AI agents