What Is Context Compaction in AI?
Context compaction is when an AI agent summarises its own earlier history to make room in the context window. It is why a long session forgets your rules.
Context compaction is what an AI tool does when a conversation gets too long for its context window: instead of failing, it replaces a chunk of the earlier history with a shorter summary of that history, then carries on. You usually are not told it happened. The visible symptom is an assistant that was following your rules perfectly for an hour and then quietly stops.
If the term context window itself is fuzzy, read what is a context window first. Compaction is the thing that happens when you hit the edge of it.
How context compaction works
The mechanics are unglamorous. A long session is held as a list of messages. When the total is approaching the limit, the tool takes the oldest portion of that list, asks a model to summarise it, and swaps many messages for one paragraph.
The session approaches the context limit, often at some threshold below it rather than exactly at it.
An earlier slice of the history is selected. Recent turns are normally protected, because recency is the cheapest proxy for relevance.
That slice is summarised, sometimes by a smaller and cheaper model than the one you are talking to.
The original messages are discarded and the summary takes their place. From here on, the model can only see the summary.
Step four is the one that matters. Compaction is lossy and irreversible within the session. The detail is not archived somewhere the model can go and fetch, it is gone from what the model can see.
What gets lost, specifically
Summaries preserve narrative and drop constraints. A summariser asked to compress fifty turns will reliably keep what the project is and what was built. It will reliably lose things that read as incidental at the time:
One-line rules you stated once. "Never edit files in /legacy" or "use British spelling" survives as a fact about the conversation roughly never.
Exact values. The specific version number, the precise column name, the particular error string you pasted in turn nine.
Negative instructions. "Do not use that library" compresses badly, because a summary of what did not happen is unintuitive to write.
Why a decision was made. The conclusion survives, the reasoning behind it usually does not, so the model will cheerfully revisit a settled question.
This interacts badly with a separate known weakness, which is that models attend unevenly to long inputs and are worst in the middle, described in the lost in the middle problem. A constraint that survived compaction can still be sitting in the least-read part of the prompt.
How to tell compaction just happened
Most tools give some signal, though rarely a loud one. Watch for:
An explicit note in the interface, often small text about summarising or condensing the conversation.
A sudden drop in reported token usage mid-session, which is the clearest tell if your tool shows it.
The behavioural one: the assistant re-asks something you already answered, or re-suggests an approach you rejected an hour ago.
That last pattern is worth taking seriously rather than finding annoying. It is the model telling you its memory of your constraint is gone. Note that this is a different failure from a prompt that degrades because the model changed underneath you, which we cover in how to debug a prompt that stopped working.
Working around it
You cannot turn compaction off in most tools, and you would not want to, because the alternative is the session dying. What you can do is make your constraints survive it.
Tactic | Why it survives compaction |
|---|---|
Put rules in a file the agent re-reads, not in chat | Files are re-read on demand. Chat history is what gets summarised. |
Restate the three rules that matter every 20 turns or so | A recently restated rule sits in the protected recent window. |
Use a project or system instruction field if the tool has one | These are normally outside the compactable history by design. |
Start a fresh session at natural task boundaries | A short deliberate handover beats a long lossy summary. |
Ask the model to list its current constraints | Cheap audit. If a rule is missing from the list, it is gone, and you restate it now. |
That last one is the highest-value habit, because it converts an invisible failure into a visible one in a single question. On the cost side, note that compaction and prompt caching pull in opposite directions: compaction rewrites the start of your context, which is exactly what a cache wants to keep stable. For the layer beneath all of this, see how AI models work.
A related technique works earlier in the pipeline: prompt compression shortens the input before it reaches the model, instead of summarizing the conversation afterwards.
Frequently asked questions
Is context compaction the same as summarisation?
Compaction uses summarisation as its method, but the point is freeing space in a live session rather than producing a summary for you to read. You are not the audience for the output. The model is.
Does a bigger context window remove the problem?
It delays it. A million-token window pushes compaction much further out, and for most sessions you will never reach it. For long agent runs that read whole codebases, you still will.
Can I recover what was compacted away?
Not from inside the session. If the tool keeps a full transcript on disk or in its history, you can reread it yourself and paste back what matters, but the model cannot retrieve it on its own.
Why did my agent forget an instruction from the start of the session?
Almost certainly compaction. The start of the session is the first thing selected for summarising, and a single-line rule is the first thing a summary drops.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
