How to Prompt AI to Summarize a Long Document
A summary that is 80% right is more dangerous than one that is obviously wrong. Three failure modes cause nearly all of it, and each has a different fix.
You paste a 90 page supplier contract into a chat window, ask for a summary, and get back eight confident bullet points. They read well. Two of them describe clauses that are genuinely in the document. One describes a termination provision that does not exist anywhere in it. Knowing how to prompt AI to summarize a long document is mostly about preventing that specific outcome, because the summary that is 80% right is more dangerous than the one that is obviously wrong. Three failure modes cause almost all of it, and each has a different fix.
Failure one: the middle of the document disappears
Long-context models do not attend evenly across their input. The effect is well documented: in Lost in the Middle: How Language Models Use Long Contexts, Liu and colleagues found that performance is highest when relevant information sits at the very beginning or very end of the input, and degrades substantially when the model has to retrieve something from the middle.
For summarisation this shows up as a summary that is detailed about the introduction, detailed about the conclusion, and vague about everything between. Which, in a contract, is where the obligations live. That kind of accuracy loss as a document grows, well before the context window fills up, reflects how model performance degrades over a long context, not just where the hard limit sits.
The fix is positional, not verbal. Asking the model to "pay attention to the whole document" does nothing. What works is forcing traversal:
Work through the document section by section, in order.
For each section, output:
SECTION: <heading or first line>
KEY POINTS: <up to 3, each with a short verbatim quote>
Do not summarise the document as a whole yet.You are converting one summarisation task into N smaller ones, each with the relevant text nearby. The overall summary comes afterwards, built from that intermediate output rather than from the raw document. This is the single highest-return change you can make.
Failure two: silent truncation
The second failure mode produces no error at all. Your document exceeds what actually fits, something upstream trims it, and the model summarises what it received without knowing anything is missing. In a chat interface you may never be told. In your own code, an oversized input can be truncated by whatever assembled the request.
The symptom is a summary that simply stops being accurate after a certain point, with no acknowledgement.
The fix is verification, not hope. Before trusting any long-document summary, ask a question the model can only answer if it saw the end:
What is the final numbered section in this document, and
quote its first sentence verbatim.If it cannot answer, or invents something, you never had the whole document. This takes five seconds and catches the failure that is otherwise invisible. Understanding what a context window actually is makes it obvious why this happens, but the check is worth running even when you are confident the document fits.
Failure three: invented structure
The third mode is the one from the opening. Asked to summarise, models produce well-formed summaries, and a well-formed summary of a contract contains a termination clause. If the document does not have one, the shape of the expected output exerts pressure toward inventing it.
Section headings are especially prone to this, because the model is pattern-completing a document type rather than reporting a document.
The fix is grounding every claim in quoted text. Require a verbatim anchor for each point:
Every bullet must include a quote of 5 to 15 words copied
exactly from the document, followed by the section it came from.
If you cannot find supporting text for something you would
normally expect in this kind of document, list it under
NOT PRESENT instead of describing it.That last instruction matters more than it looks. It gives the model a legitimate place to put its expectation, so it does not have to smuggle it into the summary as fact. The general problem is covered in how to tell if an AI answer is hallucinated, but for summarisation specifically, forced quoting is the countermeasure that works.
How to prompt AI to summarize a long document, end to end
Combining the fixes gives a structure that is longer than most people's summarisation prompts and considerably more reliable.
You are summarising a long document. Follow this exactly.
STEP 1 - INVENTORY
List every section heading in order. If the document has no
headings, list the first sentence of each major block.
STEP 2 - PER SECTION
For each section from step 1, output:
SECTION: <name>
POINTS: up to 3 bullets, each ending with a verbatim quote
of 5-15 words in quotation marks
If a section contains nothing substantive, write: no substantive content
STEP 3 - SYNTHESIS
Using only the output of step 2, write a summary of at most
250 words.
STEP 4 - GAPS
List anything a reader of this document type would expect but
that does not appear. Label this NOT PRESENT.
Rules: quote only text that appears in the document. Do not
infer. If you are unsure whether something is present, put it
in NOT PRESENT.The intermediate output is not waste. It is the audit trail that lets you check the summary without rereading the source, and it is why this approach beats a single instruction even when the whole document fits comfortably in context.
When to chunk instead
Above roughly 50,000 tokens, or whenever the document exceeds what your model handles well, split it yourself rather than relying on the model to cope. A workable recipe:
Split on structure, not size. Chapter, section or clause boundaries. Splitting mid-sentence at a fixed character count destroys exactly the context the summary needs.
Overlap by a few hundred tokens. Enough that a point spanning a boundary appears whole in at least one chunk.
Keep chunks at 4,000 to 8,000 tokens. Small enough that positional degradation is not a factor, large enough that each chunk still has usable context.
Carry a running header. Put the document title and section path at the top of every chunk so the model always knows what it is reading.
Summarise each chunk with the same prompt, then synthesise from the chunk summaries in a separate call.
Never synthesise from summaries of summaries more than one level deep. Each round of compression loses specifics, and by the third pass you have a summary of a vibe.
That final step is a chained pipeline rather than one prompt, and the general pattern is worth learning properly if you do this regularly, since chaining prompts applies to far more than summarisation.
Match the summary to what you will do with it
One more thing improves output more than any phrasing trick: telling the model what the summary is for. "Summarise this contract" and "summarise this contract so I can decide whether to sign it this week" produce genuinely different documents, because the second one implies which details are load-bearing.
Useful framings to state explicitly:
The decision. "I need to decide whether to renew."
The audience. "For someone who has never seen this document."
The length budget. A word count, not "brief".
The bias. "When in doubt, include the detail rather than dropping it."
Stating a length budget also stops the model doing the thing where a 90 page document and a 4 page document produce identically sized summaries. If you need repeatable output across many documents, the same discipline that produces consistent AI output every time applies here: fix the format, fix the length, and give the model somewhere to put uncertainty.
Frequently asked questions
What is the best prompt for summarising a long document?
One that forces section-by-section traversal, requires a verbatim quote behind every claim, and asks explicitly for what is missing. A single instruction such as "summarise this" reliably underperforms a four-step structure, because it lets the model produce a plausible shape instead of reporting the actual contents.
Why does AI miss things in the middle of long documents?
Because attention across a long input is uneven. Research on long-context behaviour found accuracy is highest for information at the start and end of the input and drops for material in the middle, so a document processed in one pass gets systematically thinner coverage through its centre.
How do I know if the model actually read the whole document?
Ask it to quote the last section verbatim. If it cannot, or produces something that is not in the document, the input was truncated somewhere and the summary only covers part of the source.
Should I split a document up or use a model with a bigger context window?
A larger context window removes the hard limit but not the positional degradation, so for documents where accuracy matters, chunking with structural splits usually produces better summaries than pasting everything into a large window and hoping.
How do I stop AI inventing sections that are not in the document?
Require a verbatim quote for every bullet and give it an explicit place to record expected-but-absent items. Most invention in summarisation comes from the model completing the pattern of a document type, and a NOT PRESENT list gives that impulse somewhere harmless to go. The broader technique sits under prompt engineering and generalises well beyond summaries.
How did this land?
About the author
Developer Advocate
Steve builds something with Swarmz every week and writes up what worked, what broke, and what he'd do differently. Tutorials and hands-on guides are his lane.
