How to Prompt AI to Compare Two Documents

If you want to prompt AI to compare two documents, paste two contracts in and ask what the differences are, and you will get a fluent paragraph about each one followed by a short list of things that changed. The list will be incomplete, and the omissions will be the clauses that exist in one document and not the other.

That is not a model quality problem. It is a prompt shape problem, and it has a specific fix.

Why comparison collapses into summarisation

Comparison is a relational task. For every claim, the model has to hold both documents in mind and decide whether they agree, disagree, or whether one of them is silent. Summarisation is a sequential task, and it is what these models are overwhelmingly trained to do.

Given an open instruction, the model takes the easier path: summarise A, summarise B, then observe a few salient contrasts. The salience is the problem. Differences that are visually obvious get reported. Absences get dropped entirely, because there is no text in document B for the model to notice.

A clause that simply is not there is the single most expensive thing to miss in a contract review, and it is exactly the thing this failure mode hides.

The prompt that makes AI compare two documents properly

Stop asking for differences. Ask for a row per claim, with a category that makes absence an explicit output.

Compare Document A and Document B.

Work claim by claim, not document by document.

For every substantive claim in EITHER document, output one row:
| Topic | Document A says | Document B says | Verdict |

Verdict must be exactly one of:
- SAME       (both say materially the same thing)
- DIFFERENT  (both address it, and they differ)
- ONLY_IN_A  (A addresses it, B is silent)
- ONLY_IN_B  (B addresses it, A is silent)

Rules:
- If a document is silent on a topic, write "not addressed" in that
  column. Never leave a cell blank and never infer intent from silence.
- Quote at most 15 words from each document per cell.
- Do not summarise either document. Output the table only.
- After the table, list every ONLY_IN_A and ONLY_IN_B row again under
  the heading OMISSIONS.

Document A:
<<<
{document_a}
>>>

Document B:
<<<
{document_b}
>>>

Three things in there are doing the work.

• "Claim by claim, not document by document" blocks the sequential path directly.

• The two ONLY_IN categories turn an absence from something the model has to notice into something it has to produce. That is the entire trick.

• Repeating the omissions after the table costs a few tokens and forces a second pass over the rows that matter most, which measurably improves recall on them.

Longer than the context window

Two 40-page documents will not sit comfortably in one prompt, and even where they fit, comparison quality degrades badly across long contexts. Chunking naively makes it worse, because chunk 3 of A does not correspond to chunk 3 of B.

Align first, then compare:

1. Extract a section outline from each document separately. Heading text and a one-line description, nothing more.

2. Match the outlines to each other in a separate call. Ask for pairs, plus an explicit list of sections in A with no counterpart in B and vice versa. That list is already an omissions report.

3. Compare matched pairs one at a time, using the table prompt above on just those two sections.

4. Concatenate the tables. The unmatched sections from step 2 become ONLY_IN rows without any further model work.

This costs more calls and produces a dramatically better result, because every individual comparison is short and every alignment decision is made explicitly rather than implicitly. The chunking mechanics carry over from how to prompt AI to summarize a long document.

Make the output machine-checkable

A markdown table is readable and hard to verify. If this is running more than once, ask for structured output instead and validate it:

{
  "rows": [
    {
      "topic": "Termination notice period",
      "a": "30 days written notice",
      "b": "not addressed",
      "verdict": "ONLY_IN_A",
      "a_quote": "either party may terminate on 30 days",
      "b_quote": null
    }
  ]
}

Then assert mechanically: every row has a verdict from the allowed set, every ONLY_IN_A row has a null b_quote, and every DIFFERENT row has both quotes present. Those three checks catch most of the ways the output silently degrades. Getting reliable structure out is covered in how to get JSON output from AI.

Three things that still go wrong

The quote check is the highest-value one. It is a substring test, it takes one line, and it catches the failure mode where the comparison is fluent and invented. That is the same class of problem as any other unverified extraction, and the general shape is in how to prompt AI to extract data from a document.

One more habit worth keeping: instruct the model to say when it cannot determine a verdict rather than guessing one. The reasoning behind that is in how to make AI say I don't know.

Frequently asked questions

Why does AI miss differences between two documents?

Because an open comparison instruction gets answered as two summaries plus a few salient contrasts. Differences that consist of something being absent produce no text to notice, so they get dropped. A required ONLY_IN category fixes it.

Can AI replace a legal redline?

For a first pass on a contract you are reading anyway, it is a genuine time saver. For anything you are signing, it is a reading aid and not a substitute for a lawyer. Verify every quote against the source before relying on a row.

What if the documents are too long for the context window?

Extract section outlines from each, match the outlines to each other in a separate call, then compare matched sections pairwise. Alignment before chunking is what makes this work.

Should I use a diff tool instead?

If the two documents are versions of the same file, yes, a text diff is faster and exact. Use a model when the documents are independently written and the correspondence is by meaning rather than by line.

The general principles behind prompts like this one are collected in prompt engineering.