Make AI Show Which Part of a Document It Used

To make AI show which part of a document it used, require a verbatim quotation next to every claim, and then check that the quotation exists in the source with a plain text search. Asking for a page number or a section name is not enough, because both can be produced from a vague memory of the document. A quoted span cannot: either those exact words are in your file or they are not, and that turns verification from careful rereading into a control-F.

Why this is a different problem from fake citations

There are two failure modes that look similar and need different fixes. One is a model inventing a reference to a source that does not exist, which is the subject of stopping AI from making up citations. The other is the model having your real document right there in the context and still producing a claim that the document does not support, by blending it with prior knowledge, by over-generalising one sentence, or by answering a question the document does not address.

The second is far more common when you paste in your own material, and it is harder to catch, because everything on screen looks plausible and derived from the file you supplied. Verbatim quoting solves it by making the link between claim and source mechanically checkable.

The prompt

The structure matters more than the wording. Three requirements: quote exactly, mark anything unsupported, and keep the claim and the evidence adjacent rather than in a list at the end.

Answer the question using ONLY the document below.

For every factual claim in your answer, immediately follow it with the
exact words from the document that support it, in this format:

  Claim text. [QUOTE: "verbatim text copied character for character"]

Rules:
- Quotes must be copied exactly. Do not paraphrase, tidy, or shorten
  inside the quote marks. Use an ellipsis only to skip whole clauses.
- Keep each quote between 5 and 30 words: long enough to find, short
  enough to read.
- If a claim needs several parts of the document, give one quote per part.
- If the document does not answer something, write
  [NOT IN DOCUMENT] and say what would be needed to answer it.
- Do not use knowledge from outside the document, even if you are
  confident it is correct.

Question: {your question}

Document:
{paste document}

The adjacency requirement is doing real work. If quotes are gathered into a bibliography at the end, you have to map each one back to the claim it supports, and that mapping is exactly where an unsupported claim hides. Interleaving them means an unsupported sentence is visibly missing its quote. It is a variation of the general principle that asking AI to explain its reasoning before it answers changes the answer rather than just annotating it.

Verifying which part of the document AI actually used

The point of verbatim quotes is that checking them is mechanical. For a handful of claims, search the document for each quoted string. For a longer answer, paste the response into a script:

import re, sys

doc = open(sys.argv[1]).read()
answer = open(sys.argv[2]).read()

def norm(s):
    return re.sub(r"\s+", " ", s).strip().lower()

nd = norm(doc)
quotes = re.findall(r'\[QUOTE: "(.*?)"\]', answer, re.S)

print(f"{len(quotes)} quotes")
for q in quotes:
    ok = norm(q) in nd
    print(("  found  " if ok else "  MISSING") + " " + q[:70])

Normalising whitespace before comparing matters, because models reflow line breaks from PDFs and a literal comparison fails on formatting alone. A quote that is still missing after normalisation is a real signal: either the model paraphrased inside quote marks, which means it is reconstructing rather than copying, or the claim is not in the document at all.

In practice one missing quote in an answer is usually a paraphrase and worth a second look. Two or more usually means the question was not answerable from the document and the model produced a reasonable-sounding answer anyway.

Two refinements are worth adding once the basic loop works. First, ask for the quotes in document order, which makes it obvious when an answer leans entirely on one page and ignores the rest of the file. Second, ask the model to state, at the end, which sections of the document it did not use at all. A model that claims to have used everything on a forty page contract is telling you something about its own confidence rather than about the document, and the sections it names as unused are a fast way to spot that the question was pointed at the wrong part of the file.

When the answer is a synthesis

The honest limitation: some correct answers are not quotable. If you ask whether a contract permits something, and the answer comes from combining a definition on page 2 with an exception on page 9, no single span supports it. Handle that with an explicit category rather than pretending otherwise:

The inferred-from-absence case is where models are weakest and where the damage is largest, since a document being silent on something is often the answer that matters. Ask directly what the document does not say, as a separate question, rather than hoping it surfaces in a summary. Prompting AI to find what is missing covers that as its own technique.

Where it fits with extraction and summarising

This pairs naturally with other document work rather than replacing it. When you are pulling fields out of a document, quoting the source span for each field turns a spot check into a full audit, which extends prompting AI to extract data from a document. When you are summarising, requiring quotes for the specific claims but not the connective prose keeps a summary readable while still making the load-bearing statements checkable, which is the practical middle ground for summarising a PDF accurately. Both sit inside the broader set of habits in our guide to prompt engineering.

Two practical notes. Quoting increases output length substantially, often by half again, so budget for it on long documents. And with very long inputs the quality of quoting degrades in the middle of the context in the same way recall does, so for a hundred page document you will get better results processing it in sections than asking one question across the whole thing.

Frequently asked questions

How do I make AI show which part of a document it used?

Require a verbatim quote of 5 to 30 words immediately after each claim, in a fixed format you can search for, and tell it to mark anything the document does not answer. The verbatim requirement is what makes the result checkable, since a paraphrase cannot be verified by searching.

Why does the model quote text that is not in my document?

Usually because it paraphrased inside the quote marks, which happens when the instruction says cite rather than copy exactly. Sometimes it means the claim came from the model's own knowledge rather than your file. Either way, treat a quote you cannot find as a claim you have not verified.

Does asking for quotes make the answer more accurate?

It makes unsupported claims visible, which is not the same thing. The model can still reach a wrong conclusion from correctly quoted text. What you gain is the ability to check each step quickly, rather than having to reread the whole document to trust the answer.

What should I do when the answer spans several pages?

Ask for one quote per component plus a short statement of how they combine. Then verify each quote by searching and read the joining logic yourself, because the reasoning that connects two accurate quotes is the part no string search can check.

Can I use this on a very long document?

Yes, but split it. Quoting accuracy falls off in the middle of a long context in the same way as recall, so a hundred page document gives better results processed in sections with the answers combined afterwards than asked as a single question.