AI Generated Invoice Fraud: Spotting It and Stopping It

Typos, wrong logos and stilted English were never your fraud control. They were a free crutch, and it has been kicked away. The defence has to move from how it looks to what it matches.

Cecilia Iona
Cecilia Iona
Senior Editor, AI & Product
23 August 20261 min read

A fake invoice used to announce itself. Odd phrasing, a stretched logo, a greeting addressed to Dear Sir/Madam, a PDF that looked like it had been rebuilt from a screenshot. Those tells are gone. Generated text is fluent, generated layouts are clean, and a scraped logo renders correctly. Nothing about AI generated invoice fraud is new as a crime. What changed is the part your team was actually relying on to catch it.

Which means the useful question is no longer whether an invoice looks legitimate. It is whether it matches something you already know independently of the document in front of you.

What the tells were really doing

Cosmetic errors worked as a filter because producing a convincing forgery took effort, and most attackers were volume operators who would not spend it. That was an economic barrier disguised as a detection method. When the cost of a convincing document falls to near zero, the barrier disappears and you discover you never had a control, you had a coincidence.

Worth saying plainly: this is not new fraud, it is the same business email compromise the FBI has been documenting for a decade, with a better printer. The bureau's own guidance already points at the answer, recommending you verify requests for changes in account information through secondary channels. That advice predates the current generation of tools and survives it intact, because it never depended on the document looking wrong.

Three signals that still catch AI generated invoice fraud

Structural signals survive because they are about facts outside the document, not about its surface.

  1. The bank details changed. This is the single highest-value signal in payment fraud. A real supplier changes bank details rarely, and almost never in the same message as an invoice. Treat every change as hostile until independently confirmed.

  2. The invoice does not match a purchase order and a receipt. A three-way match is boring, decades old, and immune to how good the PDF looks. If your process has no purchase order, an invoice for work nobody ordered is indistinguishable from an invoice for work somebody did.

  3. The urgency is doing work. Payment fraud runs on time pressure because verification takes hours and the window has to close before anyone checks. An invoice that needs paying today, from a supplier who has never been in a hurry, is the message to slow down on.

Note what is not on that list: anything about the writing. Do not train your finance team to read tone. It was never reliable and it is now actively misleading, in the same way it has become for AI generated scam messages more broadly.

The one control that does the work

A bank detail change freeze. Written down, it fits in four lines:

Any change to a supplier's payment details triggers verification before
the next payment, with no exceptions for urgency or seniority.

Verification means: a phone call to a number already on file from
before the change request, to a named person, confirming the change.
Not a number in the email. Not a reply to the email.

The person who verifies is not the person who processes the payment.
The verification is logged with date, number called and person spoken to.

Three properties make this hold up. It does not require anyone to notice anything, so it survives a distracted Friday afternoon. It uses a channel the attacker does not control, since they have your inbox and not your supplier's landline. And it separates verification from payment, so a single compromised person cannot complete the loop.

The obvious objection is that voice can be cloned too, and it can. That is why the control specifies a number already on file and a named person you have spoken to before, rather than any inbound call. The threat model for that variant is covered in protecting your business from AI voice cloning scams.

A checklist for the invoices you receive

Check

What you are comparing against

Who does it

Supplier exists in your records

Your own supplier list, not the email

Whoever receives it

Bank details unchanged

Details on file before this message

Finance, before payment

Invoice matches a purchase order

Your PO system

Finance

Goods or work received

Delivery note or sign-off

The person who ordered it

Amount within normal range

Prior invoices from this supplier

Finance

Payment terms unchanged

The contract

Finance

Every row compares the document to something you hold. None of them look at the document itself. That is the point.

If you use AI in your own invoicing

Two things get easier and one gets harder. Easier: matching invoices to purchase orders and flagging anomalies is genuinely well-suited to automation, and we walked through the setup in automating invoicing with AI. Harder: an automated pipeline that reads an invoice and queues a payment removes the human who might have paused. If you automate, the bank-detail check has to be a hard stop in the pipeline, not a confidence score.

Also make sure your own invoices are easy for customers to verify. Put a phone number on them that reaches a person, and tell customers in advance, in writing, that your bank details will never change by email. Being hard to impersonate is worth more than being hard to defraud.

If you already paid

  1. Call your bank immediately and ask for a recall. Same-day requests occasionally succeed; next-week requests almost never do.

  2. Report it. In the US that is ic3.gov, in the UK Action Fraud, and most other jurisdictions have an equivalent. Recovery sometimes depends on a report existing.

  3. Tell the real supplier. Their systems or their inbox may be the compromised end, and they may have other customers being targeted right now.

  4. Check whether your own mail was accessed. A convincing invoice often means someone has been reading your threads, and the immediate loss may not be the only one.

Questions

Can I detect an AI generated invoice from the file itself?

Not reliably. Detection tools for generated text and images produce false positives and negatives, and an invoice contains very little text to work with. Treat any detector output as one weak input, never as a decision.

Are small businesses actually targeted?

Yes, and often preferentially, because approval chains are short and one person frequently both verifies and pays. The control above is more important with three people than with three hundred.

What if the request comes from our own CEO's address?

Then treat it as more suspicious, not less. Authority pressure is the oldest lever in this category, and the freeze policy exists precisely so nobody has to argue about it in the moment.

Does this only apply to invoices?

No. The same pattern covers payroll redirection, changed remittance instructions and vendor onboarding forms. Anything that changes where money goes deserves the same out-of-band check. The wider set of exposures is mapped in our overview of AI risks.

How do fake testimonials and fake invoices relate?

Same underlying shift: cheap, fluent, plausible artefacts at volume. The defence is the same shape too, verify against something outside the artefact, which we applied to fake AI testimonials and case studies.

How did this land?

About the author

Cecilia Iona
Cecilia Iona

Senior Editor, AI & Product

Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.

Share

Get the next post in your inbox

One email a month. Product updates, engineering posts, and the best of Built with Swarmz.

I agree to receive emails about AI building tips and Swarmz product news. Unsubscribe any time.