What to Do If an AI Tool Leaks Your Data
A step-by-step incident response guide for founders and indie builders who just learned an AI tool or vendor they use had a data breach, covering what to check first, immediate steps, and what to ask the vendor.
If an AI tool you rely on has leaked data, yours or your customers', your first move is not panic, it is triage. Figure out exactly what the vendor actually held, rotate anything connected to it, read their disclosure against what your contract actually promises, and decide fast whether you owe your own customers a notice. What to do if an AI tool leaks your data really comes down to four moves: contain, verify, communicate, document. Below is the order to work through in the first day or two, plus the specific questions worth putting to the vendor before you take their statement at face value.
Most breach notices are written by lawyers and PR teams, in that order. Your job is to read past the reassurance and figure out what actually happened to your account, not the average customer's account.
What to Check First: What Data Did the Vendor Actually Hold
Before you do anything else, get specific about what this vendor actually had. "AI tool" covers everything from a chatbot wrapper storing your prompts to a full workflow platform holding customer records, payment metadata, and API keys to other services. The blast radius of a leak depends entirely on what you fed it in the first place, which is exactly why it is worth periodically auditing whether it was safe to give an AI tool access to your data before you connected it to anything sensitive.
Account credentials and API keys stored in or connected to the tool
Customer or user data you uploaded, synced, or piped through it
Business data: financials, contracts, internal docs, source code
Prompts and outputs, which can contain more sensitive detail than people realize
Payment information, if the tool handles billing
A leak of prompt logs is a different problem than a leak of a connected database. Both matter. They do not require the same response.
What to Do If an AI Tool Leaks Your Data: The First 24 to 72 Hours
Once you know roughly what was exposed, work through this in order. Speed matters more than elegance here.
Confirm the breach through the vendor's own disclosure or a credible source, not a screenshot on social media.
Rotate every API key, password, OAuth token, and webhook secret tied to that tool, immediately, even before you know the full scope.
Check your own logs and the vendor's audit logs, if available, for unusual activity on your account in the window around the breach.
Inventory what data actually flowed through the tool, using the categories above as a starting point.
Pull the vendor's data processing agreement, terms of service, and any security addendum, and read the breach and liability clauses specifically.
Determine whether you have a downstream duty to notify your own customers or users, based on what was exposed and where they are located.
Loop in whoever handles legal or compliance for you, even if that is just you and an hour with a lawyer.
Document everything: when you learned about it, what the vendor said and when, what you did and when. You may need this later for an insurer, a regulator, or an angry customer.
Watch for secondary attacks, phishing attempts or credential-stuffing that use the leaked information, for weeks after the initial incident.
Decide, with a clear head and not in the first panicked hour, whether you keep using this vendor.
Read the Disclosure Like a Contract, Not a Press Release
Vendor breach disclosures are written to limit liability and manage headlines. That does not make them useless, but it means you should read them the way you would read a contract, not a news article. Pull the actual document you signed, the DPA or the security terms in the ToS, and check what it commits the vendor to: notification timelines, what counts as a reportable incident, what remediation they owe you, and whether you are entitled to details beyond the public statement.
What GDPR Actually Requires, and What It Doesn't
If you are a data controller under GDPR and this vendor is your processor, the vendor is required to notify you "without undue delay" after becoming aware of a breach, with no fixed number of hours attached to that step. Your own clock as the controller starts once you become aware: under Article 33 of the GDPR, you generally have to notify your supervisory authority without undue delay and, where feasible, within 72 hours, unless the breach is unlikely to pose a risk to the people whose data it is. Missing the window is not automatically fatal, but you then owe the authority an explanation for the delay.
If you are not operating under GDPR, do not assume there is no clock at all. In the United States, breach notification is a patchwork of state laws rather than one federal rule, and the trigger and timeline depend on the type of data and the state your affected users live in. The FTC's data breach response guidance is a reasonable starting point for the general sequence: secure operations first, then work through legal notification requirements, which will vary by jurisdiction and data type.
Do You Have a Downstream Notification Duty to Your Customers?
This is the question people skip because it is uncomfortable: does this vendor's breach create a notification duty of your own? If you run any kind of product or service and this AI tool touched data belonging to your users, the vendor's incident can become your incident, legally and reputationally.
Work through it like this:
Are you a data controller for the affected information, or just a processor passing it along? That changes who is actually on the hook to notify.
Does the exposed data include anything that triggers a legal notification duty where your customers live: names paired with financial details, health data, government IDs, login credentials?
Does your own privacy policy or your contracts with customers promise anything about vendor security or breach notice, regardless of what the law strictly requires?
Would a reasonable customer want to know, even if you are not legally required to tell them? Reputational risk does not wait for a statute.
This is also where it is worth separating a live data leak from a slower, quieter risk: whether the vendor was using your inputs to train its models in the first place. Those are different failure modes with different consequences, and it is worth knowing how to check if an AI tool trains on your data as part of understanding what you actually exposed, since data absorbed into a model does not behave like data sitting in a leaked database.
What to Ask the Vendor
A generic apology email from the vendor is not enough to close this out. Push for specifics, in writing if you can get them.
What data types were exposed, specifically for accounts like ours, not just in the aggregate?
When did the breach actually occur, when was it detected, and when was it disclosed? The gaps between those dates tell you a lot.
Was the exposed data encrypted at rest and in transit, and were any credentials hashed and salted or stored in plain text?
Is there any indication that our data was involved in model training, and could it resurface in outputs to other users?
What containment and remediation steps have been taken, and what is still in progress?
Will you provide a written incident report we can keep on file for our own compliance or insurance purposes?
What changes are you making so this does not happen again?
If the answers are vague or slow to arrive, that tells you something too. A vendor's response to its own breach is one of the clearest signals you will ever get about how seriously it takes your data the rest of the time.
Preventing the Next One
A leak is also a forcing function to look harder at how you choose AI vendors in the first place, since most of this scramble is easier to avoid than to clean up after. Before connecting a new tool to anything sensitive, it is worth working through how to vet an AI vendor and asking the security and data-handling questions upfront rather than after an incident forces them. It is also worth thinking about what happens to your data when an AI company shuts down entirely, since a breach and a shutdown raise a lot of the same questions about where your data actually ends up. Both fit into the wider landscape of AI safety and risk, which is worth reading before your next integration, not after the next incident.
FAQ
What should I do first if an AI tool I use gets breached?
Confirm the breach through the vendor's official disclosure, then immediately rotate every credential, API key, and token connected to that tool before you do anything else. Figuring out the full scope can wait a few hours; locking down access should not.
Do I have to tell my customers if an AI vendor I use had a data breach?
It depends on what data was exposed, where your customers are located, and whether you are legally a controller or processor for that data. Many jurisdictions only require notice for specific data types like financial or health information, but reputational expectations often go further than the legal minimum.
How long does a company have to report a data breach under GDPR?
Under Article 33 of the GDPR, a controller must notify its supervisory authority without undue delay and, where feasible, within 72 hours of becoming aware of a breach, unless the breach is unlikely to pose a risk to the affected individuals. A processor, which is often the role an AI vendor plays, must notify the controller without undue delay but has no fixed hour count attached to that step.
Should I stop using an AI tool after it has a data breach?
Not automatically. Weigh how sensitive the exposed data was, how quickly and clearly the vendor communicated, what concrete remediation they took, and whether their answers to direct questions were specific or evasive. A fast, transparent response is a different signal than silence followed by a vague statement.
What information should I ask an AI vendor for after a breach?
Ask exactly what data types were exposed for accounts like yours, the timeline between occurrence, detection, and disclosure, whether data was encrypted or hashed, whether your data was involved in model training, and for a written incident report you can keep on file.
Related: whether an AI coding agent specifically can leak your API keys
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
