Do AI Text Detectors Work? What the Evidence Shows
Do AI text detectors work? OpenAI shut down its own, and research shows bias against non-native writers. What the evidence says and what to use instead.
AI text detectors are not reliable enough to settle whether a person wrote something. OpenAI launched its own AI text classifier in January 2023 and withdrew it in July 2023 over a "low rate of accuracy". Research on other detectors found they consistently misclassify writing by non-native English speakers as AI-generated. A detector score is a weak hint, never proof.
What OpenAI found about its own tool
When OpenAI shut down its classifier, Search Engine Land reported that the tool was retired because of its low accuracy. OpenAI's own evaluation, as widely reported at the time, had it flagging 26 percent of AI-written text correctly while wrongly labelling 9 percent of human-written text as AI. A tool that misses three quarters of what it looks for and accuses roughly one in eleven humans is not something to base a decision on.
The bias problem
A research paper, GPT detectors are biased against non-native English writers, tested widely used detectors and found that they consistently misclassified non-native English writing samples as AI-generated, while native samples were identified accurately. The authors caution against using detectors in educational or evaluative settings because they can disadvantage non-native speakers. They also report that simple prompting could both reduce the bias and help text evade the detectors, which tells you how thin the signal is.
The mechanism is easy to see. Many detectors lean on how predictable the wording is. Plain, careful, limited-vocabulary writing looks predictable. So does boilerplate, a template, or text edited for clarity.
Why the problem is hard
There is no fingerprint. Modern models write in many styles, and a person can edit model output until nothing is left to find.
Detection and generation improve against each other, so any accuracy figure ages quickly.
Short texts give almost no signal.
The costs are lopsided. A false negative loses little. A false positive can cost someone a grade, a job, or a client.
When a score is usable
Use it as one input, with a low bar for ignoring it. A reasonable rule: if a detector score would change a consequential decision about a person, do not use it alone. Ask for the draft history, talk to the writer about the content, or set work that shows the thinking in person.
What works better
Check the claims. AI text often fails on specifics. Verify names, figures and quotes. Our guide to telling if an AI answer is hallucinated covers this.
Look at process evidence. Version history, notes, and drafts show how a document grew.
Look for context fit. Does it reflect what only that person would know?
For other media, the tells differ. See how to spot AI-generated photos and job listings written entirely by AI.
More on the wider topic in the AI risks guide.
FAQ
Are AI detectors accurate?
Not reliably. OpenAI retired its own classifier over low accuracy, and independent research found detectors misjudge non-native English writing.
Can a detector prove someone used AI?
No. A score is a probability estimate from a model with known error rates, not evidence of authorship.
Why do detectors flag human writing?
Many measure how predictable text is. Clear, simple or formulaic human writing can look predictable too.
Is there a watermark that settles it?
Some vendors mark their own output, but a watermark only helps when the text came from that vendor and was not rewritten. It does not cover text in general.
How did this land?
About the author
Senior Editor, AI & Product
Cecilia leads the Swarmz editorial desk. She has spent a decade turning complex AI and product topics into writing people actually finish, and she owns the blog's quality bar.
