Is It Safe to Use AI for Legal Advice?

Whether it is safe to use AI for legal advice depends on which task you mean. It is safe to use AI to understand a legal concept, draft a first pass at a document, or organize your questions before a consultation. It is risky, and sometimes career-ending, to let it decide what the law actually says in your case, because AI models have a documented habit of inventing case law that sounds completely real and is not. The safe line runs between using AI to prepare and using it to decide.

This is the newest entry in a series asking the same question about different high-stakes domains. We found the same pattern with investment advice, tax advice, and medical advice: AI is a strong research assistant and a weak final authority whenever the answer is a specific, current, binding fact rather than a general concept. Law is arguably the sharpest version of this problem, because the fabrication risk is not theoretical. It has already put fake cases in front of real judges.

What AI is genuinely useful for in a legal situation

These uses are low risk because you can check the output yourself or a professional checks it for you before it matters:

• Explaining a term or process. What is a demand letter, what does an indemnification clause typically do, how does small claims court generally work. Concepts are stable and easy to verify against a plain-language source.

• Drafting a first pass. A rough outline of a cease-and-desist letter or a set of clauses to discuss with a lawyer, produced as a draft for review, not a final document.

• Summarizing a document you already have. Pulling the key obligations and dates out of a contract or notice so you know what to ask.

• Preparing for a consultation. Turning a messy situation into a clear, chronological set of facts and questions saves a lawyer's billable time.

Where it turns risky

When AI invents the law: real cases, real sanctions

Legal citation is a worst case for AI fabrication because a fake case looks exactly like a real one. It has a plausible party name, a plausible court, a plausible year, and a quote that sounds like something a judge would write. Nothing about the format signals that it is invented. That combination has already produced a growing body of real court sanctions.

The case that put this on the map was Mata v. Avianca in the Southern District of New York. Attorney Steven Schwartz used ChatGPT to research a brief opposing an airline's motion to dismiss, and the brief cited six court decisions as precedent, all of them fabricated, complete with fake quotes and fake citations to other fake cases. When challenged, Schwartz reportedly asked ChatGPT to confirm the cases were real, and it told him they were. In June 2023, Judge P. Kevin Castel sanctioned Schwartz, his colleague, and their firm $5,000, and ordered them to send a corrective letter to every real judge named in a fake opinion.

That was not an isolated event. In April 2026, the Alabama Supreme Court threw out an appeal in Ibach v. Stewart after finding the attorney's briefs riddled with, in the court's words, an astounding number of invalid, inaccurate, and irrelevant citations, including cases that did not appear to exist at all. The court ordered the attorney to pay $17,200 in fees and costs, referred him to the state bar for discipline, and barred him from filing anything else in that court without a second attorney co-signing. A separate federal case that same year saw a US district judge in Alabama reprimand three attorneys at a large firm representing the state prison system for filings citing nonexistent, ChatGPT-generated cases.

These are not three unlucky lawyers. Legal researcher Damien Charlotin maintains a public AI Hallucination Cases Database that only logs cases where a court explicitly found a party relied on fabricated AI output, and by late summer 2026 it had logged close to two thousand of them worldwide, with new entries arriving at a rate of several per day. Sanctions have ranged from a few thousand dollars to tens of thousands, plus bar referrals, adverse rulings, and, for attorneys, professional embarrassment on the public record.

Why this keeps happening even to careful people

A language model does not look up a case and report it. It generates the most statistically likely next words given everything it has seen, which for a citation means a plausible case name, reporter volume, page number, and year, with no underlying database check confirming any of it exists. Asking it to double-check its own answer often just produces a second, equally confident fabrication, which is exactly what happened in the Avianca case. That is also why courts in multiple countries have since adopted standing orders requiring attorneys to certify that AI-assisted research was checked against a real legal database before filing.

A safe way to use AI for legal advice

Use AI to get oriented, draft, and organize, never to source. Ask it to explain a concept, outline a document, or summarize a contract you already have, then verify anything that reads as a specific rule, citation, or deadline against a primary source: a court's own database, a statute, or a licensed attorney. Never submit, sign, or act on an AI-generated citation you have not independently pulled up and read. If real money, a deadline, or a court filing is involved, a licensed lawyer needs to sign off, the same way a human professional closes the loop in tax and investment decisions too. This trade-off, real research power paired with a confident habit of inventing facts, is covered more broadly in our overview of AI risks.

Two habits help alongside this. If you are pasting a contract, a demand letter, or case details into a chatbot, know what happens to those prompts after you send them before you paste anything privileged. And since the failure in every case above was missing a confident AI-generated fake, it is the same instinct that helps you recognize an AI-generated phishing email: polished, plausible, and wrong shows up well beyond the courtroom.

Frequently asked questions

Can AI replace a lawyer?

No. AI cannot be held professionally liable, cannot represent you in court, and has no reliable way to confirm that a case, statute, or rule it cites actually exists. It can speed up drafting and research, but the judgment, verification, and accountability of a licensed attorney are not replaceable by a chat interface.

Why does AI make up fake case citations?

Because a language model predicts plausible text rather than retrieving verified facts from a database. A fabricated case citation has the same shape and confidence as a real one, so nothing in the output itself signals that it was invented. Asking the model to double-check its own citation often just produces a second, equally confident fabrication.

Is it safe to paste a contract or legal document into ChatGPT?

Treat it like handing the document to an unknown third party. Check the tool's data retention and training policy, avoid pasting anything covered by attorney-client privilege into a consumer tool without checking your firm's policy first, and strip identifying details when you can.

How do I check if an AI-cited case is real?

Look it up directly in a primary source, your court's public docket, Westlaw, Lexis, or a free option like Google Scholar's case law search, using the exact case name and citation given. If you cannot find the case, or the quote attributed to it does not appear in the actual opinion, do not cite it.

This same safe-to-prepare, risky-to-decide split holds for immigration advice, where filing deadlines and category-specific eligibility rules raise the stakes of getting AI's role wrong.

The same safe-to-prepare, risky-to-decide framework applies to HR decisions too. is it safe to use AI for HR advice works through the HR-specific version of this question.