It usually starts the same way. Someone representing themselves asks an AI chatbot to help write a court filing. The result reads well. It cites cases with real-sounding names, proper-looking citations, even quotes. They file it.
Then the judge, or the other side, tries to look those cases up. Some of them do not exist. The filing gets struck, a hearing gets scheduled, and sometimes a fine follows. The person had no idea the cases were invented.
This is now happening in courts every day. If you are handling a case without a lawyer and using AI to help, this is the one thing you need to understand.
What "AI Hallucination" Means in a Court Filing
AI chatbots do not look things up the way a legal database does. They generate text that sounds right, based on patterns. Most of the time that works. But when asked for a case that supports your argument, a chatbot can produce one that fits perfectly, because it made it up.
In court filings, this shows up in three ways:
- →A case that does not exist, with an invented name and citation.
- →A real case described wrongly, cited for something it never said.
- →A made-up quotation, placed in quotation marks as if the court wrote it.
The second and third are harder to catch, because a quick search shows that the case is real.
How Big the Problem Is
The best public record comes from the AI Hallucination Cases Database, maintained by legal researcher Damien Charlotin, who teaches at Sciences Po in Paris. It collects court decisions where a judge found, or strongly indicated, that a filing contained AI-generated errors. By September 2026 it listed more than 2,000 cases worldwide, and earlier in 2026 Charlotin described new cases arriving at a pace of about five a day.
Two caveats matter. The database only counts cases where a court actually noticed and said so, so Charlotin calls it "necessarily an undercount." And the United States dominates it partly because American court records are easy to search, not only because Americans use more AI.
A recent analysis of the database by the legal publisher LawDistrict adds more detail. It reports that about half of the logged errors were fabricated citations, a quarter were real cases described incorrectly, and a fifth were false quotations, and that recorded fines have passed $1.15 million. Those breakdowns are LawDistrict's own analysis; you can read their methodology here.
Why People Without Lawyers Are Hit Hardest
An October 2025 study from Stanford Law School's Center for Internet and Society, which reviewed the database, found that self-represented litigants account for the majority of U.S. cases. LawDistrict's later analysis puts the share at about 60%.
The reason is not that self-represented people are careless. It is that no one checks their work. A lawyer who uses AI has, at least in theory, legal databases, training, and colleagues to catch a fake case. Someone filing alone often has only the chatbot, and it sounds confident either way.
Here is the hard part: courts generally hold you to the same duty as a lawyer. In federal court, Rule 11 of the Federal Rules of Civil Procedure says that anyone who signs a filing, whether "an attorney or unrepresented party," is certifying that its legal arguments are supported by existing law. Many state courts have similar rules. Judges often show some patience with people who are not lawyers, but "the AI told me" is not a defense to citing a case that does not exist.
The Case That Made Everyone Notice
In Mata v. Avianca, a personal injury case in federal court in New York, the plaintiff's lawyers filed a brief built partly on cases ChatGPT had invented. When they asked the chatbot whether the cases were real, it assured them they were. On June 22, 2023, Judge P. Kevin Castel fined the lawyers $5,000 and found they had acted in bad faith, in part because they kept standing by the fake cases after being questioned.
That detail is the most useful lesson in the whole story. Asking the same AI whether its answer is real is not a check. It may simply say yes.
What Courts Do When They Catch It
Responses vary widely from court to court, even for similar mistakes. Outcomes seen so far include a warning, an order to explain yourself, the filing being thrown out, a fine, paying the other side's costs of dealing with it, and, in some cases, losing the motion or the whole case. Lawyers can also be reported to their state bar.
One pattern stands out in how judges react: people who admit the mistake quickly and fix it tend to fare much better than people who deny it or double down.
How to Check Every Citation Yourself
You do not need a paid legal database. CourtListener and Google Scholar's case law search are free and cover a large share of published court opinions. For each case your draft cites:
- →Find it. Search the exact case name and citation.
- →Match every detail. The parties, court, year, volume, and page should all line up. A real citation that leads to a different case is still wrong.
- →Read the part you are relying on. Make sure the case actually says what your filing claims. Search the opinion for any quotation, word for word.
- →If you cannot find it, take it out. Do not keep a case because the AI insists it exists.
How to Use AI Without Getting Burned
AI can still help a lot when you are on your own. The safe uses are about understanding and organizing, not about supplying authority:
- →Explaining a court order, a notice, or a legal term in plain English.
- →Organizing your facts and a timeline before you write.
- →Drafting structure and wording, which you then check and fill with law you have verified yourself.
- →Listing the citations in a draft so you can check each one, rather than trusting them as a group.
Look for tools that admit what they do not know. When we tested our own case brief tool by giving it a case we had invented, it declined to describe any facts or holding and said it could not confirm the case existed. That is the behavior you want. But even a tool that behaves well is not a substitute for reading the case yourself.
Finally, check your court's rules. A growing number of courts and judges require you to disclose AI use in filings, or to certify that you checked every citation. The court's website or the clerk's office can tell you.
If You Already Filed Something With a Bad Citation
- →Act quickly. Check every citation in the filing now, before the court or the other side does.
- →Tell the court. File a short notice correcting or withdrawing the bad citations, and explain honestly what happened.
- →Do not defend a case you cannot find. Judges are consistently harder on people who stand by fake citations than on people who own the mistake.
- →Consider getting help. Many courts have self-help centers, and legal aid organizations may be able to review a filing.
Paste your draft to get each citation pulled out and flagged if it looks incomplete. It lists citations for you to verify; it does not confirm they are real.
This article is general legal information, not legal advice. Court rules on AI use are changing quickly and vary by court. Figures from the AI Hallucination Cases Database change daily; numbers here reflect published counts and analyses as of September 2026.
Editorial note: AI For Legal Research publishes independent content. We do not accept payment for editorial coverage or review scores. Nothing on this site constitutes legal advice. Always consult a qualified attorney for legal matters.