AI for Lawyers: What Happens When Lawyers Trust AI Too Much?

AI for Lawyers: Across US courts, more than 700 cases now involve AI-fabricated citations. The pattern in every single one is identical and entirely preventable.

The Modern Counsel · 6 min read

In May 2026, a federal judge in Oregon handed down the largest AI-hallucination penalty in American legal history: $110,000, against two lawyers who had submitted 23 fabricated case citations and eight invented quotations to the court. The case was dismissed shortly after.

That same month, an Alabama family lost a trust dispute entirely because their lawyer filed citations to nonexistent cases. The Alabama Supreme Court dismissed the appeal, called the conduct egregious, and barred the lawyer from ever filing in that court again without a co-counsel sign-off.

These aren’t isolated incidents. According to legal analytics tracking by LexisNexis and Bloomberg Law, more than 700 court cases now involve AI-generated hallucinations or fabricated content, a number that has grown sharply as AI adoption in legal practice has accelerated. A database maintained by a senior research fellow at the Paris School of Advanced Business Studies tracks well over 1,400 instances where courts have caught fabricated AI material in legal filings.

The case that started it all

The cautionary tale every law student now studies began in 2023, when two New York attorneys submitted a brief containing six completely fabricated case citations generated by ChatGPT. When the inconsistencies were flagged, one of the attorneys reportedly asked ChatGPT to confirm the cases were real, and the AI tool obligingly confirmed its own fabrications. Both lawyers were fined and required to personally notify every judge falsely cited in their brief.

It would be comforting to think that the case scared the profession straight. It didn’t. A federal court in California later sanctioned two major, well-resourced law firms, not solo practitioners, but firms known for serious litigation capability, after their attorneys submitted a brief containing numerous hallucinated citations generated using a combination of AI research tools. The presiding special master admitted he’d “almost” included the fabricated citations in his own order before catching the error himself, later calling the situation “scary.”

“If fake cases become prevalent and effective, they will undermine the integrity of the legal system and erode trust in judicial orders.”

Why warnings alone don’t work

This is the part of the story that should genuinely concern every managing partner: education and warnings, on their own, have produced disappointingly modest results. Judges have explicitly and repeatedly warned the profession. Continuing legal education modules exist specifically on this risk. And lawyers keep filing hallucinated material anyway, not because they don’t know AI can fabricate but because verification habits haven’t kept pace with adoption speed.

Even immigration law, a high-stakes, time-pressured practice area, hasn’t been immune. One attorney was sanctioned for submitting fabricated legal quotations generated by an AI tool in an emergency deportation case, after working an expedited timeline while unwell. The court found the conduct violated professional rules and constituted bad faith, noting the attorney had either consciously avoided checking his sources or ignored an explicit warning from opposing counsel about the fake quotes.

The number that should change how you think about this

Here’s the figure that reframes the entire conversation: roughly 79% of lawyers now report using AI tools in some capacity in their practice, according to recent industry tracking. This is no longer a story about a reckless minority experimenting with a novel technology. AI use in legal practice is now the default, not the exception, which means verification discipline can’t be optional, occasional, or assumed. It has to be procedural.

What this means for firms building AI into daily practice

None of this is an argument against using AI in legal work. The lawyers being sanctioned weren’t punished for using AI; they were punished for failing to verify what it produced before putting their professional signature underneath it. That distinction matters enormously, and it’s the one that gets lost in alarmist coverage of this trend.

The pattern across every sanctioned case, without exception, is the same: a lawyer treated AI output as a finished product rather than a first draft requiring the same verification rigour as a junior associate’s research memo. The lawyers who avoid this outcome aren’t the ones avoiding AI; they’re the ones who’ve built citation-checking into their workflow as a non-negotiable step, every single time, regardless of deadline pressure.

For African firms moving into AI-assisted research and drafting, a shift already well underway, as covered elsewhere in this newsletter, the lesson from over 700 US cases is a gift, not a threat. It’s a fully documented map of exactly where the failure points are, before those failure points show up in African courtrooms instead.

Platforms like Smart Legal Africa help law firms establish structured workflows where legal documents are reviewed, approved, version-controlled, and managed through defined processes before they leave the firm. Combined with responsible AI use, these controls help reduce operational risk while improving efficiency and consistency across legal teams.