Insights / The Value of Nurse-Verified AI

AI Hallucinations in Medical Record Review: The Risks Personal Injury Attorneys Can’t Ignore

By Breen LNC.

Short answer: A hallucination is confident AI output that is false. Reported rates vary widely with the task and the setup, so no single number applies. In clinical summary research, omissions are often more common than invented facts. Attorneys need a process that catches both.

What the word means

NIST prefers the term “confabulation,” which it defines as confidently stated but erroneous or false content, and says it follows from how these models predict output from patterns in their training data.

What the numbers say, and what they don’t

Reported rates depend heavily on the task.

  • Legal research. Stanford researchers reported in a 2024 preprint that leading legal research tools hallucinated between 17% and 33% of the time on a set of 202 queries. The published version says more than 17%. The authors wrote, “We demonstrate that the providers’ claims are overstated.” Those tools have changed since, and they don’t do medical review.
  • Clinical note generation. A 2025 npj Digital Medicine paper reported a 1.47% hallucination rate per annotated sentence in its tested setup. The authors work at a company that builds these tools.
  • Hospital summaries. A 2026 pilot reported hallucinations in 2 of 100 rated summaries and omissions in 25.

You can’t average these together. Nor can you assume any of them applies to the product your firm uses. We did not find a published hallucination rate for medical chronologies built for litigation.

The risk that gets less attention

Look at that last study again. Physicians reported omissions far more often than hallucinations. A fabricated fact can be disproved by opening the page. An omitted fact leaves no trace in the output, so a citation check won’t catch it. A reviewer has to know what should be there.

What attorneys can do

  1. Require a page cite for each medical fact, and spot check that your team is opening them.
  2. Have a clinician review the record itself, so missing items get noticed.
  3. Treat vendor claims of accuracy as claims. Ask how they were measured and on what kind of files.
  4. Don’t ask the tool to vouch for itself. In Mata v. Avianca, a lawyer asked ChatGPT whether a case was real, and it answered that it was.

That last example is the one to remember. Verification has to come from outside the system that produced the answer, which is why Breen has a nurse read the page.

Frequently asked questions

How often does AI hallucinate in medical records?

No single rate applies. Published figures vary by task and setup, such as 1.47% per sentence in one note generation study and 2 of 100 summaries in one hospital pilot. We did not find a rate for litigation chronologies.

Can I ask the AI tool to check its own answer?

That is not reliable verification. In Mata v. Avianca, a lawyer asked ChatGPT whether a case was real and it said yes, though the case did not exist. Check the source itself.

Sources

General information for legal professionals. This is not legal or medical advice.