This is general information, not legal or regulatory advice, and Graft is not a law firm. For advice on your obligations under the SRA Standards and Regulations, speak to your COLP, COFA, or a qualified solicitor.
A hallucination is what happens when a generative AI tool produces something that reads as fact and is not. In legal work it takes a particular form: the model invents a case, complete with a plausible name, citation, and holding, that has never existed. The SRA's warning notice on the misuse of AI names this as one of its two central concerns, and the reason is that these inventions have already reached courtrooms.
The SRA puts it plainly: "AI tools can produce 'hallucinations', generating fictitious cases, references or seemingly factual assertions that may appear convincing despite having no basis in fact." The word to hold onto is "convincing." A fabricated citation does not announce itself. It looks exactly like a real one, which is why it slips past a busy fee-earner and into a filing.
The case that put it on the map
In 2023 a US personal-injury matter, Mata v Avianca, became the cautionary tale the whole profession now cites. Two New York lawyers submitted a brief containing six case citations generated by ChatGPT. None of the cases were real. When the court asked for copies, the tool obligingly produced fabricated extracts of judgments that did not exist.
On 22 June 2023, Judge P. Kevin Castel of the Southern District of New York sanctioned the two lawyers and their firm $5,000, finding they had acted in bad faith and then stood by the fake citations when challenged. The fine was modest. The reputational cost, reported worldwide, was not.
It reached the English courts
The English legal system has now had its own reckoning, and the response was sharper.
In June 2025 the Divisional Court, led by Dame Victoria Sharp, President of the King's Bench Division, sitting with Mr Justice Johnson, handed down a single judgment covering two matters: Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank. In the first, five of the authorities cited did not exist. In the second, of 45 citations put before the court, 18 were fabricated.
The court's language left little room to hide behind the tool. "Freely available generative artificial intelligence tools, trained on a large language model such as ChatGPT are not capable of conducting reliable legal research," it held, and the duty that follows is unambiguous. Those who use it anyway, the court said, "have a professional duty therefore to check the accuracy of such research by reference to authoritative sources, before using it." The court went further on consequences, warning that placing false material before a court can amount to contempt, and in the most serious cases to the criminal offence of perverting the course of justice. Where a lawyer puts false citations before the court, it added, "it is likely to be appropriate for the court to make a reference to the regulator."
It is not only solicitors. In Harber v HMRC, decided in December 2023, a litigant in person supported her tax appeal with nine tribunal "decisions" that turned out to be AI-generated inventions. The tribunal, borrowing the reasoning from Mata, noted that fake authorities waste everyone's time and corrode confidence in the case law that the system relies on. The appeal was dismissed on its merits regardless.
This is no longer rare
It is tempting to treat these as freak events. The numbers say otherwise. Damien Charlotin, a researcher who maintains a public database of AI hallucination cases, has tracked more than 1,600 instances worldwide as of mid-2026, up from around 200 a year earlier, and the count climbs most weeks. A large share involve practising lawyers rather than self-represented litigants, and the UK's share, though smaller than the United States', is rising.
What the SRA expects
The notice is direct about where responsibility sits. "Reliance on an output of AI would not be a suitable defence" when false citations reach a court. And the duty does not move: "You remain accountable for all work and advice delivered to clients, whether or not AI has been used in its preparation." The regulator closes with a line firms should read carefully: "If you fail to have proper regard to this warning notice, you are at risk of disciplinary action."
The failure is a missing check, not the tool
Read together, the cases share one feature. The problem was never that a lawyer used AI. It was that AI output reached a court without anyone verifying it against a real source. Every judgment above turns on the same absent step: the check.
That reframes the task. The goal is not to ban the tool, which the SRA does not ask for and which, as with any shadow use, tends to drive the habit underground. The goal is to know where AI sits in each workflow and to make sure a human review stands between its output and anything that leaves the firm. That is a question about your processes, and it is answerable.
Where Graft fits
Graft connects to the systems your firm already runs on and reads how work actually flows, which surfaces where AI is used in each workflow and whether a human check is in the loop. That gives you the oversight trail the SRA asks for: not a policy asserting that review happens, but evidence of where it does and where it does not, so you can close the gaps before a court finds them for you.
Graft's free connected assessment maps where AI is already at work in your firm by reading your real systems. For the fuller picture, read AI for law firms, or the companion pieces on shadow AI in law firms and whether ChatGPT is confidential for lawyers.