Top US law firm admits AI hallucinations in legal filing

Wall Street law firm Sullivan & Cromwell has apologized to a federal judge after a court filing included about 40 wrong citations and other errors linked to AI-generated content. 

The issue came to light in a letter sent by Andrew Dietderich, co-head of the firm’s global restructuring team, to Chief Judge Martin Glenn of the US Bankruptcy Court for the Southern District of New York.

Dietderich said the filing was part of an emergency motion submitted nine days earlier. He told the court that the firm accepts responsibility for the mistakes and said existing review steps did not work as intended in this case.

In the letter, Dietderich said the firm regretted what happened and acknowledged its duty to ensure that court submissions are accurate. He wrote, ”We deeply regret that this has occurred.”

He also said, ”The Firm and I are keenly aware of our responsibility to ensure the accuracy of all submissions.” Dietderich added that he takes responsibility for the failure tied to the filing.

The court filing contained around 40 incorrect citations and other errors. The firm said these problems were caused by AI hallucinations and mistakes that were not caught before submission.

Dietderich said Sullivan & Cromwell already has internal rules for AI use. Those rules include checking citations before material is filed in court.

Existing AI review policies were not followed

According to Dietderich, the firm had policies in place to prevent this type of problem, but the required procedures were not followed on this occasion. He said the review process failed to identify the false citations and other issues.

”Regrettably, this review process did not identify the inaccurate citations generated by AI, nor did it identify other errors that appear to have resulted in whole or in part from manual error,” he wrote.

The case adds to a growing list of legal filings affected by AI mistakes. A database run by legal technologist Damien Charlotin has recorded 1,334 incidents of AI hallucinations in court filings worldwide, with more than 900 of them in the United States.

Charlotin said many of these cases involve made-up citations. In some cases, AI-generated legal arguments have also appeared in filings.

Internal review begins after rival firm spots mistakes

Dietderich said Sullivan & Cromwell has started an internal investigation into what led to the errors. He said the firm took immediate remedial steps and is reviewing whether more training or stronger internal checks are needed.

He also noted that the mistakes were brought to the firm’s attention by rival law firm Boies Schiller Flexner LLP. Dietderich said he contacted the firm directly to thank them and apologize.

Sullivan & Cromwell is one of the largest law firms in the United States by revenue. The firm has also been involved in high-profile matters, including representing FTX in its bankruptcy case.

The incident has added fresh attention to the risks tied to AI use in legal work, especially when human review does not catch basic errors before filing.