Fake Precedents, Even Prompts Copied Verbatim... 80% of Judges "Have Experienced AI Hallucinations"

Fake judgments modeled on Supreme Court rulings emerge
80% of judges: "We have encountered AI hallucinations"
Lawyers submit filings without checking
A farce accounting for more than half of cases
91.7% of judges: "Institutional sanctions are needed"
National Court Administration vows strict action, including fines and discipline

As the use of artificial intelligence (AI) in judicial proceedings grows, cases are mounting in which litigants submit “fake precedents” or laws that do not exist. Eight out of 10 judges handling cases said they had encountered this. The National Court Administration is working on strong measures, including imposing fines.

The National Court Administration announced the results of a survey titled “Survey on Citations of False or Non-Existent Precedents and Other Materials Due to AI Hallucinations,” conducted from October 6 to 16 among 360 judges at courts nationwide.

About 80% of the judges who responded and handle civil, family, and administrative cases said they had encountered false or non-existent precedents being cited in court filings. Among the specific types reported (multiple responses allowed), citations of “non-existent precedents” were the most common, reported by 196 judges (80.7%), followed by citations of “false precedents,” reported by 159 (65.4%); citations of “false or non-existent laws,” reported by 41 (16.9%); and citations of “other materials, such as literature,” reported by 10 (4.1%).

The problem is that these errors are being submitted to courts without being filtered out. The most common response on who submitted fake precedents and other materials was “the party themselves (representing themselves in court),” cited by 164 judges (69.5%). However, 136 judges (57.6%) also named “a lawyer representing a party or serving as defense counsel.”

Numerous bizarre cases were also reported. In one, a “false judgment” was submitted that had been created with AI by copying the format of a Supreme Court ruling. In another, a ghost precedent was submitted with a Supreme Court case number alongside unfamiliar legal reasoning, even though no such case could be found in a search. Some filings even included copied text from conversations with AI or prompts asking for litigation strategy.

Submitting such “fake materials” delays courts’ review of cases and ultimately infringes on the public’s right to a fair trial. A total of 91.7% of judges agreed that “institutional measures are needed.” In particular, 64.8% said such measures were “very necessary,” underscoring the seriousness of the issue. Proposed sanctions (multiple responses allowed) included establishing a legal basis for imposing fines (61.4%), requiring verification of cited precedents and other materials (61.1%), making it mandatory to disclose the use of AI (60.6%), and referring lawyers for disciplinary action in cases of repeated false citations (55.4%).

An official from the National Court Administration said, “Citing fake precedents is a serious problem that burdens both the opposing party and the court and delays trials.” The official added, “We will consult with relevant agencies, including the Ministry of Justice, to pursue amendments to the Civil and Criminal Procedure Acts to establish a legal basis for imposing fines, and submit our views to the National Assembly.” The courts also plan to encourage parties involved in litigation to verify the accuracy of cited materials themselves, including by adding guidance on “Precautions When Using AI” to the e-litigation website.

This content was produced with the assistance of AI translation services.