As AI ushers in drastic changes for the future of many professions, the legal sphere has emerged as one space where its use has seen especially concerning consequences. Cases across Australia where lawyers have cited non-existent cases or misquotations have left some legal experts calling AI “a dangerous mirage” in the profession.
AI Misuse Across Australian Courts
Last month, a federal court confirmed a Western Australian lawyer had been referred to the state’s Legal Practice Board and ordered to pay over $8000 in costs. The lawyer, who was involved in an immigration case, admitted he used multiple AI tools to create his submissions, resulting in false case citations, apologising for neglecting to “independently verif[y] all citations through established legal databases.”
Legal professionals stress the need for human verification when using AI.
Justice Arron Gerrard, while reassuring that the court did not subscribe to “a luddite approach” towards AI, admonished the risk of its use “under[mining]” solid cases “by rank incompetence” and wasting time and resources of the court.
Earlier in August, a Victorian defence lawyer representing a minor accused of murder was caught using AI in their submissions, which contained non-existent case citations and misquotes from parliamentary speeches. In July, Melbourne law firm Massar Briggs Law was ordered to pay costs for similarly faulty submissions containing AI ‘hallucinations’ in a native title case.
AI ‘Hallucinations’ and Further Problems for the Law
Thomson Reuters reports that AI use among legal professionals has almost doubled in the past year. While the sheer volume of legal literature and paperwork lawyers have to wade through can make AI tools attractive, large language models like ChatGPT and CoPilot are far from reliable.
AI ‘hallucinations’ refer to responses that are incorrect, fabricated, or irrelevant to the prompt. For example, an AI tool might cite a case that isn’t relevant to the submission a lawyer is drafting, or one that doesn’t exist altogether.
Because consumer grade AI models are trained on masses of data, misleading or incorrect can inform its responses. Historical biases in the information used by these models, or limited geographical range, can also compromise AI’s usefulness for legal submissions. More broadly, the use of AI tools in the justice sphere is problematic, given questions around their logical capacity and moral judgement.
‘Professional grade’ tools do exist, trained exclusively on a controlled dataset of legal documents. However, even with these specialised models, ‘hallucination’ is possible.
Last year, authorities in NSW, Victoria and WA also warned lawyers not to enter confidential or commercially sensitive information into generative AI tools. This came after a worker entered confidential information into ChatGPT, resulting in serious risks to the future placement of a child in a custody case.
On the other side of the bench, cases are emerging worldwide of individuals being falsely recorded as criminals by AI tools. One German journalist found himself described by CoPilot as the perpetrator of numerous cases he’d reported on – including a professed child molester, a drug dealer, and a con-man preying on widowers.
In Australia, Victorian mayor Brian Hood launched legal action against OpenAI after ChatGPT also wrongly described him as a convicted criminal. Hood later abandoned his case due to the overwhelming costs of legal proceedings.
While a blanket ban on AI use in legal proceedings is “neither practical nor appropriate,” says Australia’s Law Council President, Juliana Warner, there is “serious concern” over these emerging issues. “Where these tools are utilised by lawyers, this must be done with extreme care,” Warner stresses, “lawyers must always keep front of mind their professional and ethical obligations to the court and to their clients.”
Cover image by Sasun Bughdaryan on Unsplash.
Follow Maddie’s journalism on X.
Sign Up To Our Free Newsletter