A Federal Court judge has warned that artificial intelligence could erode professional skills, compromise the integrity of court processes and leave judges increasingly responsible for working out what is real and what is machine-generated.
Justice Emilios Kyrou, speaking at the National Judicial Institute of Canada in June, said Australian courts and tribunals were approaching artificial intelligence in markedly different ways, with no uniform national approach to its use.
For lawyers, his comments provide a striking insight into how AI is changing the relationship between the profession and the courts – and the additional burden it can place on judges when AI-generated material finds its way into proceedings.
Justice Kyrou said the use of AI by Australian courts and tribunals could be considered across four areas: judicial decision-making, research and other non-decision-making uses by judges and their staff, administrative functions, and use by parties, lawyers and witnesses.
The most significant line, he said, should be drawn around judicial decision-making. Australian judicial guidance now provides that judges must not use generative AI for judicial decision-making, although supportive uses such as research are permitted provided the material is checked and verified.
Justice Kyrou said the prohibition reflected fundamental concerns about judicial independence. “A decision-maker would act inconsistently with their oath of office if they shared their decision-making function with an AI application,” he said.
The ‘black box’ problem
Justice Kyrou drew on comments by former High Court Chief Justice Robert French, who has warned of the risk that technology could progressively “de-skill” the human capacity to resolve disputes.
In a paper delivered in March, French considered the so-called “black box problem” – the difficulty of comprehending precisely how an AI system arrives at a particular outcome.
This raises an uncomfortable question about the use of AI in dispute resolution: “whether using an AI tool in resolving disputes is any different from resolving them by a throw of the dice.”
For judges, the stakes are particularly high. AI-generated content can be inaccurate or completely fictitious, he said, and flawed material relied upon in judicial decision-making could “damage public trust and confidence in courts and tribunals, and undermine the rule of law”.
When AI becomes someone else’s problem
The consequences are not confined to judges using AI themselves.
Justice Kyrou said lawyers, parties and witnesses had embraced AI for tasks ranging from legal research and document review to discovery, chronologies, summaries, presentations, translation and e-filing.
But when AI-generated material is not checked, the burden can shift to everyone else in the proceeding.
“A key challenge in relation to the use of AI by parties and lawyers is that it places the onus on the opposing party and their lawyers, and on judges and tribunal members, to determine what is real and what is hallucinatory,” he said.
He pointed to examples of self-represented litigants filing material containing non-existent cases or legal propositions with no foundation in the case law. There have also been several instances of Australian lawyers being referred for disciplinary investigation after using inaccurate AI-generated material without checking it.
Justice Kyrou said this could impose a “heavy burden” on judges and tribunal members, who must spend time checking the accuracy and reliability of material submitted to them.
Even proofreading can change the law
Justice Kyrou also highlighted a less obvious risk: lawyers and judges may not always know when they are using AI.
He pointed to Microsoft Copilot as an example. While it can be used to proofread documents, suggested corrections may extend beyond typographical errors and include changes to substantive content.
“There can be a fine line between using AI tools for research and improved readability, and AI inspired alterations which may ostensibly improve clarity but, on close analysis, alter legal meaning,” he said.
The point is particularly relevant as AI becomes embedded in everyday software rather than appearing only as a clearly identifiable chatbot or specialist legal tool.
‘I do not knowingly use AI’
Justice Kyrou’s speech also touched on his personal position. “I do not knowingly use AI for personal or official purposes,” he said.
His reason was straightforward: “I do not understand it sufficiently and therefore feel uncomfortable in using it.”
He compared his approach to investing in financial products. “I do not invest in financial products that I do not understand,” he said, while acknowledging the difficulty of knowing whether AI has been incorporated into apparently conventional research tools.
His position on judicial decision-making was unequivocal.
He supports the Council of Chief Justices’ position that judges should be prohibited from using generative AI in judicial decision-making.
At a personal level, he said he wanted his decisions to reflect his “unique voice and style”. “I do not want to share my voice with computer generated content,” he said.
His concern was that AI is based on an instantaneous collation of data rather than possessing a “core unifying value system”.
AI isn’t going away
Justice Kyrou’s speech was not an argument for abandoning AI.
He acknowledged that courts can use it for administrative purposes and that judges can use AI as a research tool, provided they personally check its “accuracy, reliability and relevance”.
The Federal Court itself is cautiously trialling AI for corporate services, while Court staff use Microsoft Copilot to transcribe internal meetings and produce summaries. The Administrative Review Tribunal is similarly trialling AI tools for functions including research, meeting productivity, cyber security and technical work.
The challenge, Justice Kyrou suggested, is not whether AI will become part of the justice system. It already has. The challenge is deciding where the machine should stop and the lawyer or judge must take over.
You can read Justice Kyrou’s full speech on the Federal Court website.
