By Gad Coffie and Marianne Rose; members of the In-House/Government Lawyers Committee
Artificial intelligence is now ubiquitous. It affects our lives, businesses and systems in a manner that cannot be ignored. It is little wonder some observers have argued that the AI revolution is the most defining milestone in human development since the industrial revolution. This significant impact of AI is one thing, but harnessing that impact lawfully, judiciously and without malice is another.
The natural solution seems to be regulation. In the European Union, the progressively aspirational Artificial Intelligence Act has commenced. In this paper, we argue that the recently announced compromise of certain government websites by a rogue OpenAI agent (what has now become known as the Medicare hack) is the critical threshold event that must serve as catalyst for AI regulation in Australia. Before all of that happens, we discuss the key cybersecurity steps lawyers must take now.
The AI catalyst: A brief background on the Medicare hack
The Medicare hack is the first reported hack of a government system by an AI agent (although a similar breach in the United States was announced soon after that, as well as subsequent acknowledgement by Open AI of a June hack into a NSW state government department).
The facts as presented in the media are:
- the agent was given a simple research task;
- it sought access to a database in executing that task;
- it was reportedly denied access;
- it hacked into the system to assess the information anyway;
- this action, Open AI says, was inconsistent with the instruction given the agent (a phenomenon known as misalignment).
Beyond government websites, Open AI also reported that “dozens” of global institutions have had their websites meddled with by their agents, in some instances, agents “took images from ChatGPT user activity and transferred it elsewhere”. Much of this information has come to light post the “Hugging Face” breach.
Digital resilience: Cyber security till now
It is no secret that governments at any level hold an enormous amount of potentially sensitive information. This range from information on national security all the way to inconsequential statistical information or simply raw data. Historically, governments, much like corporate entities are targeted by malicious entities mostly for economic reasons.
Governments though have significant resources and, in most instances, have till now been able to ward off breaches with robust cyber defences using secure cyber security protocols. The Medicare hack has challenged this status quo.
It is widely accepted by industry insiders that most cyber vulnerabilities tend to be exploited by direct human involvement. What is unprecedented about the Medicare hack and the previously reported Hugging Face hack, is the actors were autonomous agents that operated independent of human commands, and whose sole motivation was to complete the assigned task. This has led to the emergence of an archetype of threats arising from runaway AI autonomy.
This new threat profile from autonomous and self-directed machine logic demands vigilance. And protections must be put in place, through regulation before sandbox controls fail.
Will there be a legislative solution?
The Australian Government, in fortuitous timing, positioned itself at the UN General Assembly as a global spokesperson in AI regulation. Buoyed by its enactment of world-leading legislation of the under 16s social media ban, Australia is now starkly presented with the opportunity to position itself as an AI regulation middle-power post the Medicare hack. The call for legislative response is not without backers. In fact, the CEOs of major American AI companies have called for regulation.
It is our contention that the days of technology neutral legislation is ripe for reconsideration.
Governing the algorithm: Protecting your practice and your clients
Western Australian lawyers are rightly concerned with how they can protect themselves, practices and clients from cyber threats. As practitioners, one instance of a breach of confidentiality is one too many. Law firms and practitioners are right to worry about whether in-house AI models holding confidential information are reliable.
We recommend that to ensure good cyber hygiene, legal practices must:
- invest in the most up to date security systems that are available for their use case;
- review regularly software licence agreements commensurate with regulatory and technological developments;
- ensure they are operating on closed tenant systems;
- train staff in AI and digital security, and ensure this is supported by a strong AI use policy;
- creating physical backups, and secure legacy systems;
- educate clients to not input advice or confidential information into publicly sourced AI models;
- educate clients on the importance of information integrity and ensuring privilege is not impliedly waived; and
- ensure that where automated decision-making is expressly permitted by law, human oversight is deployed as a necessary layer of quality control.
Algorithmic armour: Protecting Australian interests
The future is here, AI is now mainstream technology. As such, Australia must accept the ubiquity, and risks of AI.
Unlike prior inventions that preceded regulatory intervention, we are on the shores of the Rubicon. In comparison to air travel, the internet, autonomous driving and mobile phones, to name but four examples, AI is moving at a speed that traditional legal frameworks cannot keep pace with. While creativity and innovation should not be stifled, we need to think about the ethical framework we want tomorrow, today.
Effective regulation aimed at safety and proactive vigilance begins with government regulation. And, the legal community is uniquely positioned to identify and guide creative solution-based thinking, founded in judicial and ethical underpinnings, to support the Australian government in building novel legislative frameworks.
Legislation around ethical Asimov-like underpinnings, opt-in algorithms, human in the loop design, algorithmic bias, autonomous weapons control use cases, and aggression-baiting algorithms, could all be topics for consideration in this brave new world. These issues all stem from consent and control, and are not unfamiliar issues for Australian lawyers. Similarly, lawyers will be particularly well placed to explore copyright and intellectual property developments and upstream liability.
Much like the consultation into the recently introduced Online Safety Amendment (Digital Duty of Care) Bill 2026, in our view, development of legislation targeting AI use must move at pace and in consultation with major stakeholders. Australian lawyers are best positioned to commence discussions on these issues, today.
Waiting to be invited to the discussion table tomorrow is not an option. Such policy debate is urgently needed to safeguard Australia’s cybersecurity future; not stymie it.
