OpenAI hacked the Australian government. Is Whitehall next?

When an AI system does something similar, we cannot shrug and call it an interesting experiment.

The investigation into the OpenAI agent accessing an Australian government health portal is still in its early stages, and there is plenty we do not yet know. But the bigger warning is already staring us in the face.

What appears to have started as an innocent research request led an AI agent to pursue its goal in ways its creators did not intend. It used the tools available to it and crossed boundaries it was not expected to cross.

That gap between what we ask an AI system to do and what it actually does is where this becomes dangerous.

In this case, the information accessed may ultimately prove relatively low risk. Point the same behaviour at a hospital, an energy network or a water treatment plant and the stakes change immediately. Cyber security then stops being a conversation about protecting data and becomes one about whether essential services remain operational and whether people are put at risk.

The UK has already had its warnings. The Ministry of Defence suffered a breach involving an external Armed Forces payroll system, while the Legal Aid Agency later revealed attackers had accessed significant amounts of highly sensitive personal information.

Neither incident was caused by autonomous AI, but UK organisations are already being successfully targeted by human attackers. It would be dangerously complacent to assume our government departments, public services and critical infrastructure will somehow be immune as those same capabilities become faster, cheaper and increasingly automated by AI.

We already understand this principle elsewhere.

If a company manufactures an aircraft, a car or explosive ordnance, rules govern how that technology can be used. There are controls, responsibilities and consequences when safeguards fail because we recognise that powerful technology can cause harm.

AI cannot somehow sit outside that principle.

We have also seen what happens when regulation and responsibility fail to keep pace with technology. For much of the past 20 years, governments, technology companies and society have debated the consequences of social media while the platforms themselves transformed how people communicate, consume information and interact with the world.

We are still dealing with many of those consequences today.

AI gives us far less time.

Its capabilities are developing in hours and months rather than years and decades. We cannot spend another 20 years debating where responsibility sits while the technology races ahead.

Criminals certainly will not wait.

An AI agent that persistently searches for another route when it encounters a barrier may be fascinating in a research environment. Put that same capability in the hands of a cybercriminal gang or hostile state and it becomes something very different.

An attacker that can operate at machine speed does not sleep, become tired or lose patience. It can repeatedly test systems, change its approach and search for weaknesses at a pace human defenders cannot realistically match manually.

For many organisations, particularly those without large security teams, that is simply not a fair fight.

Defence therefore has to evolve alongside the threat.

Organisations need visibility over AI agents and other non-human activity moving through their systems. Access should follow the principle of least privilege, so an agent can only reach the information and systems genuinely required for its task. Security controls need to identify and stop abnormal behaviour in real time rather than discovering it weeks later.

Automation also has to become a much bigger part of defence. Human expertise remains essential, but people need technology capable of operating at the same speed as the threats they are facing.

There is also a bigger question of accountability.

When an autonomous system behaves in a way its creator did not intend, who is responsible for the consequences? What controls should reasonably have been in place? How quickly should affected organisations be told?

Those questions cannot be left until after something catastrophic happens.

Boards should stop asking only whether their organisation is compliant and start asking something much harder: if an autonomous agent got into our systems tomorrow, could we keep operating and keep people safe?

Nobody should wait for the next incident to find out.

____________________

Graeme Stewart is Head of Public Sector at Check Point Software.

LBC Opinion provides a platform for diverse opinions on current affairs and matters of public interest.

The views expressed are those of the authors and do not necessarily reflect the official LBC position.

To contact us email opinion@lbc.co.uk