AI is moving fast but is accountability keeping pace?

The power of AI capabilities is clear, AI is on international geopolitical agendas, and business strategies all seeking competitive advantage.

However, as organisations race to adopt it, one question is becoming harder to avoid: who is responsible if it goes wrong?

AI is not a traditional technology. It may form an entire system or a component of another software system. It’s also likely to integrate with other systems that rely on its outputs, creating a web of complex decisions and inference on data that is constantly transforming.

When an AI system output is inaccurate, risks data privacy, uses inadequate data, reinforces bias, or makes a decision that cannot be properly explained, care needs to be taken and accountability comes into focus.

Alongside a patchwork of global AI regulations to try and mitigate the potential harms, international standards like the NIST AI Risk Management Framework and OECD AI Principles have all developed guidance and procedures to encourage safe AI.

Harmonised themes have emerged, one prominent feature being ‘Accountability’. The thinking is that accountability, all the way through the lifecycle, will not only breed responsible AI and a human-centric approach, but also promote transparency and confidence in its use.

Until now, oversight of technology has often been treated as an IT matter. For AI, business leaders need to be included.

Checks and balances are needed throughout the lifecycle of AI products, users should know when they are interacting with AI or its outputs so that they can exercise their personal judgement and, in turn, help mitigate potential harms resulting from risks such as hallucinations, bias, or inaccuracies.

The pressure is increasing. Regulators and users are asking tougher questions on data protection, transparency, intellectual property rights, due diligence, and automated decision-making. High-profile legal cases involving the world’s biggest technology companies have also shown that questions around how AI models are trained, governed, and used are far from resolved.

The rapid undercurrent of technology advancements is also adding to this complex mix of legal and regulatory requirements, increasing the ambiguity.

For business leaders, this creates a simple reality. AI cannot scale without AI governance.

This doesn’t mean slowing innovation with unnecessary process. Good governance should help businesses move with greater confidence. It means knowing which AI tools are being used, what data they rely on, who approves decisions, who is responsible for what, and what happens when something goes wrong.

Without clear governance, organisations can end up with disconnected AI tools, duplicated efforts, inconsistent supplier arrangements, and high risk.

Worse still, employees may also use unapproved AI tools or ‘Shadow AI’ if the business does not provide guidance on what can be used, when, and how. This creates risk for the business and uncertainty for those who rely upon its services.

Governance means bringing legal, compliance, cyber security, commercial, operational, and technology teams together from the outset. It means choosing suppliers carefully, keeping records, tracing data use, monitoring systems after launch, and making sure humans remain involved where judgement matters.

AI can deliver real value. Yet, trust is the real currency in AI and will decide whether those benefits last.

_

Rebecca Pluthero, Senior Legal Counsel, Artificial Intelligence, InterSystems Corporation

LBC Opinion provides a platform for diverse opinions on current affairs and matters of public interest.

The views expressed are those of the authors and do not necessarily reflect the official LBC position.

To contact us email opinion@lbc.co.uk