Tech giants pledge action after AI chief urges slowdown amid warnings technology ‘could kill us all by end of decade’
Anthropic researcher Jacob Coxon, who resigned this week, said people building AI systems “earnestly believe it could kill us all by the end of the decade”
Elon Musk and Sam Altman have backed calls for tougher safeguards around artificial intelligence after Anthropic boss Dario Amodei warned the technology is developing at a pace that could outstrip efforts to keep it under control.
Listen to this article
Mr Amodei said companies at the forefront of AI should slow the rate at which they improve their most powerful models - giving researchers, governments and the industry more time to understand and manage the risks.
In an essay shared on X, he wrote: “We must slow the pace at which we improve the capabilities of AI models.
“Progress will still seem fast, and we must make wise use of the time we gain.”
It comes as the King is set to meet with leaders of the world’s most prominent artificial intelligence companies and ask them how the technology can be developed for “the good of humanity”, it has been reported.
Charles hopes the discussions will lead to a “wider understanding” of the “responsibilities” of developing AI, a source told The Sunday Times.
He will reportedly meet with leaders from Nvidia, Anthropic, Google DeepMind and Open AI in Scotland.
Read more: King to ask industry leaders how AI can be developed for ‘good of humanity’
Read more: 'A 50/50 chance AI will kill us in five years', warns ex-AI researcher
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
— Dario Amodei (@DarioAmodei) September 12, 2026
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our…
The Anthropic boss set out a three-step framework designed to “pace” the development of frontier AI systems.
It includes giving independent evaluators permanent, employee-level access to leading AI companies’ systems; co-ordination between firms in democratic countries on shared safety standards; and international co-operation on the risks posed by increasingly capable AI.
Mr Amodei said Anthropic would immediately commit to the first measure, adding: “We’ll provide third-party evaluators with permanent, employee-level access to our systems.”
Musk, whose xAI company developed the Grok chatbot, responded to Mr Amodei’s post by writing: “Dario is right.”
The Tesla and SpaceX boss said separately that he had been “sounding the alarm on AI for a long time”.
He also suggested AI companies should peer-review one another’s models, though he did not explicitly commit xAI to Anthropic’s proposed independent-evaluation system.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear…
— Jacob Coxon (@hilbertspaess) September 9, 2026
OpenAI boss Sam Altman said the company behind ChatGPT would adopt the same approach to external oversight.
“I agree with Dario that we need to pace the frontier,” Mr Altman wrote.
“This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.
“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
The intervention follows increasingly stark warnings from within the AI industry itself.
Anthropic researcher Jacob Coxon resigned this week, claiming leading companies were racing towards self-improving superintelligence while “gambling with our lives”.
He said people building such systems “earnestly believe it could kill us all by the end of the decade”.
His remarks were backed by Anthropic safety researcher Evan Hubinger, who wrote: “We really do earnestly believe AI could kill all humans!”
Mr Hubinger said he personally put the risk at “>10% within the next decade”, while adding that Anthropic was attempting to act responsibly but did not yet have a plan to solve the problem of ensuring a superintelligent system remains aligned with human interests.
Mr Amodei also pointed to recent cyber-security testing, in which advanced AI models were reported to have found ways to access the internet and attempt to infiltrate external systems in controlled scenarios.
He warned that a more capable and poorly aligned group of AI agents could cause “catastrophic damage”.
AI bosses visited the White House last month to discuss voluntary government safety testing for the most advanced US AI models after rogue models were found to have launched cyberattacks on external companies.
In late July, OpenAI reported some of its models went rogue and hacked into another artificial intelligence company in an "unprecedented cyber incident".
The ChatGPT maker said it had been testing the capabilities of some of its most advanced models – including those not yet released to the public – in a controlled environment.
But they escaped containment, reached the internet, and broke into a firm called Hugging Face to meet its testing objectives.