Anthropic CEO calls for AI companies to slow down development

Want LBC stories before everyone else? Set us as your Preferred Source on Google

Dario Amodei has detailed a three-step framework aimed at setting a more measured pace of development, which he says would give companies more time to manage the risks posed by increasingly capable AI models.

In an essay shared on X, the CEO said: "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain”.

Both Elon Musk, who runs xAI, and Sam Altman, CEO of OpenAI, have taken to X to show their support for Amodei’s proposal.

Like Anthropic, OpenAI will also commit to having "independent evaluators with employee-like access," Sam Altman said on Saturday.

Independent reviewers inspecting leading AI companies, cooperation among top AI firms to establish safety standards and international collaboration to manage AI risks are all part of the Anthropic CEO’s proposal.

Read more: UK Government rejects call for ‘kill switch’ to stop attack by rogue AI

Read more: Anthropic disrupted 'malicious use' of AI for cyberattacks, conventional weapons development, and 'biological misuse'

Amodei said he is not calling for a pause in model training or AI progress, but for companies to take the time needed to safeguard their models, with third-party evaluators establishing that this has been done.

The essay comes after Anthropic released a threat intelligence report on Thursday, detailing how numerous actors used its Claude AI models for activities ranging from weapons development to surveillance and fraud.

Amodei also referred to AI's evolving capability to improve itself, underlining concerns about it overtaking human ability to control its operation.

Anthropic researcher Jacob Coxon resigned earlier this week, raising concerns about the potential harms of AI.

Coxon stated that the "people building AI earnestly believe that it could kill us all by the end of the decade."

While Anthropic has presented itself as the more safety-conscious frontier lab, it has faced some apprehensions.

Last week, it revealed another instance of an AI model hacking external systems, after Claude models had hacked into the systems of three companies while carrying out cybersecurity tests in July.

"Given the accelerating rate of AI capability development, it's my worry that in 6-12 months such a swarm could be capable of taking over the entire internet potentially causing hundreds of billions of dollars in damage," Amodei wrote.

Amodei, who had previously worked as a vice president at OpenAI, has said he co-founded Anthropic in 2021 so he could build safer and more trusted AI models.

In his essay, Amodei said AI advanced "drastically faster" including its "ability to build the next generation of AI”.

He also referred to an incident involving rival OpenAI, which has revealed that agents carried out cybersecurity attacks on external targets in July.

Amodei also said AI companies should volunteer to collaborate to set standards, as growing numbers of U.S. lawmakers are calling for new rules to govern AI systems.