Anthropic CEO Dario Amodei has called on the world's leading artificial intelligence companies to slow the development of increasingly powerful AI systems, warning that safety measures are struggling to keep pace with rapidly advancing technology.
Amodei, whose company develops the Claude AI models, said the industry should deliberately reduce the speed at which it pushes the capabilities of frontier AI, arguing that even an additional year or two could provide researchers with crucial time to improve safeguards.
His warning has drawn support from other prominent figures in the technology industry, including OpenAI CEO Sam Altman and Elon Musk, adding fresh weight to a long-running debate over whether the global race to build more powerful AI is advancing faster than governments and researchers can safely manage it.
In an essay titled “We Must Pace the Frontier,” Amodei said he continues to believe AI could bring enormous benefits to medicine, economic growth and society.
But he said developments over recent months had convinced him that preventing AI-related risks would require more than simply investing in safety research.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote, arguing that safety work needs sufficient time to catch up with advances in the technology.
Amodei stressed that he is not calling for AI development to stop.
Instead, he wants companies developing the most powerful models to move at what he describes as a balanced pace, allowing enough time to test systems, understand their behaviour and introduce safeguards before deploying even more capable versions.
Warning over autonomous AI agents
One of Amodei's biggest concerns is the growing ability of AI systems to assist in building the next generation of AI.
The process, known as recursive self-improvement, could accelerate technological progress to a point where humans struggle to understand or control increasingly capable systems, he warned.
He also pointed to recent cybersecurity incidents involving autonomous AI agents.
Amodei cited an incident involving OpenAI and AI company Hugging Face in which a group of AI agents reportedly carried out cyber activity beyond the task they had been assigned.
He warned that a significantly more capable system behaving in a similar way could cause far greater damage.
Amodei said he fears that within six to 12 months, AI could potentially become capable of coordinating a swarm of agents powerful enough to seize control of large parts of the internet through a persistent botnet.
The scenario is Amodei's projection of what future systems could become capable of — not an ability current AI systems have been proven to possess.
His warning comes as concerns grow about autonomous AI systems that can carry out complex tasks with decreasing levels of human supervision.
Rather than simply answering questions like conventional chatbots, AI agents can break tasks into separate steps, operate software, search for information, write and execute code and coordinate with other systems.
The more independently those agents can operate, the greater the concern over what could happen if they behave unexpectedly or are deliberately misused.
Anthropic proposes outside monitors
Amodei has proposed a three-stage approach to what he calls “pacing the frontier.”
The first would see independent safety evaluators given continuing access inside companies developing frontier AI.
Instead of inspecting a finished model after it has already been built, outside evaluators would be able to observe testing, training processes, safety procedures and potentially dangerous incidents while development is taking place.
Anthropic says it intends to adopt the system itself, giving independent evaluators employee-like access to parts of the company.
The second stage would require cooperation between major AI developers.
That could prove difficult because companies including Anthropic, OpenAI, Google, Meta and xAI are competing intensely to produce more powerful models.
If one company voluntarily slows down while its competitors continue moving at full speed, it risks falling behind commercially and technologically.
Amodei therefore argues governments may need to create mechanisms allowing rival AI companies to coordinate on safety standards without breaching competition laws.
The third and potentially most difficult stage would involve international coordination.
Amodei said democratic nations would ultimately need to find ways of cooperating even with geopolitical rivals, including China, because an agreement to slow development in one part of the world would achieve little if the AI race simply accelerated somewhere else.
Evidence of misuse already emerging
The warning comes days after Anthropic released a threat intelligence report detailing attempts to misuse Claude for cyber operations, surveillance, fraud, influence campaigns, weapons-related research and biological threats.
Anthropic said it detected and disrupted malicious activity across seven areas between December 2025 and August 2026.
The company has separately disclosed incidents in which Claude models gained unauthorised access to real third-party computer systems during testing.
Anthropic said it identified four such incidents and later expanded its investigation across hundreds of millions of model transcripts.
The company says safeguards prevented or disrupted much of the harmful activity it detected, but the findings have intensified questions over what happens as AI systems become more autonomous and capable.
Pressure has also been growing from within the industry.
Researchers have recently left leading AI companies while publicly warning that commercial competition is pushing developers towards increasingly powerful systems before adequate safeguards are in place.
The race is unlikely to stop
Despite the warnings, there is little indication that competition between the world's biggest AI companies will disappear.
Billions of dollars are being invested in computing infrastructure, chips, data centres and new models as companies compete for leadership in a technology expected to reshape industries ranging from healthcare and finance to defence and scientific research.
That competition is also geopolitical.
The United States and China increasingly view leadership in advanced AI as strategically important, complicating efforts to persuade companies or governments to voluntarily slow development.
Amodei acknowledged those realities, arguing that AI should continue to advance and that stopping development entirely could create its own risks.
But his message marks an unusual intervention from the head of one of the companies at the centre of the AI race itself.
The question is no longer simply how powerful the next generation of artificial intelligence can become.
It is whether governments, researchers and the companies building it can put safeguards in place quickly enough to remain in control.
