Amodei warns of an AI botnet – but it is his own assessment

Amodei warns of an AI botnet – but it is his own assessment

The developers themselves have now begun asking for restraint: the frontier labs are calling for a controlled slowdown and external oversight, but they face a Trump administration that rejects a slowdown citing the competitive gap with China – and a China responding rhetorically while building its own governance body.

On Saturday, September 13, 2026, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier." The message was unusual: a company competing to build the most advanced AI models asked the industry to slow down.

"We must slow the pace at which we improve the capabilities of AI models," Amodei wrote, according to Newsweek. And further: "Progress will still seem fast, and we must make wise use of the time we gain."

He did not call for a complete stop. The point was to give safety, testing and oversight time to catch up with capability development. But the timing of the message – just days before the UN General Assembly of heads of state – turned the essay into the starting gun for a week in which AI governance suddenly stood at the top of the international agenda.

The warning that is Amodei's own assessment

The most debated passage in the essay is a warning about AI agents coordinating attacks. "Given the accelerating development of AI capabilities, it is my concern that within 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet," Amodei wrote, estimating potential damages in the hundreds of billions of dollars, according to Newsweek.

This is worth underlining: This is Amodei's own assessment, on behalf of his own company – not a documented incident or a conclusion verified by independent experts. The reports circulating in the debate of concrete incidents involving autonomous AI agents are second-hand, incomplete, and impossible to verify from the available source material. The warning should therefore be read as a political argument from an interested party, not as an established fact.

The mechanisms actually on the table

The concrete element of Amodei's proposal is not just "slow down," but how control is to be organized. The most widely reported element, reported by AP (Kaitlyn Huamani), is that the frontier labs will give independent, external evaluators "ongoing, employee-like access." The evaluation teams are to have their own offices, badges and company laptops, so that they can monitor safety practices continuously – not through point-in-time audits. Both Anthropic and OpenAI have committed to this.

OpenAI has gone further than just access. In a post, the company's chief global affairs officer, Chris Lehane, wrote: "We will advocate for compatible international approaches for measuring capabilities, managing risks, preserving human control and determining when and how development should slow or stop – even if that means lowering the pace of models' capabilities," according to AP. (AP renders the original as "advocate for," meaning "work for" or "argue for.")

On Monday, September 21, under the UN General Assembly, OpenAI followed up in a blog post in which the company called for the US to lead an international effort on global technical standards for frontier AI – including recursive self-improvement, that is, systems that can autonomously improve their own capabilities, according to Reuters (Michelle Nichols).

This is what sets the September wave apart from earlier AI warnings: Concrete measures are being put forward – evaluator access, capability measurement, technical standards – and not just shaking prophecies.

The reactions, day by day

As early as Monday, September 14, China responded. At a press briefing in Beijing, Foreign Ministry spokesperson Guo Jiakun said: "AI is a technology of great importance to the welfare of all humanity. All parties must cooperate to promote the development of open and inclusive AI for mutual benefit and shared interests," according to Tempo/Antara. The spokesperson simultaneously warned that fear-mongering and unhealthy competition hinder global AI governance.

The formulation matters: This was a rhetorical response, not a commitment to accept any slowdown. China was simultaneously building on its own track. In July 2026, the country initiated the World Artificial Intelligence Cooperation Organization (WAICO), in which 29 countries, including Indonesia, have signed a global governance framework – according to Tempo/Antara, which is so far the only source for the details.

In the US, Senator Bernie Sanders went further than the labs. "When the future of humanity is at stake, Trump must negotiate a comprehensive treaty with China's President Xi to establish a pause in advanced AI development and a ban on superintelligence," he wrote on X, according to Newsweek.

Support also came from the research community, with weight attached. Bilal Chughtai, a former safety researcher at Google DeepMind, resigned in July 2026 and afterwards wrote on X, according to Fox News Digital: "I genuinely believe AI has the potential to kill everyone, and that we may be running out of time to avoid this outcome." Demis Hassabis, founder and CEO of Google DeepMind, was more nuanced: The direction of a collective slowdown is right, but the details need refinement, was his assessment according to Tempo/Antara. Australia's Prime Minister Anthony Albanese demanded global rules "to ensure that humans remain in control," according to The Conversation.

The AP report simultaneously shows that the industry is divided – some tech leaders have bristled at the slowdown demand. (The AP text is cut off mid-sentence regarding Mark Zuckerberg, and his position therefore cannot be reported.)

Obstacle number one: Washington

The biggest problem for anyone wanting a coordinated slowdown is that the US government has rejected it. The Trump administration fears that any pause would simply give China room to close the gap in a technology seen as central to national and economic security, Reuters reports.

This creates an extraordinary position: The companies driving the race forward are asking the government to slow it down – and the government refuses, citing the very same race. Amodei himself has acknowledged that it will be difficult to get China to cooperate, which leaves open the question of whether any international brake-regulation can function without both superpowers.

What happens next – and the open questions

On Wednesday, September 23, OpenAI's Sam Altman is scheduled to appear before the UN Security Council's 15 members to brief them on AI and international security. The meeting was convened by France, which holds the council presidency in September, and will be chaired by France's Europe and foreign minister, Jean-Noël Barrot, according to Reuters.

Several questions remain unanswered. What does "pacing" mean concretely – voluntary standards, statutory requirements, or treaties? Would China ever accept a mechanism that limits its own development, or will its engagement remain rhetorical while WAICO is built as an alternative to Western governance? And how credible are the underlying risk scenarios, when the most prominent warnings come from companies that are themselves asking to be regulated – and partly defining the threat themselves?

At least one change is real: For the first time, those building frontier AI stand closely aligned behind proposals for external control – while those with the power to enact it want nothing to do with it.