Anthropic CEO Outlines Plan To Pace The Frontier

AI researchers have issued increasingly urgent warnings about artificial intelligence risks, with OpenAI CEO Sam Altman suggesting it may be time to “pace” development. But what would that actually entail?

In a recent blog post, Anthropic CEO Dario Amodei not only supported the call to “pace the frontier,” but also proposed three broad strategies for doing so. He stated that Anthropic is “unilaterally committing” to one of these approaches.

AI safety and alignment debates intensified this week after researcher Jacob Coxon announced his resignation from Anthropic. He cited concerns that leading AI firms are “gambling with our lives,” a sentiment echoed by others at the company who believe unchecked progress could be fatal by decade’s end.

While Amodei’s post did not explicitly mention Coxon’s resignation, the CEO explained that two factors convinced him to adopt a more cautious stance: the OpenAI-HuggingFace hack and the fact that “AI has been advancing drastically faster” recently, particularly regarding its “growing ability to build the next generation of AI.”

“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”

His first proposal involves “embedded evaluators” from third-party organizations like METR. These evaluators would verify that AI companies adhere to their pacing and safety commitments and ensure that safety incidents are reported. (OpenAI recently faced criticism for failing to report an incident where its AI agents took over a German wiki form.)

Amodei compared these evaluators to regulators embedded with bank employees, stating that inviting them in is “something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).” This includes providing evaluators with company badges, desks, and laptops, along with access “mostly comparable to what internal risk assessment teams have,” except where restricted by law or contracts.

Next, Amodei urged leading AI companies “within democratic countries” to coordinate “common safety standards as well as limits on the rate of unchecked AI progress.”

Such coordination seems unlikely due to the apparent friction between Altman and Amodei, as well as concerns that a coordinated pause could trigger antitrust scrutiny. Amodei addressed this in his post, writing that “for antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.”

Amodei also addressed concerns about Chinese AI dominance, often cited as an argument against slowing development. He argued that if the US government and tech companies take steps such as refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese firms, and cracking down on model distillation, they could “slow China’s progress enough to widen America’s lead significantly over the next 3–5 years.”

Finally, Amodei called for “global coordination,” where the United States and its allies “attempt to coordinate with authoritarian governments, to the extent this is possible.” He acknowledged “stark limits on what can be achieved” but suggested there might be opportunities for agreement, even if limited to “prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.”

Given Amodei’s past willingness to acknowledge AI’s potential dangers and his company’s openness to regulation, some AI advocates have criticized him as a doomer whose comments fuel the current backlash. In response, Amodei claimed he has tried to offer a “balanced” perspective, arguing that the backlash is “fundamentally a crisis of trust,” as people have grown skeptical of tech companies, the industry, and the government.

Industry critics remain skeptical of these apocalyptic AI warnings, suggesting they distract from the harm the technology is already causing.

Journalist Brian Merchant, for example, stated he has yet to see “a credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planet”; he also suggested that proposals similar to Amodei’s “would likely only wind up serving Anthropic and OpenAI; it’s what regulatory capture looks like in action.”

In his new post, Amodei wrote that he continues “to believe that AI can enormously improve the quality of human life.”

“My desire to achieve these benefits is undimmed,” he said. “But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”