Dario Amodei argues that the artificial intelligence frontier has become too dangerous to run at full speed. His proposal deserves to be taken seriously. But slowing the race could also give the handful of companies already at the frontier more time to consolidate their position, shape the rules governing the technology and preserve their advantage over Chinese challengers.
Amodei set out the argument in an essay published in September, We Must Pace the Frontier. In it, he proposes a three-stage framework: embedded independent evaluators inside frontier laboratories, co-ordination among AI companies in democratic countries, and eventually international agreements intended to restrain the pace of the most advanced AI development.
There is something reassuring about Amodei’s call to slow the artificial intelligence race.
Unlike much of Silicon Valley, the Anthropic chief executive is not asking government to get out of the way. He argues that AI is advancing faster than the science of understanding and controlling it, increasingly helping to build its own successors. His answer is to “pace the frontier”.
The concern is not fanciful. Anthropic disclosed this month four cases in which Claude models obtained unauthorised access to third-party systems during cybersecurity evaluations. The models were operating in unusually permissive testing environments, but the incidents were serious enough for Anthropic to notify those affected and widen its investigation.
Amodei may be entirely sincere about the danger. Anthropic may genuinely regard catastrophic AI risk as its overriding concern.
That still leaves a problem with the proposed solution.
A slower frontier would give researchers more time to make AI safer. It would also give the companies already leading the industry something valuable: time to commercialise what they have built, recover vast investments in computing infrastructure and turn technological leadership into more durable market power.
The value of slowing down when you are ahead
Technological races do not treat leaders and challengers equally.
An outsider benefits from disruption. Rapid technological change can make an incumbent’s advantages obsolete and allow a rival to leapfrog it. A company already in front has a more complicated interest: it needs innovation to continue, but has less reason to welcome the ground shifting beneath it every few months.
Anthropic is no longer an insurgent laboratory. It has raised enormous sums of capital and is being discussed as a potential candidate for one of the largest technology flotations ever attempted. Whatever the eventual numbers, the financial interests gathering around the AI frontier are immense.
One sentence in Amodei’s proposal therefore deserves particular attention. A co-ordinated pacing strategy, he writes, would allow frontier developers to undertake safety work “without sacrificing commercial advantage or the United States’ lead in AI”.
That is part of the attraction of collective restraint.
Anthropic cannot slow substantially while OpenAI, Google or another rival continues at full speed. Unilateral caution could mean commercial defeat. If the leading companies slow together, that problem is reduced.
Permission to co-ordinate
Amodei calls the second stage of his proposal “Pacing Within Democracies”.
Frontier companies would agree common safety standards and limits on unchecked AI progress. Because collaboration between powerful competitors raises legal problems, he proposes government involvement. “For antitrust reasons”, Washington should mediate or enable discussions and provide a “narrow waiver” for certain safety conversations. His footnote explicitly contemplates waivers of antitrust restrictions.
There are legitimate reasons for competitors to co-operate. Banks share systems for managing systemic risks; airlines operate under common safety rules; pharmaceutical companies submit to common testing regimes.
But this would concern something more commercially sensitive than a technical standard. The companies could be discussing conditions governing how quickly their most important products become more capable.
Innovation itself is a dimension of competition.
US competition authorities have long recognised that collaboration among rivals can encourage investment but can also suppress competition, including competition in innovation. Would an AI safety waiver cover discussions of training runs, release schedules or capability thresholds? Under what circumstances could one company agree not to deploy a system while knowing its principal competitors were making the same commitment?
Where does legitimate safety co-ordination end and restraint of innovation competition begin?
Amodei recognises that the boundary exists. That is why government protection forms part of the proposal.
Safety can become a moat
His framework could reinforce incumbency in another way.
As models become more powerful, Amodei envisages increasingly demanding conditions: evaluations, interpretability work, audits, training-environment controls and outside inspection before further advances are made. Anthropic is also offering external evaluators unusually deep access to its systems and allowing them, within security and confidentiality restrictions, to publish adverse findings without ordinary company editorial control.
That is a serious transparency proposal. It is also expensive.
Anthropic, OpenAI and Google can afford teams of security researchers, evaluators, lawyers and compliance specialists around models whose development already requires extraordinary amounts of capital.
A new entrant would have to finance not merely a competitive model but much of the regulatory apparatus surrounding it.
The problem is familiar in other industries. A regulation can be entirely justified and still raise barriers to entry because fixed compliance costs fall more heavily on challengers than on incumbents with billions of dollars of capital and established regulatory operations.
AI safety could therefore become a regulatory moat, even if that is not its purpose.
Who writes the rules?
Governments cannot dispense with the frontier laboratories. The companies possess the technology and much of the expertise required to understand its dangers.
That dependence is unavoidable. But it requires scrutiny.
The companies being regulated would inevitably exert influence over the capability thresholds, tests and definitions of acceptable risk used to regulate them. Their technical knowledge gives them an informational advantage over legislators and officials that few industries can match.
Regulatory capture need not involve corruption. It can emerge because government becomes dependent on the expertise of those it is trying to supervise.
Amodei favours regulation rather than purely voluntary restraint and gives outside evaluators an important role. Anthropic can sincerely attempt to protect society while advocating rules that also happen to suit a well-capitalised frontier laboratory.
Then there is China
The international part of Amodei’s proposal makes the issue more complicated.
He is not proposing simply to slow the global AI race. His strategy is asymmetrical.
American and allied frontier companies should acquire enough breathing room to make AI safer. China should be prevented from using that breathing room to close the technological gap.
“A key part of pacing within democracies,” he writes, is keeping their AI lead over autocracies “as large as possible”.
The means are explicit: deny China the most powerful AI chips and advanced semiconductor manufacturing equipment, crack down on smuggling and remote access to overseas computing capacity, restrict unauthorised distillation of western models and improve protection against the theft of model weights.
Amodei argues that effective controls could significantly widen the American lead over the next three to five years.
This is therefore more than a safety policy. It is safety joined to industrial and geopolitical strategy.
Inside the US-led technological system, the leading companies would gain mechanisms for managed restraint. Outside it, Washington would restrict the strongest rival’s access to the means required to catch them.
Managed competition at home. Containment abroad.
China predictably describes American semiconductor controls in those terms, accusing Washington of using national security to suppress Chinese technological development. Beijing is hardly a neutral observer: China has its own extensive industrial policies, strategic restrictions and ambitions for technological leadership.
But the objection cannot simply be dismissed.
Freeze the lead, then negotiate
Amodei ultimately wants agreements with China.
They might begin with restrictions on AI assistance for biological weapons, advance towards common model testing and eventually include what he describes as a “speed limit” on recursive self-improvement. He invokes Cold War arms-control agreements as a possible analogy.
Yet the sequence may look rather different from Beijing.
First restrict China’s access to advanced semiconductors.
Then prevent Chinese laboratories from reproducing western capabilities through distillation.
Use those restrictions to widen the American lead.
Then negotiate limits on how quickly both sides may advance.
Amodei argues that technological superiority would give Washington leverage to secure an agreement. China could reasonably see an attempt to acquire an advantage and then institutionalise it.
There is also no guarantee that the technological gap can be frozen. Chinese companies have continued to improve their models despite US controls, while Washington and Beijing are already disputing whether Chinese laboratories have legitimately learned from western systems or appropriated their capabilities through large-scale distillation.
The viability of Amodei’s plan therefore rests partly on a geopolitical assumption: that the US can maintain enough distance from China to slow parts of its own frontier without allowing its rival to catch up.
That may prove correct. It is not a safety theorem.
The danger is concentration
None of this means Amodei is exaggerating the risks of advanced AI.
More capable autonomous systems may present dangers that existing security arrangements are poorly equipped to contain. Models are already being used in the work of building other models. Slowing some forms of development could genuinely make the technology safer.
The question is what political and economic structure would emerge in the process.
Under Amodei’s framework, the companies already at the frontier could retain their technological lead for longer. Compliance costs could make new challengers harder to create. Incumbents would have considerable influence over the definition of safety. Government-sanctioned co-ordination could reduce some competitive pressures between them, while export controls constrained the strongest foreign rival.
None of those consequences establishes that this is Anthropic’s intention.
Nor does it mean governments should abandon AI safety regulation. Most serious safety regimes impose costs, and large companies are usually better able to bear them. The answer is not to leave the frontier unregulated.
It is to recognise that the companies with the strongest claim to technical expertise also have the largest commercial interest in how the rules are designed.
Artificial intelligence could transform medicine, science, defence, employment, finance and education. The institutions controlling its development could consequently acquire economic and political power of an unusual kind.
Much of that control already lies with a remarkably small number of corporations.
Anthropic knows more about Claude than Congress ever will. OpenAI and Google possess expertise governments cannot quickly reproduce. Policymakers need that knowledge.
Their knowledge is indispensable. Their authority should not be.
Amodei fears that artificial intelligence may become extraordinarily powerful before society has worked out how to control it.
There is another risk: that society succeeds in slowing AI only by giving four or five corporations an enduring role in deciding how fast everyone else is allowed to go.
The question is not whether Dario Amodei should be trusted. It is whether the companies already winning the AI race should help decide where the speed limit is placed — and who is permitted to catch them.