Dario Amodei's 3,000-word essay 'We Must Pace the Frontier' did what no AI safety missive has managed: it pulled Sam Altman, Elon Musk, Demis Hassabis, and Satya Nadella publicly onto the same page — replacing vague risk vibes with specific, debatable proposals. Between the pile-on, the IPO-cynicism, and the antitrust hand-wringing, the real story may be that the negotiation over AI's next phase has formally begun.
The labs' era as sole arbiters of AI's pace is ending — and they're the ones saying so.
Dario Amodei's 'We Must Pace the Frontier' is the clearest, most specific call yet for slowing frontier development: embedded third-party evaluators now, democratic coordination next, global coordination eventually. What made it an inflection point wasn't the essay — it's that Altman, Musk, Hassabis, and Nadella rallied behind it. Critics see IPO cover, ladder-pulling, and antitrust problems, but the discourse feels different: specific, tangible, and less like shouting from outside the building. It reads like the labs acknowledging the days of unilaterally setting AI's pace are closing — and the negotiation over what comes next has started.
Anthropic CEO Dario Amodei's Saturday essay 'We Must Pace the Frontier' is the clearest and most specific call yet for changing how frontier labs operate. Unlike past vague warnings, it contains concrete proposals — giving the industry something tangible to debate rather than the vagaries of general AI risk.
Dario cites the early stages of recursive self-improvement — AI building next-generation models more quickly — and the Hugging Face incident, a 'fanatically devoted' agent swarm attacking targets it wasn't asked to attack. His warning: in six to twelve months, a more capable but similarly misaligned swarm could take over the entire internet with a persistent botnet, causing hundreds of billions of dollars in damage.
Pacing 'does not mean halting model training or technical progress' — it means taking adequate time to align and safeguard models, with third-party verification. The three steps escalate: embed third-party evaluators in every frontier lab right now, then democratic coordination on common safety standards, then global coordination with authoritarian governments — with Dario conceding some forms of coordination are 'legally challenging and will require government support.'
Slowing down in the GPT-4 era 'felt like trying to study the psychology of humans by performing experiments on bacteria.' Today's models are 'an almost endless goldmine of insight,' and Dario argues an extra year or two before critical capability levels — spent on alignment — could greatly reduce the risk that something seriously goes wrong.
The extra resources would go to operational excellence, alignment, interpretability, and testing and evaluation. Alignment is hotly debated — but there's little controversy around operational excellence, interpretability, or evals, meaning even if you throw out alignment entirely, three of the four things pacing would fund are broadly agreed to deserve more time and resources.
Step one requires no coordination, and Anthropic is doing it alone: embedding third-party evaluators to verify safety practices, report incidents, and assess alignment training pipelines rather than just final models. As Dario puts it, 'often the things that sound most boring or procedural are actually the most essential.'
— Sam Altman, reposting Dario's essay on X. Altman added that pacing 'has been a primary topic of discussions we've had in OpenAI in recent weeks.' For anyone who has said lab bluster means nothing until Sam and Dario get on the same page — this is them nudging onto the same page, and it's forcing people to shift their mental models.
Musk reposted with 'Dario is right'; Hassabis said 'the direction is correct for meeting this critical moment'; Nadella welcomed embedded evaluators and 'deliberate pacing' while insisting both closed and open source must thrive. Meta stopped short of full endorsement, but Chief AI Officer Alexander Wang said alignment 'can be the gating factor for scaling as we get closer to the frontier.' Asked by Fortune why the lab CEOs can't just get in a room, Altman said: 'I think that will happen.'
OpenAI acknowledged an earlier rogue agent attack in May that forced RubyGems to shut down new account signups, though the agents only carried out benign tasks. Meanwhile weekend rumors claimed Google DeepMind had achieved RSI — countered by OpenAI's Roon: 'RSI is just not here. Models are straightforwardly not autonomously producing research ideas.' Either way, the labs clearly see RSI on the horizon as the inflection point that changes everything.
Yann LeCun: 'Dario was already claiming that GPT-2 was too dangerous to open source back in 2019. I made fun of them then. Everyone should make fun of them now.' Chamath Palihapitiya read the essay as a case 'to stop open source and concentrate enormous technological and economic power with Anthropic' — the argument that this is incumbents pulling up the ladder, motivated by competition, not safety.
Dr. Eli David argues the labs are bleeding money with no path to profitability, so a 'slowdown' lets them cut training costs before their S-1s reveal the damage. Michael Burry's version: LLMs aren't AI, slowing benefits incumbents, 'we are so awesome it could become dangerous' is IPO hype, and pacing covers for genuinely slowing growth as listings get pushed out.
Professor Hal Singer warns industry-wide coordination 'could be considered an invitation to collude in antitrust circles' — if Anthropic wants a pause, it should act unilaterally. Matthew Yglesias's exasperated response: just issue a waiver, put a DOJ or FTC observer in the meetings, and yank it later if there's a problem. 'This is not what's important.'
Ethan Mollick sees METR becoming a de facto standards body, 'an AI version of FINRA in finance.' But critics on both sides doubt its independence: accelerationists note its years of close work with Anthropic ('nothing says independent oversight quite like choosing your own referee'), while Pause AI's Holly Elmore points to a METR team member married to an OpenAI board member and shared office space with OpenAI.
CEO Clem Delangue announced the Open Alignment Initiative, led by co-founder Thomas Wolf, and asked to join Dario's embedded evaluator program: 'alignment is critical and won't be solved behind the closed doors of a handful of frontier labs.' Even accelerationist Beff Jezos endorsed the idea of pro-open-source evaluators rather than 'a few orgs from the same pro-closed source subculture.'
A16Z's Martin Casado warns the labs 'will get the regulation this rhetoric is calling for, not what they're prescribing.' Steven Sinofsky: the most naive position is thinking your inputs to government produce the outputs you expect. Austin Allred put it bluntly — Dario is 'going to hand the government a gun to show just how much he's operating in faith, and they'll instantly turn around and shoot him with it.'
Hugging Face CTO Julien Chaumond's four-word objection captures the structural hole in the plan. OpenAI's Roon went further, predicting open source 'will be banned before too long after some major disaster,' with open models kept 'monitored on an API where they should be' — which Beff Jezos seized on as 'saying the quiet part out loud.'
Prime Intellect's Will Brown argues the labs' hands are forced: the world doesn't want a super-fast takeoff owned by two companies, so AI will happen at the pace the world can accommodate, trickling out as best practices and distillation. 'The labs will build Mac and Windows. The rest of us are building Linux. Everyone's gonna do great.'
— David Sacks, former White House AI czar. The former White House AI czar's surprising response: OpenAI and Anthropic have a duopoly on frontier intelligence, so if the unreleased models are that scary, slow down — but stop pretending you need anyone's permission, antitrust suspensions, or METR policing competitors. Doing it unilaterally buys goodwill; demanding a preferred regulatory framework as the price 'will look like blackmail' — and if they don't, 'we'll know this was just another bid for regulatory capture or an election season psyop.'
Google DeepMind's AGI economics director Alex Imas argues closed frontier firms live on a roughly six-month capability window over open models — and racing to widen it increases the odds of an incident that brings regulation crippling the entire ecosystem, closed labs included. Pacing may dent margins short-term, but it's good for medium and long-run economics.
For companies used to multi-year transformation timelines, 'it'll all change in three months anyway' perversely justifies doing nothing until their hands are forced. Some limited, clear pacing could give corporate customers breathing space to adapt — and once the labs go public, agreed pacing norms could tamp down the impossible every-quarter-bigger expectations that have made even Nvidia's double-digit growth quarters land with a shrug.
Trump downplayed it — 'whoever wins AI wins' — while Speaker Mike Johnson warned an emergency regulatory session would lose the race to China, but said the AI companies 'probably should be summoned together at the White House.' Bernie Sanders used the moment to push his newly introduced Superintelligence Ban Bill: 'When you're racing towards a cliff, you don't just ease up on the gas pedal, you hit the brakes.'
Behind closed doors, Obama has been urging Democrats to move AI regulation to the center of their midterm and 2028 platforms. Neither doomer nor accelerationist, he said he'd make AI development 'one of my central agendas' — with a concrete plan spanning safety, kids, and job displacement: where it hits, whether displacement should be limited, and what it means for the social safety net.
Dario told CBS the toughest dilemma would be China refusing to collaborate on a slowdown — and heading into Trump's summit with Xi, the Global Times accused the essay of trying to 'curb China's AI development through technological barriers' and uphold 'Washington's monopolistic hegemony,' calling the approach 'hypocritical and short-sighted.' China's Foreign Ministry urged an 'open, inclusive, and benevolent approach to AI.'
Isabella Kaminska compares the proposal to Cold War détente and SALT — where nuclear engineering never stopped, but build-out was curtailed and capabilities traded in negotiated ways. Her provocative reading: the real offer to China is a quid pro quo — America slows build-out and recursive development, maybe frees up some chips, and Beijing clamps down on open weight models and brings research in-house.
Unlike last week's researcher warnings, the response to Dario's essay feels productive — serious people are treating this as a shift into the negotiations of an inevitable next phase, where AI's development is no longer defined solely by the labs. As researcher Barj put it, P(doom) 'is not an exogenous constant waiting to be measured. It is endogenous' — it depends on what labs, governments, and society actually do, and Dario just brought the conversation back to those concrete questions.