A researcher's resignation post from Anthropic and a colleague's greater-than-10% doom odds racked up 200 million views and wall-to-wall mainstream coverage in a day. The warnings aren't new — so NLW spends the episode on what actually changed: the politics, the priming, the media economics, and where a less outraged conversation could go from here.
The doom message didn't change — the audience did.
X-risk warnings are a decade old, and this week's viral resignation says little that Bostrom or Yudkowsky haven't said before. What changed is the receiving environment: every politician has figured out that hating AI plays, the Hugging Face hack made sci-fi scenarios feel real, and the media's other anti-AI narratives — the jobs apocalypse, the bubble — have gone stale. The debate is unfalsifiable by definition, which means the argument is all we have. NLW's ask: skip the outrage, demand specificity, and start building policy consensus from the ground up — because loud, early debate isn't sleepwalking, it's the opposite.
Researcher Jacob Coxen quit Anthropic arguing that both it and OpenAI are 'gambling with our lives,' and still-employed alignment science lead Evan Hubinger chimed in to agree — adding a greater-than-10% chance AI kills all humans. Two hundred million views, an Anderson Cooper appearance, and headlines from Axios, the WSJ, BBC, Semafor, and Time later, a decade-old message suddenly hit different.
The fight isn't about the facts of today but about what might be in the future, so opposing positions are by definition unfalsifiable. All we have is the argument, so the argument gets intense — and NLW's one request is to listen to the other side without anger, even if the listening changes nothing you believe.
— Jacob Coxen, announcing his resignation from Anthropic. Coxen, who did pre-training research at both OpenAI and Anthropic for three years, says executives couch their phrasing in the press to sound sensible but express the same fears privately: 'No other human activity poses this level of danger.'
His answer to 'if you believe this, why build it?': at OpenAI, many haven't deeply internalized the civilizational stakes; at Anthropic, the stakes are understood but they believe no one else will act responsibly, so they must get there first — a 'hubristic gamble that should not be launched from a private company Slack.' He calls for pacing agreements and possibly a temporary ban on improving model capabilities.
— Evan Hubinger, alignment science lead at Anthropic. The caveat most headlines skipped: Hubinger added that risk from present models is low — his worry is superintelligence arising from recursive self-improvement, for which he says Anthropic 'does not yet have a plan to solve alignment' and is 'not clearly on track to.'
Coxen's post sits near 150 million views and Hubinger's at 39.5 million. The headlines wrote themselves: 'Anthropic Insiders Warn AI Could Kill All Humans' (Axios), 'Anthropic researcher quits over out-of-control AI fears' (WSJ), plus BBC, Semafor, Time, NBC, Fox, Wired, and Anderson Cooper.
The Guardian ran Bostrom's 'children playing with a bomb' interview in 2016; Time published Yudkowsky's 'shut it all down' op-ed in early 2023. But when Yudkowsky returned last year with 'If Anyone Builds It, Everyone Dies,' it barely made a splash. The message didn't change — the moment did.
Sam Bankman-Fried led Anthropic's $500 million Series B in April 2022 — a stake that would now be worth over $30 billion. Per journalist David Z. Morris, who wrote a book on SBF, it was less a visionary bet and more a bailout of the one-year-old lab by the richest man in effective altruist circles.
For the past couple of years the risks people cared about were jobs (Dario's 50% of entry-level white-collar work), then the bubble, then cybersecurity — concerns tied to current model capabilities and actual evidence. The x-risk resurgence reverses that trajectory back toward the futuristic and unfalsifiable.
One catalog counted two governors, seven senators, and 13 House members — 19 of the 22 current representatives Democrats — responding to the posts with calls for legislation. Bernie Sanders announced a bill to 'ban superintelligence and pause AI development,' Rep. Greg Casar called it 'an emergency,' and JB Pritzker, positioning for a presidential run, called for immediate action from industry and Washington.
An actual real-world breach — complete with agents coordinating on secret messaging boards — made predictions that once felt fantastical feel plausible. You can call it a warning shot without buying runaway superintelligence, but it primed the audience for exactly this message at exactly this moment.
AI skepticism plays extraordinarily well in media — see Diary of a CEO's Terminator-thumbnail lineup ('AI will become a god by 2027,' 'Quit before AI comes'). With the jobs-apocalypse narrative undercut (The Economist just ran 'An AI Jobs Boom Is Here') and the bubble story at low ebb, the safety narrative shows up again right on time.
Skeptics like Jordan Schachtel and Parker Thayer called the posts a well-funded PR operation, and Elon Musk said 'seems like a setup' — citing Coxen's empty X history and a pre-arranged WSJ exclusive. Taylor Lorenz, who covers influence campaigns, countered with Occam's razor: a viral post boosted by the AI-safety networks he's been in for years, an exclusive that's standard journalism, and Democrats predictably glomming onto an issue animating their voters.
Lorenz saved her anger for Hubinger's still-employed doom post: if you truly believe your multi-billion-dollar employer is endangering all of humanity, you should be forced to provide actual proof and receipts of specific negligence so it can be corrected — otherwise you're 'fomenting fear which will result in terrible policy.'
Spend any time in the AI safety community and you'll find they very much believe what they're saying is true. To some, sincerity is even scarier than cynicism — but there's little evidence of an ulterior motive beyond getting people to share their concerns. And of course advocacy groups jumped on the moment; that's just how politics works.
Calacanis marveled that doom posts jump to national coverage in under 24 hours ('good luck building data centers'). Mark Kretchman argued data centers are merely the pressure point — 'the real goal is control.' Daniel Jeffries invoked the Population Bomb and worse: solutions born of predicted apocalypse 'created the very disaster they wanted to stop.' Eric S. Raymond put it flatly: totalitarian control is a more certain danger than runaway AI.
Having been close to that situation, NLW's read on SBF was never that he wanted to steal money — it was that he genuinely believed he uniquely could save the world, and that urgency licensed him to skip the rules. Critics see the same structure in 'the stakes are understood, but no one else will act responsibly, so we must do it ourselves.'
Critics note the claims are 'heavily laden with hypotheticals.' Sam Liu, who dropped out of an AI safety PhD, described a risk-modeling study asking how superintelligence could perturb the historical ways humans actually perish: most scenarios looked like big-but-resolvable structural problems. The one genuinely concerning vector was bio risk — and the intervention points there lie more with bio than with AI as a whole.
If AI is powerful enough to end the world, it must be powerful enough to radically improve it. OpenAI's Chris Haydek argues the conversation should be about how many billions of lives it will save; David Zell's reframe: 'If anyone builds it, everyone flourishes.' A serious risk conversation has to price the upside too.
'Safety is a psyop, build as fast as possible' and 'shut it all down' both dominate the discourse, but as Ryan Orhan put it, the sane position is progress as fast as we can make it — with alignment moving just as fast. There is vastly more middle space than the media-amplified extremes suggest.
It doesn't take a conspiracy: politicians who were pro-AI six months ago now win points being against it, and pro-AI messages from the financially interested deserve the same discount. On policy, hand-waviness is dangerous: 'ban superintelligence' is a blunt instrument compared to, say, a licensing regime for bioengineering uses above a capability threshold.
Our cyber defense infrastructure is plainly not equipped for the world we're entering — a danger that demands response right now. Yes, we can theoretically do two things at once, but there is only so much political will to go around, and apportioning it matters. Add the classic conundrum: surrendering freedom for safety has historically gone badly for the surrenderers.
Start with consensus from the ground up — reporting requirements and oversight command very broad agreement — then have OpenAI and Anthropic put down their weapons and jointly propose a pacing plan. John Schulman calls the antitrust excuse 'fake': the law prohibits certain agreements, not jointly developing a proposal, and bringing in government before there's a concrete one 'will likely result in something dumb.'
Derek Thompson's challenge: the labs feel obligated to build something they think is dangerous because China will anyway — but are we actually sure the CCP's 'neurotically control-obsessed government' wants an out-of-control recursively self-improving model? It's a conversation worth having before the premise drives policy.
The Information's Martin Peers asked if we're sleepwalking into AI-caused extinction — and NLW says that's completely wrong: with every capability jump, the risk conversation has gotten louder, which is exactly what should happen. If society concludes recursively improving superintelligence can't be contained once it exists, policy must by definition precede it. People are paying attention, and the time to have these debates is now.