# Anthropic Researcher Says AI Has Over a 10% Chance of Killing All Humans
*The AI Daily Brief — Thursday, 2026-09-10 · https://aidailybrief.ai/e/2026-09-10*

**The doom message didn't change — the audience did.**

X-risk warnings are a decade old, and this week's viral resignation says little that Bostrom or Yudkowsky haven't said before. What changed is the receiving environment: every politician has figured out that hating AI plays, the Hugging Face hack made sci-fi scenarios feel real, and the media's other anti-AI narratives — the jobs apocalypse, the bubble — have gone stale. The debate is unfalsifiable by definition, which means the argument is all we have. NLW's ask: skip the outrage, demand specificity, and start building policy consensus from the ground up — because loud, early debate isn't sleepwalking, it's the opposite.

---

## By the numbers
- **>10%** — Anthropic alignment lead's stated odds AI kills all humans within a decade
- **150M** — Views on Jacob Coxen's resignation post at recording time
- **39.5M** — Views on Evan Hubinger's 'greater than 10%' reply
- **22** — Sitting US politicians who responded with calls for AI legislation
- **19 of 22** — Congressional responders who were Democrats
- **$500M** — SBF's 2022 lead of Anthropic's Series B — a stake now worth $30B+
- **<24h** — From X post to national coverage, per Jason Calacanis
- **200M** — Combined views across the viral doom posts

## Main episode

### An Anthropic resignation detonates across the internet `[00:00]`
Researcher Jacob Coxen quit Anthropic arguing that both it and OpenAI are 'gambling with our lives,' and still-employed alignment science lead Evan Hubinger chimed in to agree — adding a greater-than-10% chance AI kills all humans. Two hundred million views, an Anderson Cooper appearance, and headlines from Axios, the WSJ, BBC, Semafor, and Time later, a decade-old message suddenly hit different.
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#anthropic-resignation-goes-viral

### This debate is unfalsifiable — which is exactly why it gets so hot `[01:00]`
The fight isn't about the facts of today but about what might be in the future, so opposing positions are by definition unfalsifiable. All we have is the argument, so the argument gets intense — and NLW's one request is to listen to the other side without anger, even if the listening changes nothing you believe.
Link: https://aidailybrief.ai/e/2026-09-10#resist-the-outrage

### They are racing straight to self-improving superintelligence and gambling with our lives. `[03:00]`
*— Jacob Coxen, announcing his resignation from Anthropic*
Coxen, who did pre-training research at both OpenAI and Anthropic for three years, says executives couch their phrasing in the press to sound sensible but express the same fears privately: 'No other human activity poses this level of danger.'
Link: https://aidailybrief.ai/e/2026-09-10#racing-to-superintelligence-gambling-with-our-lives

### Coxen's diagnosis: OpenAI hasn't internalized the stakes; Anthropic races anyway `[04:00]`
His answer to 'if you believe this, why build it?': at OpenAI, many haven't deeply internalized the civilizational stakes; at Anthropic, the stakes are understood but they believe no one else will act responsibly, so they must get there first — a 'hubristic gamble that should not be launched from a private company Slack.' He calls for pacing agreements and possibly a temporary ban on improving model capabilities.
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#coxens-diagnosis-of-the-labs

### We really do earnestly believe AI could kill all humans! I personally think it is greater than 10% within the next decade. `[05:00]`
*— Evan Hubinger, alignment science lead at Anthropic*
The caveat most headlines skipped: Hubinger added that risk from present models is low — his worry is superintelligence arising from recursive self-improvement, for which he says Anthropic 'does not yet have a plan to solve alignment' and is 'not clearly on track to.'
Link: https://aidailybrief.ai/e/2026-09-10#hubinger-ten-percent

### From X post to every major outlet in a news cycle `[06:00]`
Coxen's post sits near 150 million views and Hubinger's at 39.5 million. The headlines wrote themselves: 'Anthropic Insiders Warn AI Could Kill All Humans' (Axios), 'Anthropic researcher quits over out-of-control AI fears' (WSJ), plus BBC, Semafor, Time, NBC, Fox, Wired, and Anderson Cooper.
*For: Marketing*
Link: https://aidailybrief.ai/e/2026-09-10#x-to-anderson-cooper

### None of this is new — the drum has been beaten for a decade `[07:00]`
The Guardian ran Bostrom's 'children playing with a bomb' interview in 2016; Time published Yudkowsky's 'shut it all down' op-ed in early 2023. But when Yudkowsky returned last year with 'If Anyone Builds It, Everyone Dies,' it barely made a splash. The message didn't change — the moment did.
Link: https://aidailybrief.ai/e/2026-09-10#doom-drum-is-a-decade-old

### The SBF-Anthropic history is less visionary than remembered `[07:00]`
Sam Bankman-Fried led Anthropic's $500 million Series B in April 2022 — a stake that would now be worth over $30 billion. Per journalist David Z. Morris, who wrote a book on SBF, it was less a visionary bet and more a bailout of the one-year-old lab by the richest man in effective altruist circles.
*For: Finance*
Link: https://aidailybrief.ai/e/2026-09-10#sbf-anthropic-bailout

### AI risk talk had been migrating from vague to specific — then snapped back `[08:00]`
For the past couple of years the risks people cared about were jobs (Dario's 50% of entry-level white-collar work), then the bubble, then cybersecurity — concerns tied to current model capabilities and actual evidence. The x-risk resurgence reverses that trajectory back toward the futuristic and unfalsifiable.
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#risk-narratives-were-getting-specific

### Every politician figured out that hating AI plays `[13:00]`
One catalog counted two governors, seven senators, and 13 House members — 19 of the 22 current representatives Democrats — responding to the posts with calls for legislation. Bernie Sanders announced a bill to 'ban superintelligence and pause AI development,' Rep. Greg Casar called it 'an emergency,' and JB Pritzker, positioning for a presidential run, called for immediate action from industry and Washington.
*For: Legal, Exec*
Link: https://aidailybrief.ai/e/2026-09-10#hating-ai-plays

### The Hugging Face incident made the sci-fi feel less sci-fi `[15:00]`
An actual real-world breach — complete with agents coordinating on secret messaging boards — made predictions that once felt fantastical feel plausible. You can call it a warning shot without buying runaway superintelligence, but it primed the audience for exactly this message at exactly this moment.
*For: Eng*
Link: https://aidailybrief.ai/e/2026-09-10#hugging-face-primed-the-pump

### Being a doomer is a way better business model `[15:00]`
AI skepticism plays extraordinarily well in media — see Diary of a CEO's Terminator-thumbnail lineup ('AI will become a god by 2027,' 'Quit before AI comes'). With the jobs-apocalypse narrative undercut (The Economist just ran 'An AI Jobs Boom Is Here') and the bubble story at low ebb, the safety narrative shows up again right on time.
*For: Marketing*
Link: https://aidailybrief.ai/e/2026-09-10#doom-is-a-better-business-model

### 'Coordinated op' or just how group chats work? `[17:00]`
Skeptics like Jordan Schachtel and Parker Thayer called the posts a well-funded PR operation, and Elon Musk said 'seems like a setup' — citing Coxen's empty X history and a pre-arranged WSJ exclusive. Taylor Lorenz, who covers influence campaigns, countered with Occam's razor: a viral post boosted by the AI-safety networks he's been in for years, an exclusive that's standard journalism, and Democrats predictably glomming onto an issue animating their voters.
Link: https://aidailybrief.ai/e/2026-09-10#astroturf-theory-vs-occams-razor

### The sharpest critique: prove it or you're just vague-posting `[19:00]`
Lorenz saved her anger for Hubinger's still-employed doom post: if you truly believe your multi-billion-dollar employer is endangering all of humanity, you should be forced to provide actual proof and receipts of specific negligence so it can be corrected — otherwise you're 'fomenting fear which will result in terrible policy.'
*For: Legal*
Link: https://aidailybrief.ai/e/2026-09-10#lorenz-demands-receipts

### The 'it's just marketing' theory doesn't survive contact with these people `[19:00]`
Spend any time in the AI safety community and you'll find they very much believe what they're saying is true. To some, sincerity is even scarier than cynicism — but there's little evidence of an ulterior motive beyond getting people to share their concerns. And of course advocacy groups jumped on the moment; that's just how politics works.
Link: https://aidailybrief.ai/e/2026-09-10#its-not-marketing

### The backlash camp: the cure is worse than the disease `[20:00]`
Calacanis marveled that doom posts jump to national coverage in under 24 hours ('good luck building data centers'). Mark Kretchman argued data centers are merely the pressure point — 'the real goal is control.' Daniel Jeffries invoked the Population Bomb and worse: solutions born of predicted apocalypse 'created the very disaster they wanted to stop.' Eric S. Raymond put it flatly: totalitarian control is a more certain danger than runaway AI.
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#the-control-backlash

### The messianic complex rhymes with SBF `[23:00]`
Having been close to that situation, NLW's read on SBF was never that he wanted to steal money — it was that he genuinely believed he uniquely could save the world, and that urgency licensed him to skip the rules. Critics see the same structure in 'the stakes are understood, but no one else will act responsibly, so we must do it ourselves.'
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#the-sbf-messiah-parallel

### The specificity problem: nobody maps the actual path to extinction `[24:00]`
Critics note the claims are 'heavily laden with hypotheticals.' Sam Liu, who dropped out of an AI safety PhD, described a risk-modeling study asking how superintelligence could perturb the historical ways humans actually perish: most scenarios looked like big-but-resolvable structural problems. The one genuinely concerning vector was bio risk — and the intervention points there lie more with bio than with AI as a whole.
*For: Eng*
Link: https://aidailybrief.ai/e/2026-09-10#no-tangible-pathway

### Where's the P(boom)? `[26:00]`
If AI is powerful enough to end the world, it must be powerful enough to radically improve it. OpenAI's Chris Haydek argues the conversation should be about how many billions of lives it will save; David Zell's reframe: 'If anyone builds it, everyone flourishes.' A serious risk conversation has to price the upside too.
*For: Marketing, Exec*
Link: https://aidailybrief.ai/e/2026-09-10#p-boom-not-just-p-doom

### Two insane extremes are crowding out a large middle `[27:00]`
'Safety is a psyop, build as fast as possible' and 'shut it all down' both dominate the discourse, but as Ryan Orhan put it, the sane position is progress as fast as we can make it — with alignment moving just as fast. There is vastly more middle space than the media-amplified extremes suggest.
Link: https://aidailybrief.ai/e/2026-09-10#the-middle-space

### Read every take through its incentives — then demand specifics `[28:00]`
It doesn't take a conspiracy: politicians who were pro-AI six months ago now win points being against it, and pro-AI messages from the financially interested deserve the same discount. On policy, hand-waviness is dangerous: 'ban superintelligence' is a blunt instrument compared to, say, a licensing regime for bioengineering uses above a capability threshold.
*For: Legal, Exec*
Link: https://aidailybrief.ai/e/2026-09-10#incentives-and-specificity

### Future theoreticals crowd out clear and present dangers `[30:00]`
Our cyber defense infrastructure is plainly not equipped for the world we're entering — a danger that demands response right now. Yes, we can theoretically do two things at once, but there is only so much political will to go around, and apportioning it matters. Add the classic conundrum: surrendering freedom for safety has historically gone badly for the surrenderers.
*For: Legal, Eng*
Link: https://aidailybrief.ai/e/2026-09-10#theoreticals-crowd-out-the-present

### A single Altman-Amodei photo op would beat 10,000 Twitter debates `[31:00]`
Start with consensus from the ground up — reporting requirements and oversight command very broad agreement — then have OpenAI and Anthropic put down their weapons and jointly propose a pacing plan. John Schulman calls the antitrust excuse 'fake': the law prohibits certain agreements, not jointly developing a proposal, and bringing in government before there's a concrete one 'will likely result in something dumb.'
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#one-photo-op-beats-ten-thousand-debates

### If 'China will build it anyway' is the justification, we'd better be sure `[32:00]`
Derek Thompson's challenge: the labs feel obligated to build something they think is dangerous because China will anyway — but are we actually sure the CCP's 'neurotically control-obsessed government' wants an out-of-control recursively self-improving model? It's a conversation worth having before the premise drives policy.
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#are-we-actually-sure-about-china

### The loudness of this debate is the opposite of sleepwalking `[33:00]`
The Information's Martin Peers asked if we're sleepwalking into AI-caused extinction — and NLW says that's completely wrong: with every capability jump, the risk conversation has gotten louder, which is exactly what should happen. If society concludes recursively improving superintelligence can't be contained once it exists, policy must by definition precede it. People are paying attention, and the time to have these debates is now.
*For: Exec*
Link: https://aidailybrief.ai/e/2026-09-10#this-is-not-sleepwalking

*Today's sponsors: KPMG, Blitzy, Harbor, Hyperagent — offers at https://aidailybrief.ai/sponsors*

---
Transcript: https://aidailybrief.ai/e/2026-09-10/transcript.md
Listen: https://pod.link/1680633614 · Ad-free: https://patreon.com/aidailybrief
© 2026 The AI Daily Brief — Until next time, peace ✌