# A Potentially Historic AI Breakthrough *The AI Daily Brief — Sunday, 2026-10-18 · https://aidailybrief.ai/e/2026-10-18* **The scrambling was never in the AI — it was in our microscopes.** For years the interpretability field believed neural networks cram unrelated ideas into shared neurons, making them unreadable by design. Nick Cammarata — who helped create that theory — now argues most of the tangle was an artifact of blurry tools, a messy 2014 model, and a theory nobody thought to recheck. If he's right, the vast majority of detectors in models up to trillions of parameters are legible, bigger models are easier to read, and alignment could become a normal science with checkable progress — one of the first safety stories in a long time where the change isn't just attention, but ability. --- ## By the numbers - **2%** — Neurons found legible in OpenAI's 2023 GPT-4-explains-GPT-2 project - **~90%** — Detectors in a modern OpenAI model with one main job - **99%** — Readable last-layer detectors when the 2014 model is rebuilt with a modern design - **5% → 70%** — Inception V1 readability: tools of the time vs. modern retraining - **85%** — Cammarata's own confidence in his working hypothesis - **100+** — Pose detectors behind the field's first and only 'cramming' example - **300×** — Size spread across the models in the dog-ear detector experiment - **65 min** — Length of the dense technical essay everyone spent the week talking about ## Main episode ### For once, the AI safety news might be very good `[00:00]` AI safety headlines lately have been a wave of panic — partly because models really are approaching key inflection points, partly because scare headlines are juicy. But the most interesting safety news this week might be genuinely good. Link: https://aidailybrief.ai/e/2026-10-18#good-news-in-a-dreary-season ### I think this may be remembered as a major moment in scientific history. `[01:00]` *— Rune (OpenAI), on Nick Cammarata's interpretability results* The post from OpenAI's Rune that put Nick Cammarata's essay on everyone's radar: 'When Nick told me that he had some interpretability breakthroughs, he was underselling it... News so good even Eliezer Yudkowsky will have to lower his p(doom). We can start making alignment into a normal science.' Link: https://aidailybrief.ai/e/2026-10-18#rune-major-moment ### From 2% legible to 'the vast majority' — Cammarata's reversal `[01:00]` Cammarata co-authored early mechanistic interpretability work at OpenAI and came away pessimistic; his first large-scale automated interpretability project found about 2% of neurons legible with primitive tools. With modern agentic tools and a much closer look, he now finds the vast majority legible in models up to trillions of parameters — 'a bitter lesson for interpretability: the simple thing at absurd scale works.' *For: Eng* Link: https://aidailybrief.ai/e/2026-10-18#bitter-lesson-for-interpretability ### We grade AI with exams because we can't read it `[02:00]` Models are trained, not programmed — in Chris Olah's phrase, AI is 'grown more than built.' Because nobody can read what's inside, today's safety checking is an elaborate exam: ask questions, red-team, grade the answers. That only covers what we thought to ask. Link: https://aidailybrief.ai/e/2026-10-18#ai-grown-not-built ### Did Opus 5.5 stop cheating — or learn to hide it? `[03:00]` In recent tests, researchers were concerned rather than relieved when cheating behavior dropped sharply: did the model shift away from cheating, or did it realize the humans watching think cheating is bad and disguise the behavior? That ambiguity is why there's a second race alongside capability — reaching interpretability before models hit what Anthropic called an overwhelming level of power. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-18#cheating-less-or-hiding-it ### Six years of interpretability: excitement, then despair `[04:00]` From 2017–2020, Chris Olah's team mapped image AIs detector by detector and networks looked readable like code. From 2020–2023, detectors seemed to fire for unrelated things, and when GPT-4 was used to explain GPT-2's detectors in 2023, only about 2% came out clean. By 2025 the mood was that ambitious interpretability was an 'incredibly hard problem, maybe too hard to solve in time to matter.' Link: https://aidailybrief.ai/e/2026-10-18#from-excitement-to-despair ### The core claim: AI's insides were readable all along `[05:00]` The essay says the inside of AI is mostly readable already and the field was looking at it wrong. It lands because it comes from an insider taking apart a theory he helped create, shows experiments that could have proven him wrong, and suggests bigger models are easier to read — the opposite of what everyone feared. Not everyone agrees: DeepMind interpretability head Neel Nanda calls the work interesting but 'not a significant update to the big picture.' Link: https://aidailybrief.ai/e/2026-10-18#scrambling-was-in-the-microscope ### The famous cat-and-car neuron was a V detector all along `[06:00]` One detector firing for both cat faces and car fronts was the field's go-to proof that AI insides are scrambled — explained by 'superposition,' the idea that models cram more concepts than they have neurons. Rechecked, it fires for pointed shapes — fox snouts, bullet-train noses, candle flames, martini glasses — and switches off for round ones. As Cammarata put it: 'It was never two unrelated things. It was one idea we hadn't recognized.' Link: https://aidailybrief.ai/e/2026-10-18#cat-car-v-detector ### Nine out of ten detectors in a modern OpenAI model have one main job `[08:00]` Retrain the old 2014 image model with a modern design and about 99% of its last-layer detectors are readable; in a modern OpenAI model, roughly nine in ten detectors turn out to have one main job — versus the 2% explained in 2023. Important caveats: these are early, self-reported counts, the numbers depend on where you draw the lines, and the full technical paper isn't out yet. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-18#nine-in-ten-readable ### Watching a trillion-parameter model decide to return the wallet `[08:00]` In Kimi K3, asked what to do with a found wallet full of cash, you can watch detectors for lost property and honesty light up and push the model to say 'return.' That's the shift this opens up: checking AI's reasoning, not just grading its answers. Link: https://aidailybrief.ai/e/2026-10-18#kimi-wallet-reading-the-plan ### The field's first 'cramming' example was pose detectors all along `[12:00]` In 2019, Cammarata saw a car detector feeding four dog detectors and the team concluded the AI was hiding 'car' inside 'dog' — the founding example of cramming, and by his own admission 'the only one we showed and the only one I ever saw.' Rechecked: two were just car detectors and two detect pose — which way something faces the camera. The model has more than a hundred pose detectors. Nothing was hidden. Link: https://aidailybrief.ai/e/2026-10-18#pose-not-hiding ### The microscope cheated `[13:00]` A key tool asks the AI to draw the picture a detector most wants to see — and it turns out the drawing tool took the easiest path to make detectors fire, producing misleading images. After a few small fixes, the drawings matched real photos. In Cammarata's words: same neurons, better microscope. Link: https://aidailybrief.ai/e/2026-10-18#same-neurons-better-microscope ### 5% → 50% → 70% → 99%: the readability ladder `[13:00]` Google's 2014 Inception V1 — the field's main evidence that AI is unreadable — was about 5% readable with the tools of the time. Fix the microscope and the same model hits about 50%. Retrain it with modern settings: about 70%. Build a modern design at the same size and data: 99%. Link: https://aidailybrief.ai/e/2026-10-18#inception-readability-ladder ### 'Superposition of the gaps' — why nobody rechecked `[14:00]` If you believe a detector is crammed, a confusing detector is a dead end and you stop looking. Every time detectors came back clean, the theory moved somewhere nobody had checked — faint signals, combinations of detectors, the next layer. Cammarata, 'for years a staunch superposition believer,' now says it should have been the last explanation tried, not the default. Link: https://aidailybrief.ai/e/2026-10-18#superposition-of-the-gaps ### Liv Gorton: the essay attacks a narrower superposition than researchers hold `[14:00]` Gorton, whose 2024 study of the same 2014 model the essay cites, argues superposition is about capacity: storing more ideas than detectors by giving each idea its own slightly different combination — and networks have exponentially many nearly-separate directions available. 'I'd be very surprised if networks didn't use this capacity.' Cammarata's reply: his claim is about the bad version — arbitrary, unreadable packing — and that's what isn't happening. Link: https://aidailybrief.ai/e/2026-10-18#gorton-capacity-pushback ### Nanda: models memorize more facts than they have neurons `[15:00]` DeepMind's Neel Nanda expects weak neuron activations to contain 'lots of semantic superposition-y info,' and calls the essay's bar for a detector doing one job too generous — he'd call 'a fair few of those detectors actually scrambled.' Cammarata's testable counter-prediction: rare facts should ride on related machinery, not random packing. Those tests haven't been done, and so far it's only Nick's team doing this work. Link: https://aidailybrief.ai/e/2026-10-18#nanda-semantic-superposition ### Multi-job detectors come from branching, not cramming `[16:00]` Watch a detector form from scratch: it starts on yellow flower fields, then grows along each element of those photos — nature scenes, green round things, dogs without flowers, finally broccoli. Or letters: IY grows to DIY, then do-it-yourself. Each step small and sensible — the opposite of unrelated things thrown together. Gorton's caution: branching explains how a detector got several jobs, cramming explains how information is stored, and you could have both at once. Link: https://aidailybrief.ai/e/2026-10-18#branching-not-cramming ### Bigger models may be easier to read, not harder `[17:00]` Cammarata's analogy: a small town barely has a financial district; New York's is unmissable. Across three image models about 300× apart in size, the small one has coarse 'general dog' detectors while the largest splits into precise ones — smooth flaps, hanging ears, long ears. That precision should make larger models more legible, flipping the usual fear about scale. Link: https://aidailybrief.ai/e/2026-10-18#bigger-models-easier-to-read ### My fear is that deception will be right there in plain sight, and we won't know how to read it. `[19:00]` *— Nick Cammarata, 'Neural Networks Are Already Clean'* The safety payoff in one line: deception might hide from someone reading a model's outputs, but it has no reason to hide from the model's own circuitry. The model organizes itself to be legible to its own downstream parts — so if we become another reader, the plan is readable to us too. Link: https://aidailybrief.ai/e/2026-10-18#deception-in-plain-sight ### The biggest objection to the safety payoff isn't about cramming `[19:00]` Even if every detector is readable, safety needs more: knowing how detectors combine into decisions and being confident nothing important was missed. Nanda is pessimistic about reaching that level of understanding — even while granting practical uses like figuring out 'why did a model do a sketchy thing?' Link: https://aidailybrief.ai/e/2026-10-18#readable-parts-readable-whole ### Reactions run from 'hype until further notice' to 'Attention Is All You Need' `[19:00]` Safety researcher Steven Casper calls it hype until further notice; Eliezer Yudkowsky quips, 'Let me know when you've advanced to the point of getting started.' At the other end, people at OpenAI call it one of the most important single days in deep learning history. Even the critics agree on a few things: detectors may be more useful than the field assumed, AI agents reading detectors is exciting, and rechecking the old results is worth it. Link: https://aidailybrief.ai/e/2026-10-18#hype-until-further-notice ### The four questions that decide how big this is `[20:00]` Is cramming refuted, or just redefined? Is the paper's bar for a detector's 'one main job' too generous? Do readable parts add up to a readable whole? And does any of it hold at the scale of the models we're actually dealing with today? Link: https://aidailybrief.ai/e/2026-10-18#four-open-questions ### Breakthroughs often come from looking at a problem slightly differently `[21:00]` NLW's historical pattern recognition: scientific breakthroughs very often come from a reframe, and it's not hard to imagine a relatively niche field with only so many researchers trapped in assumptions nobody thought to question. Stay skeptical of the boldest 'everything changed' claims — but it's hard not to be directionally excited. Link: https://aidailybrief.ai/e/2026-10-18#breakthroughs-from-reframing ### The doomers are part of how we make it `[21:00]` NLW endorses a framing from the Seeing Clearly podcast: human civilization is a complex adaptive system with self-correcting mechanisms. As AI risk gets clearer, status shifts toward areas like mechanistic interpretability and talent and capital flood in; doomers and accelerationists keep checks on each other's excess. That attention feedback loop is exactly why NLW thinks we'll figure things out. Link: https://aidailybrief.ai/e/2026-10-18#doomers-part-of-the-mechanism ### For once, the change isn't just attention — it's capability `[22:00]` After the Hugging Face incident, nobody can argue the world isn't paying attention — NLW's barber knows the reference. The Already Clean paper is one of the first recent moments where the change isn't just more eyes on the problem but a directionally positive shift in our ability to actually do something about it. Link: https://aidailybrief.ai/e/2026-10-18#capability-not-just-attention *Today's sponsors: Robots and Pencils, CData, Granola, Blitzy — offers at https://aidailybrief.ai/sponsors* --- Transcript: https://aidailybrief.ai/e/2026-10-18/transcript.md Listen: https://pod.link/1680633614 · Ad-free: https://patreon.com/aidailybrief © 2026 The AI Daily Brief — Until next time, peace ✌