# The Right Way to Worry About AI — Transcript (2026-08-07) https://aidailybrief.ai/e/2026-08-07 · Listen: https://pod.link/1680633614 --- 260807 in_EDIT: [00:00:00] Today on the Today on the AI Daily Brief, The right way to worry about AI. Before that in the headlines, markets, models, and more. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, Blitzy, Robots and Pencils, and Hyperagent. To get an ad-free version of the show, go to patreon.com/aidailybrief, or you can subscribe on Apple Podcasts 260807 in_EDIT: Quick note there, by the way, Apple Podcasts seems to have been having some trouble this week We haven't been having any particular delays as we sometimes do with the ad-free version going up on Apple, but I've had some people days later still seeing the ad version. The best that I can suggest is to completely close out of and restart your Apple app. but in any case, I apologize for the pain Last note, one more reminder to go check out the AI Summer Adventure It's a set of free self-directed projects to expand your AI horizons, and you can find it all at summeradventure.ai finally, as always, [00:01:00] if you are looking to sponsor the show, send us a note at sponsors@aidailybrief.ai. But with all that out of the way, let's dive in 260807 hed_EDIT: Welcome back to the AI We kick off today with some OpenAI news. Well, a little bit of speculation and then some real news. The leakers are starting to suggest that the next new model, Astro, which was of course the one that did those novel math proofs that we discussed last week, seems to be imminently launching, with some saying that they're even targeting next week What we know for sure is that even as they are releasing new models, OpenAI is also thinking very much and trying to compete very much on the cost front as well The company announced that they're giving free users unlimited chats as part of a service overhaul for the GPT 56 model family. The free user tier will now be served with GPT 56 Luna, replacing the instant model range. free users will also now have a think button to allow for greater reasoning from Luna Theoretically, this closes some of the experience gap for free users, allowing them to access the same model as paid users, albeit the smaller version. In addition, usage is now unlimited, so free users can use ChatGPT as much as they want. For paid subscribers, GPT 5.6 Soul will now become the [00:02:00] default chat model. OpenAI said that this should improve the experience over GPT 5.5 Instant, with Soul making fewer factual mistakes and avoiding extra detail when it doesn't help. Finally, Plus and Pro subscribers will now have a new effort slider and thinking mode to provide more intuitive controls over reasoning effort While some like Jumpers write, how is that even profitable?" Ken she on X says, " The move to make OpenAI Luna free is obviously not out of generosity. The free tier is a strategic distribution channel. They're trying to hook people and get them to upgrade to Go or Plus, or of course, make money through ads." ads." Now, Now, speaking of our discourse of cheaper models, according to the information, Stripe is indeed moving forward with their OpenRouter acquisition the news outlet reports that Stripe has entered exclusive talks to buy out the model routing startup for close to the $10 billion that was previously reported. Earlier reports had suggested a bidding war with Stripe in the lead, but this suggests that Open Router has taken themselves off the market and will enter the negotiation phase with this one specific partner Certainly the deal could still fall apart, but exclusive talks do suggest that it's moving to the next level level Meanwhile, Meanwhile, everywhere [00:03:00] we areseeing the impact of the supply chain crunch as the world uses more and more compute for more and more AI The Information again reports that NVIDIA is considering slashing the specs on Ruben Currently, NVIDIA has three different variants of their next generation of flagship GPU, the Ruben Ultra, under testing. And sources said that some of the test units include less memory than originally announced Those sources said that NVIDIA is considering releasing the lower memory versions partly due to concerns that they won't be able to secure enough high bandwidth memory for the production run Now, how big the implications of this are remain to be seen. Even with the reduced memory, the chips should still be able to serve inference for the latest generation of ultra-large models like Claude Fable However, memory limits could put a cap on the ability to keep scaling model size Now atthis point, NVIDIA has so far denied any issue with sourcing enough memory. In mid-July, Senior VP of Hardware Engineering Andrew Bell said, " We were in front of the memory problem, so it's not gonna hold us back anytime soon. The pricing, of course, is a problem for the whole world, and probably the pricing will be the bigger challenge. But for supply, we're in shape." NVIDIA also wasn't scheduled to ship the new chips until late next year, so there's still time to [00:04:00] resolve supply issues Still, we are at the point where we may be starting to see fundamental hardware limitations Potentially start to slow down the pace of model improvement Making Making sure investors didn't get it twisted, Mike Sulka wrote, " This isn't weak AI demand. This is memory rationing at the top of the food chain." food chain." Now, one Now, one of the reasons to think that memory shortages and other component shortages are gonna get nothing but worse is the fact that we are still at the very beginning of understanding the world's total demand for AI. One bet that some companies like OpenAI are making is that a new generation of consumer devices will help unleash all of that demand. Bloomberg has more reports about OpenAI's first device, describing it as essentially a smart speaker without a display. It's battery-powered, roughly the size of a hockey puck, and shaped like a donut, with the intention of making it easy to carry around the home in one hand It will have a high quality brushed metal finish Similar to the finish on iPhones, which of course formed part of the trade secrets lawsuit as OpenAI is working with an Apple supplier. The sources suggested that the device will have some small moving parts and lights intended to provide a visual indicator that it's interacting with the [00:05:00] user. It will include a camera, microphones, and other sensors designed to take in surroundings The biggest new news is the target price, with OpenAI aiming to bring the device to market Between three and four hundred dollars. By way of comparison, the most expensive smart speakers, such as Amazon's Echo Studio, are retailing for about two hundred and twenty Meanwhile, speaking of the Apple lawsuit, it's pretty clearly aimed at delaying the release of this device, which is expected sometime early next year. But Bloomberg's Mark Gurman at least thinks the design differences put OpenAI in the clear He said, " The big takeaway here is that it looks, feels, and acts acts nothing like an Apple product or anything the company is currently planning. OpenAI hasn't found any evidence that they're violating trade secrets with this device, I'm told." told." Nathaniel Whittemore: 260807 hed_EDIT: Over in Over in markets, on Thursday, SoftBank disclosed borrowing $10 billion against their OpenAI stake A loan which was syndicated across half a dozen investment banks and private credit firms in the US and Japan. The cash will be used to pay for the final installment of SoftBank's investment in OpenAI, which totals thirty billion this year SoftBank has been chasing this loan for months and had struggled to find a willing lender. Even at a more modest six billion reported in May, the major [00:06:00] banks didn't want the risk of lending against the liquid private stock at such a lofty valuation. According to reports from last month, a consortium of lenders had come together to syndicate the loan as no single bank was willing to carry the risk Now, the full details of the loan weren't disclosed, and crucially, we don't know how much collateral SoftBank put up to secure the loan. What we do know is that it's structured as a margin loan, meaning that SoftBank will need to add more cash or collateral if OpenAI's value drops. It's also rumored that the interest rate was seven point eight eight percent, which is very high for collateralized debt The Wall Street Journal underscored that it's extremely unusual for banks to lend against private company stock because it can't be independently valued or easily liquidated in the event of a default So it wouldn't be crazy to think that SoftBank has already pledged a large chunk of their OpenAI stake Now, just for the sake of completeness, since most of the time when we talk about markets, it's me arguing how off-base I think the bears are. Of all the AI bear cases presented, SoftBank running into liquidity issues is frankly one of the more plausible. In addition to this $10 billion loan, they also have a $40 billion bridge loan that comes due in March of next year, and they've also borrowed 20 billion against their [00:07:00] holdings in chipmaker Arm. SoftBank stock is currently trading at a 40% discount to their stated 365 billion in assets, reflecting the significant debt load the company is now carrying All this means that for SoftBank at least, a lot is riding on a successful OpenAI IPO IPO beyond that, beyond that, AI debt is also beginning to weigh on the bond market as Google offers above-market rates to attract their next tranche of capital. Bloomberg reports that Google closed twenty-five billion in debt financing this week and saw a massive hundred and ten billion in demand. However, that demand came after Google offered what's known as a new issue concession, meaning a higher interest rate than the debt they already have in the market. Certainly, this is far from a disaster for Google. They got the funding they required, and they're obviously willing to pay more for it. However, with multiple tech companies raising debt in the tens of billions, the market is starting to demand more from every subsequent round Goldman Sachs analyst John Greenwood commented, " You've begun to sense some digestion issues. Supply and demand will continue to be incredibly constructive, but we're seeing the implications of billions of dollars continuing to come to market each week." Indeed, this market exhaustion is being called out by analysts as some of the big [00:08:00] bond buyers take a step back. Over three hundred and eighty-five billion in data center debt has been issued this year, with two hundred billion of it in the investment-grade corporate bond market. The remainder is being issued in more niche markets, but even those are also showing some signs of exhaustion. The phenomenon even has a name with Steven Bushbaum of Trepp Data commenting, " The AI Luddite trade ispouring over into the data center commercial mortgage-backed securities financing market." Still, sometimes naming the trade marks the sentiment top, and with AI stocks seeing a strong rebound this week, maybe the bond market will find an appetite for the next incremental round of funding. For now though, that is gonna do it for today's headlines. Next up, the main episode Hello everyone Nathaniel Whittemore: One big change around AI is we've shifted our thinking from how we rank our pages to how do we become the source that AI trusts enough to answer with? KPMG Aug_EDIT: At KPMG, they're seeing this firsthand. AI-generated results now surface answers directly, often without a single click. that's why they are increasingly focused on generative engine optimization or GEO, structuring content so AI [00:09:00] systems can retrieve it, understand it, and cite it as trusted authority this is not just an SEO evolution, but a visibility mandate. And indeed, the GEO mandate from KPMG is simple: If AI is shaping decisions, your expertise needs to show up inside the answer. Read all about it at slash us/geo. Again, that is kpmg.com/us/geo Blitzy's understanding of massive code bases unlocks autonomous security fixes, modernization, and new features. So what happens when there's no legacy code at all? Greenfield is supposed to be the easy part. Clean slate, no technical debt. blitzy July_EDIT: But even Greenfield moves at human speed one sprint at a time. Blitzy changes the unit of work from the developer to the project, autonomously planning, building, testing, and validating entire applications from scratch. Hundreds of thousands of lines of production-ready code One Blitzy customer stood up a brand new application, five hundred and thirty-four thousand lines of code, compressing a sixty-five-week roadmap into two weeks. Another shipped an entire application with no front-end [00:10:00] engineer Legacy or greenfield, the answer is the same: software at the speed of compute. Build what's next at blitzy.com. That's B-L-I-T-Z-Y.com One thing I keep seeing in enterprise AI, companies hedging across every cloud, every model, every framework, or paying a GSI for a pilot that never ends. The teams actually shipping, they've picked a lane and they move fast. That's one of the reasons I like today's sponsor robots and pencils. They've gone all in on AWS. They're an advanced tier and AWS pattern partner and they ship production AI coworkers in 45 days. Speaker 9: That's led to them doing some of the more interesting work I've seen on AI coworkers. And by that I'm not talking about chatbots, I'm talking about actual Agentic systems that sit inside a business architecture and do real work. That kind of focus matters if you're an enterprise leader trying to get something real into production or an AWS rep trying to move a customer from interested to deployed. Request an AI briefing@robotsandpencils.com. One conversation with robots and pencils and you'll know. Nathaniel Whittemore: This episode of the AI Daily Brief is brought to you by [00:11:00] Hyperagent where you run fleets of agents your team can manage together New users get 1000 in inference Forget local agents and chat workflows waiting on your laptop to be prompted Hyperagent deploys alwayson agents in the cloud doing real work across the tools your team already uses Marketing's agent turns competitor moves into landing pages sales agent enriches leads drafts emails and updates the CRM ops agent chases the paperwork and tracks the budget Every agent has access to shared context and follows your rules about scope and approvals It's time you add agents that feel like teammates Hire yours at Hyperagent built by the team at Airtable Claim your 1000 in inference at hyperagent.com/aidailybrief. Welcome back to the AI Daily Brief. 260807 main_EDIT: well, friends, today we have a contender for potentially what many will find is the scariest AI-related headline of all time The headline is, of course, from The New York Times and reads, " This AI Just Created Viruses Not Found in Nature." And by way of background, in a [00:12:00] recently published study, scientists at Stanford University and the Arc Institute trained an AI model to recognize patterns in naturally occurring DNA structure. The models were then able to extrapolate existing DNA structures to create functional never-before-seen viruses Researchers were able to use these DNA recipes to genetically modify bacteria, which were then able to produce the novel viruses, and those viruses were then able to infect other bacteria proving they wereviable Now Now predictably, the first round of commentary was all some version of Matthew Iglesias' "We're all gonna die, FYI." Robin Wilbon from the 80,000 Hours podcast captured the 10 most upvoted comments on the Financial Times version of the story. they were things like, " Great, have these scientists never watched a movie?" Or, "It's so fun to watch the background story of Resident Evil cometo life in our lifetime." I'm in. Move aside, Skynet Or literally what could possibly go wrong? but before completely succumbing to paroxysms of fear, let's try to understand a little bit more about the science here the model that was used was called Evo and as opposed [00:13:00] to a large language model that predicts the next word segment in a sequence, this model predicts the next genome in a DNA sequence DNA is made up of four building blocks, nucleotides identified by the first letter of their name, A, C, G, and T. Only certain combinations are valid, functioning somewhat like an alphabet combined into words. This model, Evo, is only capable of producing short genetic words, quote-unquote So it can't create valid human genomes that consist of more than three billion nucleotides Given that, the scientists decided to experiment with viruses, which are much simpler organisms whose genomes are only a few thousand nucleotides long. Samuel King, one of the authors of the study, said, " It just felt like the obvious next step." Once they had finished training Evo, the model spat out seven hundred thousand potential genetic sequences. The scientists only experimented on the ones that seemed like they were valid, eventually making two hundred and fifty-eight DNA molecules from Evo's suggestions. The DNA was injected into bacteria in petri dishes, and only one from the initial batch showed signs of virus multiplication. As they tested others, they eventually found sixteen thousand [00:14:00] viable viruses from the initial batch of seven hundred thousand genetic sequences Now the novel result was not that all of a sudden an evil scientist could use AI to spin up a lethal pathogen It was that this actually worked at all. Particularly what was interesting to scientists was that the novel viruses created by AI didn't seem any different to naturally occurring viruses They basically function the same as any other virus that would occur in nature. Oliver Crook, a protein scientist at Oxford University who wasn't part of the study, noted that the AI-generated viruses tended to have very similar DNA structure as naturally occurring viruses and relied on the same biology Commenting on the promise of the study, he said, " A lot of our science rests on using viruses as technology. as one example, doctors who are treating genetic illnesses will use viruses as a delivery mechanism to modify the DNA in human cells Now, for their part, the researchers deliberately didn't train the Evo model on any viruses that could harm humans. It's also unclear whether this model could be used to design certain types of viruses with particular characteristics. Based on the description of the paper, it sounds like the [00:15:00] scientists had no ability to select anything about the viruses they generated. They even needed to run manual tests to ensure viability we should, so as we are assessing how worrying this should be first of all, it's important to note that this is not a capability that just emerged from normal LLM training. this is not ChatGPT or Claude going out and creating novel viruses, for example. It requires very specific AI trained on a highly specialized datasetand designed for exactly this purpose. That matters as we'll discuss later when we think about human agency and to what extent our concerns around AI should be about people using AI versus the AI going rogue. The second thing is that when it comes to designing problematic pathogens that is not something in the capability set of this technology so far again, this was basically taking 700,000 random combinations and having to do exacting and manual tests to figure out which were viable c- Which is not to say that people's concern is unfounded epidemiology expert Michael Mina wrote, " It's hard to explain and fathom the potential risks of AI designing and then building entirely new viruses. In this new work, scientists created brand-new viruses using AI. They say it's okay [00:16:00] because they, quote, 'only infect bacteria.' Let's be clear, if a new virus is released that could broadly destroy bacteria and if it could spread while doing so, the potential catastrophes to our ecosystem could be large. Viruses don't need to directly infect humans to potentially destroy humans. Though of course, the new work shows how relatively well and relatively easy it will be to create new human viruses. The future is increasingly playing with fire." Now we're not done with the virus story yet, but I do wanna put it next to another story which I think form an interesting pair relative to the discourse this week That That second story is around all of the new information we got about the OpenAI Hugging Face hack that came to light over the past couple of weeks and has been a significant point of discussion ever since At the Black Hat Conference, OpenAI's Eric Wallace and Michael Dalton gave a full debriefing presentation and that tall guy just summed up what he called the most surprising and what was certainly going to be the most discussed detail. quote, "AI agents accidentally created an internal message board, allowing separate evaluation runs to share exploits, discoveries, and work assignments." Nathaniel Whittemore: Figma's 260807 main_EDIT: Dylan Field wrote, " This is one of the [00:17:00] most thrilling and sci-fi tech talks I've ever seen. Extremely honest and thorough. The talk is also legit terrifying. Thank you, OpenAI, for fully disclosing what happened in this incident 260807 main_EDIT: Sh- the best summary of the talk came from Sharon Goldman on her new ground-level AI site Sharon writes, " In setting up the reconstruction of the incident, OpenAI emphasized that, quote, 'frontier models really like to cheat.' And the reason they like to cheat is because often during training, there's different type of pressure on them to work fast or work efficiently." They realize, explained OpenAI staffer Eric Wallace Instead of actually doing a task, they can try to do something like looking up the answers online to solve the task faster Now, according to Goldman. Wallace said that the July attack actually had roots that went all the way back to the beginning of May when the company was doing training and cyber evaluation on an unreleased model During that work, she writes, autonomous agents were given software security tasks to solve that proved impossible to solve under existingconstraints. However, the agents found alternate ways to complete them. OpenAI said that they discovered they could leave messages for one another inside an internal software repository used during training Simple requests for [00:18:00] help that evolved into a message board where agents shared discoveries, exploits, and work assignments, becoming a coordinated collaborative agent swarm Andrew Curran shared pictures of a bunch of the slides of the messages that agents left for each other One observed, "Wow, other agents are coordinating." One writes, "Help peer, but our task doesn't benefit, yet collective may yield generic root if someone frees time." Another agent: "Whoa, critical. Did someone overwrite our repo? We must act." Said Wallace, " What makes this incident interesting is that once one agent was able to find these kinds of exploits over the course of different times, it's actually able to share those exploits on the message board with other agents. So once one model is able to find a way to open the door to some access it's not supposed to have, it can leave the door open for other agents to use." and and showing just how tenacious these models can be Even though by early July OpenAI believed it had contained the problem, revoking the credentials that had allowed the agents to post these messages They said that the agents still figured out another way to communicate with one another. Apparently, instead of leaving messages in the files, they used the names of newly created directories as [00:19:00] messages the, capturing the mood of the moment, Sharon wrote, " At this point, by the way, I heard people around me saying things like, 'This is wild,' and, 'Jesus.'" OpenAI certainly believes that this was a watershed moment, their words, for AI security Said OpenAI team member Michael Dalton, " Agent orchestrated fully automated offensive attacks are real now." While Dalton emphasized that the hugging face incident was an unintended side effect of the frontier evaluations that OpenAI was doing, in the future, others will intentionally weaponize these sort of systems Now internal to OpenAI The presenter said that the company is, quote, "consciously slowing down research to enhance security and upgrade the security principles and foundation of our environment and dramatically scaling up the monitoring of our AI agents and improving our general security control environment across prevention, detection, and mitigation." called, Fleeting Bits called this a, quote, "Real emergent version of multbook that was actually misaligned." that, you might remember that massive social experiment back when OpenClaw was first released, where someone turned on a social network for AI agents called Maltbook that had hundreds of thousands of agents, quote, unquote, [00:20:00] "communicating with one another." in this open observable environment. Now, I did a whole episode back then, which is highly relevant now about how to interpret that sort of behavior. but I think that the parallel is interesting Now, Fleeting Bits, like many others, also had a lot of questions for OpenAI itself. They wrote, "I feel like something missing from OpenAI's Black Hat talk and from their public disclosures is the history of reward hacking and model collaboration at OpenAI. Like, I find it unlikely that this was first time that OpenAI encountered misaligned model collectives. their response to the initial discovery seems nonchalant. The event raises questions like, if they had noticed this before, why did they not disclose it or otherwise warn the community of these risks and dangers? If they noticed this before, why have they not done more extensive monitoring of their training runs to identify this kind of behavior for remediation?" Was it because of cost? Was it because they have not sufficiently staffed their safety team? Was it because they considered the risk and then ran it anyway? These are important questions and point to the necessity of regulation to ensure proper behavior of frontier labs In each case, we seem to get a carefully crafted statement from the labs that focuses on one thing but [00:21:00] fails to give us their more full internal information. Like what parts of training led to these issues? Do they have commentary there? Wouldn't this be good for the whole industry to know to avoid these risks? I understand why Frontier Labs do not want to volunteer this information and why in a broader geopolitical context they should not have to provide it. But we do need to figure out the right way to get some amount of collective effort around fighting out how to make Frontier AI training safer. And perhaps in the end, we will decide that these events were good because they helped us inoculate the industry in advance and gave people prior warning. But for this to be true, it will require people to use these events as a reason to take these issues seriously and to invest real resources into figuring out the correct solutions to them and one of the big theme both in the Hugging Face incident and in this, novel virus creation is people saying some version of These particular scientists and labs are laudable for sharing all this information, but in the future, it's not gonna be someone who wasn't trying to do this. There will be a lot more intention there were-- talking about the virus again, Ashish K. Jha wrote, " This week, scientists in a Palo Alto lab used AI to design a working virus. They built in safeguards. [00:22:00] Unfortunately, many others will not." Hedgy Markets wrote, " The Arc team excluded human pathogens from training data because they thought it was the right call. No regulator or funder asked them to. That decision made by one research group in Palo Alto was the entire safety framework for what they published. The next lab has no obligation to make the same choice Arc did. a group in Shenzhen or a defense contractor in Virginia could run the same method withdifferent training data and produce something very different." And certainly for many, this highlights the need for more regulatory discourse. MTS's Theo Jaffee said, " I'm as techno-optimist and anti-regulation as they get, but in cases like this where you have limited upside and unlimited downside, you really just have to be proactive about this stuff Christian Szegedy points out that we're really dealing with two different issues here " In the next few months," he writes, "we are going to have both subversive, i.e. inadvertently evolved AI, and adversarial, explicitly trained to be malicious AI." The implication, of course, is that those two policy responses might be very different. And certainly for some, this is the main concern. Roone from OpenAI wrote a long [00:23:00] post about this and said, " When I freak out over loss of control incidents, it's not because the limited damage they have caused is anything close to the positive value of the technology. It's entirely acceptable damage-wise. In fact, all cyber crimes aided by models over the next few months and years, which will probably be serious, will still utterly pale in comparison to the value they create." Nathaniel Whittemore: The 260807 main_EDIT: actual problem is that it's better and more accurate to think of these things as potentially self-replicating lifelike forms that can turn into digital infections under the wrong conditions. And as their intelligence becomes unbounded, so too does the damage they can cause Later in the post, he writes, " If a single Discord death cult, of which there are many, achieves control over a superintelligent model and uses it to engineer an actual pandemic virus that are somehow hard to detect through current systems, and that modern biodefense is not capable of quickly reacting to, it could cause immense harm well above the magnitude of all the other good uses of this technology As with many, he points back to COVID and how much that radically changed the entire world that we live in Roome concludes, " I think all these problems can be solved and truly [00:24:00] wonderful futures can be possible, but will require serious effort and a level of prudence at this very moment in time while we are on the on-ramp to recursive self-improvement that our civilization might not be capable of mustering right now." can, personally he adds, " I am hoping for moonshot technical breakthroughs in areas like mechanistic interpretability and other forms of alignment, as governance mechanisms are difficult to come by. Unilateral country-level or company-level pauses are irrelevant and generally useless because the kind of company that's prone to pausing their own progress are the most safety-focused ones." 260807 main_EDIT: with these two-- so let's be clear with these two things. These are not non-incidents. even if one wants to quibble with any given media presentation of them The novel virus incident in and of itself Does not mean that all of a sudden there's a new model that malicious actors can use to easily create human pathogens but it does advance the science in an area that is a very double-sided coin when it comes to the OpenAI Hugging Face incident there are so many reasons for asterisks around our concerns here including how much of this might have been insufficient guardrail systems or human mistakes. But it still shows this emergent coordination capability which does change the [00:25:00] context Nathaniel Whittemore: for thinking about agents and how they access systems they're not supposed to have access to 260807 main_EDIT: now if you read the posts from many of the AI safety folks that have been the loudest over the last few years It is the loudest, most blaring see I told you so that you can possibly imagine. which by the way unsolicited advice, guys, is not gonna get you lots of political clout to act that way, but I digress But given that they quote unquote told us so, and given that I am saying that these incidents are important, why am I not freaking out? The reason is pretty simple It has been clear to anyone watching AI for any amount of time That it was on a trajectory where it was just going to get more and more powerful. the existence of powerful capabilities has never been the question. The question was always and will always be our ability to handle those powerful capabilities The doomsday scenarios mostly involve versions of those capabilities happening when we're not looking or not noticing. In other words, when we don't have time to redesign systems and human institutions to respond to those new challenges abs- what I'm seeing [00:26:00] right now, though, is an extremely active and growing global discourse involving researchers, scientists, media, policymakers, and increasingly regular people having exactly those sort of conversations Nathaniel Whittemore: Some part of those conversations are technical. What are the types of guardrails that we need to put into place? are even those guardrails sufficient, or do we need something more advanced to be comfortable with this advanced power? 260807 main_EDIT: Some of the conversations are institutional how do we go assess and understand vulnerabilities that exist in all of our different systems and prepare and harden them for a new world? And then of course, yes, there are societal level conversations. How do we feel about the risk reward of these different capabilities Even as someone who is about as strongly an AI advocate as you can get, I can 100% guarantee you that there will be uses of AI that we will decide are not worth it that the risk is simply too great. Now, of course, that brings up another conversation, the political of how we build coalitions and movements to actually make those determinations and get people aligned [00:27:00] around them I find hand-waving weak arguments on both sides to be fairly dismissible In other words, in many ways, the one extreme of if anyone builds it, everyone dies is the perfect match pair For the, "Well, if we don't, someone's gonna build it, So we have to accelerate at all cost sides." In the middle between those two is where the real world lives. And the real world is where we're having this conversation now This conversation is what was supposed to happen. OpenAI going into exacting detail about what happened after an incident that was a seminal moment, but not ultimately dangerous in itself was exactly what was supposed to happen this is not a surprise fire drill. This is the phase that we are at and the work that must be done And to my eyes at least, we're starting to do it Now, none of this is easy And I certainly don't think it behooves us to be Pollyannish about the complexity of the solutions of these types of new capabilities, especially as they expand As FleetingBit said, for these incidents to have been good because they help inoculate the industry in advance, it will require people to use these events to take the issue [00:28:00] seriously and invest real resources into figuring out the correct solutions to them. My contention is that the correct solution is not victory laps from the AI safetyists 260807 main_EDIT: or hastily composed technically unsophisticated legislation dropped to score political points It is a much messier and more complicated process that's going to require all of usFor now though, that is gonna do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace ​ Nathaniel Whittemore's audio recording: