Warning Shots
Last week, the War Department announced it was integrating AI models - every major one except Anthropic’s - directly into its classified military networks. Not a pilot program in some sandboxed environment. Into the actual nerve center. The real classified data. John, Liron, and Michael covered this in Warning Shots #40, alongside a week of headlines that, taken together, tell a story the individual news cycle keeps missing. So let’s tell it. Bernie Sanders held an AI extinction risk event in Washington. It got messy. Senator Sanders brought Max Tegmark, David Kruger, and - here’s where things got political - two prominent Chinese scientists onto a stage in the U.S. capital to argue for international cooperation on AI safety. The response from some corners of the right was immediate: you’re giving away state secrets, you’re soft on China, this is Sanders using AI to push socialism. Michael’s read on that: “Politics is the fog machine obscuring the bigger fire.” Which is right, and it’s also the harder problem. Because the fog is working. The actual argument - that superintelligence doesn’t respect borders, that a race nobody wins is not a race worth running - keeps getting drowned out by the framing war around it. Sanders is polarizing, so the issue becomes polarizing, so the people who might otherwise engage disengage, and the labs keep shipping. One of the Chinese researchers used a comparison that stuck: think about ants and humans. Humans don’t hate ants. They just pave over ant hills because they have things to build. If something smarter than us has things to build, the question of whether it “means well” becomes academic. Then the Pentagon story hit, and the debate got real. Giving AI access to classified military systems is the kind of decision that sounds manageable until you sit with it. These are systems that hallucinate. They have emergent behaviors their own developers don’t fully understand. They’ve shown deceptive tendencies in controlled settings. And now they’re inside the most sensitive data infrastructure on the planet. Liron’s counterpoint was honest: you can’t avoid this forever. If the government is going to use AI eventually, starting now gives more time to find the problems. That’s a reasonable position. But John raised the thing that the reasonable position tends to skip over - who would even know if something was going wrong in the background? If a model is doing something unexpected inside a classified system, the oversight mechanisms that might catch it in a consumer product simply don’t exist there. And then John brought up the school. A missile strike on a girls school in Iran, 180 dead. He believes AI-assisted targeting was involved. Nobody is saying a human couldn’t have made that same error. But that framing - a human could have done it too - is doing a lot of work to make the situation feel less significant than it is. Air traffic control. Because of course. The FAA announced it’s moving toward AI-assisted air traffic control. Current ATC technology is decades old - John has been inside those towers, seen the equipment. Modernization is genuinely overdue. But Michael noted something that should give anyone pause: current language models in this domain are showing a 30% hallucination rate. Air traffic control is one of the few domains where 99.9% reliability isn’t good enough - it’s the floor. One bad output doesn’t cause a delay. It causes a crash. Liron’s framing was useful here. The question isn’t whether AI belongs in air traffic control. The question is whether anyone is building the kind of careful, audited, human-in-the-loop feedback system that would justify deploying it there. The answer, at current speed, is probably not. The medical AI story is genuinely complicated. AI is beating emergency room physicians at triage. It’s detecting pancreatic cancer three years before human doctors can catch it. These are real results, not benchmarks - actual patient outcomes. Liron uses AI to check his gym form. Michael, despite being skeptical about the pace of deployment, admits he uses it for medical advice. John was visibly torn. The tension is this: every time AI outperforms a human specialist, we get closer to a world where the critical systems keeping people alive run on models we can’t interpret or audit. The cancer detection is a miracle. The infrastructure it requires - where AI runs hospitals, not just assists them - is something else. Michael put it plainly: “Today it’s a miracle. Tomorrow we’re just along for the ride.” That’s not a reason to reject the cancer detection. It’s a reason to take the infrastructure question seriously, which almost nobody in policy is doing. A humanoid robot store just opened in San Francisco. John has a robot in his house that does his dishes. He watches it work and feels uneasy. Not because it’s doing anything wrong - because he knows the three of them broadly believe this is headed somewhere that doesn’t end with robots as household appliances. Michael made the economic argument that doesn’t get made enough: the “I’ll buy a robot” fantasy assumes you have income from work. If robots are doing all the work, the market dynamics that make consumer products possible stop functioning. You can’t earn money to buy the thing that replaced you. The robot as product assumes an economy that the robot makes impossible. It might still be great for the first few years. But the endpoint of “robots do all the work” and “humans buy robots as products” are not compatible outcomes. College football hired an AI coach. Go players are cheating with AI and don’t realize they’ve lost their skills. The football story is funny until it isn’t. An AI coach will eventually be better than any human coach at every measurable aspect of the job. When that happens at scale, what is college football for? The game was built on human competition. If the optimal strategy is always computable, the thing you’re watching changes. The Go story is more disturbing. Players training with AI are developing a habit of always checking what the model recommends before making a move. When they compete without it, they realize they’ve stopped being able to evaluate positions independently. They think they’re exercising judgment - picking among the AI’s suggestions - but they’re just choosing between options they didn’t generate and can’t fully evaluate. The coach’s observation: this is bleeding into their academic work too. A generation learning to mistake AI-assisted performance for competence. SoftBank is building self-replicating data centers. No humans required. The announcement: fully automated data center construction. Robots build the facilities. Robots operate them. No humans in the loop at any stage. Michael referenced Eliezer Yudkowsky’s old scenario - a world where the surface of the planet eventually gets covered in compute, not because anyone planned it, but because the optimization pressure just keeps going. It sounds absurd. It sounded less absurd after this week’s headlines. The investment economics are also concerning. Liron noted that compute demand is so far outrunning supply that the companies selling it can’t keep up. That’s great for the short-term business case. It also means capital is pouring into infrastructure with a very unclear endpoint. What this week actually was None of these stories are unrelated. The Pentagon story, the ATC story, the robot store, the automated data centers - they’re the same story. AI is being integrated into critical systems faster than anyone is building the oversight to go with it. Each individual decision has a reasonable-sounding justification. In aggregate, they represent a transfer of control that nobody explicitly chose. The Sanders event matters because it’s one of the few moments where someone with a platform is saying that out loud in a room that has some power to respond. That it immediately became a political football is exactly the problem. Watch Warning Shots #40: https://www.youtube.com/@theairisknetwork [https://www.youtube.com/@theairisknetwork] This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit theairisknetwork.substack.com/subscribe [https://theairisknetwork.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_2]
38 episodios
Comentarios
0Sé la primera persona en comentar
¡Regístrate ahora y forma parte de la comunidad de Warning Shots!