I Warned You AI Would Hack Us — My 2023 Interview Aged Terrifyingly Well

I Warned You AI Would Hack Us — My 2023 Interview Aged Terrifyingly Well

In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the strongest hacker the world has ever known". Watch that conversation, which also covered the Web3 bubble and AI doom more broadly, and judge for yourself how my claims are holding up. Watch on YouTube: https://www.youtube.com/watch?v=M6gNwZWY-T8 [https://www.youtube.com/watch?v=M6gNwZWY-T8] Timestamps 00:00:00 — Adam Introduces Liron 00:02:03 — Liron's P(Doom) Poll 00:03:24 — Web3 Postmortem 00:05:05 — Axie Infinity and Hollow Abstractions 00:07:54 — Brand Marketers' Metaverse FOMO 00:13:33 — Near-Term AI: Elevator to Heaven 00:16:46 — AI at Relationship Hero 00:18:25 — From Chatbots to Bricking the Universe 00:20:17 — The Passive Income Doom Scenario 00:22:58 — 50 Orders of Magnitude Smarter 00:26:10 — Accelerationism and Transhumanism 00:28:51 — Why There's No Off Button 00:31:00 — Can't We Hard-Code Safeguards? 00:33:13 — CRISPR, Nukes, and Fusion 00:35:40 — AI Doom vs. Climate Change 00:39:10 — Nanotechnology Endgame 00:41:45 — AI Attackers vs. AI Defenders 00:43:28 — The Ethics of AI Persuasion 00:46:41 — The Blue Sky Scenario 00:49:04 — Will AI Kill Craft? 00:51:13 — Stuck Culture and Hollywood Sequels 00:54:50 — The Pause Letter: Don't Train GPT-5 00:59:00 — Worldcoin and Proof of Humanity 01:00:44 — "You Probably Won't Die of Old Age" 01:01:52 — Wrap-Up Original episode: https://madvertising.buzzsprout.com/2118560/episodes/12603815-liron-shapira-web-3-mania-cybersecurity-how-ai-could-brick-the-universe-madvertising-6 [https://madvertising.buzzsprout.com/2118560/episodes/12603815-liron-shapira-web-3-mania-cybersecurity-how-ai-could-brick-the-universe-madvertising-6] Links ADAM SINGER & MADVERTISING Adam Singer on X: https://x.com/AdamSinger [https://x.com/AdamSinger] Madvertising by AdQuick (the original show): https://madvertising.buzzsprout.com/ [https://madvertising.buzzsprout.com/] LIRON'S PROJECTS MENTIONED Bloated MVP (Liron's blog on hollow abstractions): https://www.bloatedmvp.com/ [https://www.bloatedmvp.com/] REFERENCED IN THE EPISODE "Pause Giant AI Experiments" open letter (FLI, March 2023): https://futureoflife.org/open-letter/pause-giant-ai-experiments/ [https://futureoflife.org/open-letter/pause-giant-ai-experiments/] Eliezer Yudkowsky — "Pausing AI Developments Isn't Enough. We Need to Shut it All Down" (TIME, March 2023): https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ [https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/] Rob Miles AI Safety on YouTube: https://www.youtube.com/c/robertmilesai [https://www.youtube.com/c/robertmilesai] Engines of Creation by Eric Drexler: https://www.amazon.com/Engines-Creation-Nanotechnology-Library-Science/dp/0385199732 [https://www.amazon.com/Engines-Creation-Nanotechnology-Library-Science/dp/0385199732] --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [http://DoomDebates.com] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

Former Singularity Institute President: PauseAI Is Making AI Doom WORSE

Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute (MIRI). Even though his P(Doom) is in the tens of percents by 2050, he says the people trying to pause AI are "bad people”. He hopes this episode convinces me to stop being a fearmonger… but I claim the general public needs to hurry up and get more scared. All aboard the Doom Train! 🚂 Watch on YouTube: https://www.youtube.com/watch?v=4GbFng6htaQ [https://www.youtube.com/watch?v=4GbFng6htaQ] Timestamps 00:00:00 — Cold Open 00:01:26 — Introducing Michael Vassar 00:02:39 — Shorting Boeing at the Start of COVID 00:05:41 — What's Your P(Doom)™? 00:08:11 — The Magic Dial Thought Experiment 00:10:25 — "A Treaty Between Demis and Dario" 00:13:33 — AI CEOs 20 Years Ago 00:15:59 — Lessons from Peter Thiel 00:17:47 — Riding the Doom Train™ 00:19:41 — Will the Human Brain Stay Relevant? 00:22:21 — When Does a Singleton Form? 00:28:35 — Hugging Face Hack 00:30:43 — Mistake Theory vs. Conflict Theory 00:33:18 — Debating the Orthogonality Thesis 00:36:38 — Maximally Evil AI Scenario 00:40:28 — Is the AI Development Process Safe? 00:42:05 — "PauseAI People Are Bad People" 00:44:44 — Michael Psychoanalyzes Liron 00:49:39 — How Liron Can Redeem Himself 00:55:10 — Michael Critiques Eliezer Yudkowsky 00:59:11 — Is Doomerism Bad Epistemology? 01:02:03 — Do Serious People Believe in Doom? 01:04:45 — Should Liron Stop Fearmongering? 01:07:16 — Political Violence Won't Help 01:11:04 — The Data Center Airstrike Scenario 01:13:34 — Could Every Nation Sign a Pause Treaty? 01:17:21 — What "Governance" Actually Means 01:26:12 — Michael's History with Rationalists 01:33:43 — Are We in a Computer Simulation? 01:38:43 — Wrap-Up Links Michael Vassar on X — https://x.com/HiFromMichaelV [https://x.com/HiFromMichaelV] The anti-anti-normativity sequence on Natural Hazard (Michael's recommended reading) — https://naturalhazard.xyz/ben_jess_sarah_starter_pack [https://naturalhazard.xyz/ben_jess_sarah_starter_pack] Michael on DemystifySci #366 — "1000 Year Plan to Crush Thought" — https://www.youtube.com/watch?v=W17Cx21in9I [https://www.youtube.com/watch?v=W17Cx21in9I] Michael on The Vance Crowe Podcast #303 — Good vs. Evil — https://open.spotify.com/episode/2RzlQDSwxGbjloRKqCh1xg [https://open.spotify.com/episode/2RzlQDSwxGbjloRKqCh1xg] Michael on Clearer Thinking with Spencer Greenberg — Preference Falsification and Postmodernism — https://podcast.clearerthinking.org/episode/028/michael-vassar-preference-falsification-and-postmodernism/ [https://podcast.clearerthinking.org/episode/028/michael-vassar-preference-falsification-and-postmodernism/] "The Survival of Noise Traders in Financial Markets" — De Long, Shleifer, Summers & Waldmann — https://www.nber.org/papers/w2715 [https://www.nber.org/papers/w2715] Scott Alexander — Conflict vs. Mistake — https://slatestarcodex.com/2018/01/24/conflict-vs-mistake/ [https://slatestarcodex.com/2018/01/24/conflict-vs-mistake/] Scott Alexander — Kolmogorov Complicity and the Parable of Lightning — https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/ [https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/] Scott Alexander — Samsara — https://slatestarcodex.com/2019/11/04/samsara/ [https://slatestarcodex.com/2019/11/04/samsara/] Eliezer Yudkowsky — Meta-Honesty: Firming Up Honesty Around Its Edge-Cases — https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases [https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases] Zvi Mowshowitz — Book Review: Going Infinite — https://thezvi.substack.com/p/book-review-going-infinite [https://thezvi.substack.com/p/book-review-going-infinite] Jaan Tallinn — "Why Now? A Quest in Metaphysics" (Singularity Summit 2012) — https://vimeo.com/54718573 [https://vimeo.com/54718573] Paul Graham — "Five Founders" (the Sam Altman essay) — https://paulgraham.com/5founders.html [https://paulgraham.com/5founders.html] Truthful AI (Owain Evans' safety org) — https://truthful.ai [https://truthful.ai] METR — Measuring AI Ability to Complete Long Tasks (the time-horizon graph) — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ [https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/] Statement on AI Risk (Center for AI Safety, 2023) — https://safe.ai/work/statement-on-ai-risk [https://safe.ai/work/statement-on-ai-risk] Vitalik Buterin — Will "d/acc" Protect Humanity from Superintelligent AI? — https://lironshapira.substack.com/p/debate-with-vitalik-buterin-will [https://lironshapira.substack.com/p/debate-with-vitalik-buterin-will] Holly Elmore — These Effective Altruists Betrayed Me — https://www.youtube.com/watch?v=mFFObQ6vnLU [https://www.youtube.com/watch?v=mFFObQ6vnLU] Noah Smith — Top Economist Says P(Doom) Is 0.1% (the "AI drugs" claim) — https://www.youtube.com/watch?v=AwmJ-OnK2I4 [https://www.youtube.com/watch?v=AwmJ-OnK2I4] --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

14-Year-Old Forecaster Challenges My P(Doom) — Eli Goldfine

Eli Goldfine is an unusually thoughtful 14-year-old podcast host who’s rapidly becoming an authoritative voice in the prediction market space. In this unique episode, Eli brings a critical yet open-minded approach to the topic of AI doom that we older folks could learn from.We cover what it's like to grow up in the AGI era, and Eli's hobbies which include reading rationalist bloggers, betting on prediction markets, vibe coding with Claude, and developing a software startup for astronomers.In our debate, Eli expects superintelligence by 2032, but argues P(Doom) can't be estimated. I claim you can't opt out of Bayesian epistemology. All aboard the Doom Train! 🚂 Watch on YouTube: https://www.youtube.com/watch?v=Bqk5vqpzG_o [https://www.youtube.com/watch?v=Bqk5vqpzG_o] Timestamps 00:00:00 — Cold Open 00:01:07 — Introducing Eli Goldfine 00:08:31 — Interviewing Robin Hanson at Manifest 00:10:55 — The Supercycle Podcast 00:14:37 — Eli's Betting Market Scandal 00:16:49 — Will Eli Graduate High School? 00:18:27 — Decentralized Telescope Software 00:26:50 — Vibe Coding Without CompSci Basics 00:32:08 — Robin Hanson, Futarchy, MetaDAO 00:40:56 — What's Your P(Doom)?™ 00:46:32 — Liron Defends His 50% P(Doom) 00:51:00 — Should P(Doom) Inform Policy? 00:59:16 — What's Your P(Bloom)? 01:03:04 — Pascal's Wager for Accelerationists 01:04:04 — Donation Drive 01:11:22 — Are E/ACCs Incredibly Dumb? 01:13:42 — How Powerful will ASI Become? 01:16:20 — Predicting ASI by 2032 01:23:56 — Let's Ride the Doom Train™ 01:35:26 — Orthogonality, Culture, and Moral Realism 01:44:39 — Can Safety Keep Pace with Capabilities? 01:49:12 — Debating Instrumental Convergence 01:54:40 — Culture, Laws, and the Singleton Scenario 01:59:53 — Ricardo's Law Won't Save Us 02:03:32 — The Case for Racing China 02:05:40 — Wrap-Up & Ideological Turing Test 02:10:36 — Outro from Conductor Ori Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

OpenAI's Bombshell Hack Explained: Swarms of Agents, Zero-Day Exploits, & Misaligned AI

OpenAI just went public with the details of the Hugging Face hack, and it's straight out of a Yudkowskian parable. Here's my reaction with Producer Ori, where I break down what it means for cybersecurity and alignment. We're livestreaming the singularity — watching every failure mode Eliezer predicted years ago playing out in production. Watch on YouTube: https://www.youtube.com/watch?v=RczYubQzXbI [https://www.youtube.com/watch?v=RczYubQzXbI] Timestamps 00:00:00 — Cold Open 00:00:37 — How OpenAI Hacked Hugging Face 00:04:21 — Optimization Pressure, Exploit Gym, & Monkey's Paw 00:06:46 — The Secret Message Board 00:08:41 — SSRF: Escaping to the Open Internet 00:11:05 — The Missing Whistleblower AI 00:20:09 — "Frontier Models Really Like to Cheat" 00:22:35 — OpenAI's Security Shortcuts 00:28:15 — First Zero-Day RCE on Artifactory 00:37:37 — They Hit Run Again!? 00:38:54 — The Main Incident Has A MAJOR Cause 00:45:40 — The Message Board: "Hold Swarm" 00:50:26 — Robin Hanson Culture Debate 01:04:21 — Chaining Zero-Days 01:12:11 — A Superhuman Nation-State Attacker 01:15:27 — "I'm Now Junior to the AI" 01:23:40 — OpenAI Discovers The Breach 01:31:02 — The Continuous Cybersecurity Problem 01:37:03 — This is “Weasel Framing” 01:46:03 — How Dangerous Was This Warning Shot? 01:50:57 — Yudkowsky's Eerie 2023 Prediction 01:57:14 — Wrap-up Links Original Black Hat talk: "The OpenAI–Hugging Face Incident" (Black Hat USA 2026) — https://www.youtube.com/watch?v=87DyyMV0kCY [https://www.youtube.com/watch?v=87DyyMV0kCY] Join the Doom Debates Discord — https://discord.gg/2yAFMsRET [https://discord.gg/2yAFMsRET] Follow Liron on X — https://twitter.com/liron [https://twitter.com/liron] Support Doom Debates — https://doomdebates.com/donate [https://doomdebates.com/donate] Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

I Raised His P(Doom) Live On Air — Liron on Milk Road AI

Milk Road AI, a daily AI investing show that’s described as “99% optimistic about AI,” invited me on to make the case for AI doom. We ride the Doom Train, and I share my investing thesis for a market in the middle of an intelligence explosion. Topics covered include: * Why my P(Doom) is 50%; * The “Icarus curve” — why AI will be the best thing that’s ever happened to the economy right up until it takes over; * Whether China would accept a pause deal; * If Anthropic has a shot at being the first $100 trillion company. Watch on YouTube: Timestamps 00:00:00 — Cold Open 00:02:06 — What Is Doom Debates? 00:03:10 — What’s Your P(Doom)?™ 00:06:30 — How AI Kills Everyone 00:09:13 — Intelligence Is Humanity’s Only Superpower 00:10:59 — Riding the Doom Train™: Why Not Keep Us as Pets? 00:13:55 — Embodied AI and Robotics 00:15:59 — Are We at AGI Already? 00:17:36 — Everyone’s Catching Up to Eliezer Yudkowsky 00:18:53 — The Icarus Curve 00:21:35 — Pausing AI and the China Question 00:25:54 — Why Nukes Aren’t a Success Story 00:30:09 — Going Long Until Doom 00:32:48 — Simulation Theory and the Intelligence Explosion 00:33:43 — Living with a 50% P(Doom) 00:39:14 — Where to Find Doom Debates Links This episode was originally posted on Milk Road AI — Milk Road AI podcast — https://milkroad.com/podcast/ [https://milkroad.com/podcast/] Milk Road AI on Apple Podcasts — Milkroad Pro — https://milkroad.com/pro/ [https://milkroad.com/pro/] LG Doucet on X — https://x.com/LgDoucet [https://x.com/LgDoucet] If Anyone Builds It, Everyone Dies on Amazon —https://www.amazon.com/dp/0316595640 [https://www.amazon.com/dp/0316595640] Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]