EMERGENCY EPISODE: Manhattan residents need to VOTE Alex Bores for Congress next week (Jun 23)!

EMERGENCY EPISODE: Manhattan residents need to VOTE Alex Bores for Congress next week (Jun 23)!

Alex Bores is a generationally strong AI regulation candidate who needs our help to win the NY-12 congressional election next week. Bentham's Bulldog, a.k.a. Matthew Adelstein, joins me to help spread the message. PLEASE DO THIS: * Vote. If you live in Midtown Manhattan, click here [https://www.house.gov/representatives/find-your-representative] to check whether you’re in NY-12. If so, vote for Alex Bores on Tuesday, June 23rd. * Spread the word. Tell anyone you know in the district to vote for Bores — the race against Micah Lasher is neck and neck. * Donate now. If you can’t vote in NY-12, donate to the campaign [https://www.alexbores.nyc/] ASAP — donations lose value after ~June 18th because ads must be booked ~5 days in advance. Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

This Harvard Professor Says AI Alignment Will BACKFIRE - Dr. Stephen Casper

Stephen Casper, PhD, is an incoming tenure-track professor of public policy at the Harvard Kennedy School. Prof. Casper is relatively unworried about the threat of human extinction from rogue superintelligence, but he is concerned about a handful of companies wielding extremely powerful AI and enfeebling the public. That’s why he’s an AI governance hawk who advocates for taxes and regulation to keep frontier labs in check. We agree that slowing down AI development would make the world safer. But you know his position is unique when he says he’d prefer to have less research on AI alignment! Timestamps 00:00:00 — Cold Open 00:00:50 — Introducing Stephen “Cas” Casper 00:04:21 — Cas’s MIT Research: Model Tampering Attacks 00:07:14 — What’s Your P(Doom)?™ 00:08:34 — Why a 5–10% P(Doom) Feels Patronizing 00:13:07 — Crux: “The Intelligence Ceiling Might Not Be That High” 00:17:18 — Power Structures vs. the Shortcut-Finding AI 00:26:03 — Thought Experiment: A Data Center From the Year 2126 00:28:25 — Is a Data Center as Dangerous as a Nuke? 00:33:41 — Cas’s Mainline Scenario: Slow-Burn Gradual Disempowerment 00:40:49 — A Multipolar Powder Keg: 10 Companies, Thousands of Open Models 00:43:48 — Boarding the Doom Train at a Later Station 00:45:49 — Why Cas Is an “Anti-Timelines” Person 00:52:26 — Sycophancy, MechaHitler & Nudification: Foreseeable Failures 00:57:18 — Crux: Does Humanity Get Retries? 01:02:39 — The Scariest Superintelligent Optimizers Are Companies 01:06:08 — Lessons From the Deepfake Ecosystem: DALL-E 2 vs. Stable Diffusion 01:12:42 — The Case for a Pause Treaty 01:16:35 — Whac-A-Mole Forever: What Winning Looks Like 01:20:51 — Enshittification and the 84% Who Don’t Use AI 01:30:30 — Alignment Research as Safety Washing 01:34:25 — The Jevons Paradox of AI Safety 01:36:48 — Cas Would Press the Button to Halt Superalignment 01:39:21 — Wrap-Up: The Unsexy Path to Lowering P(Doom) Links Cas’s links * Cas’s website [https://stephencasper.com/] * Cas on X [https://x.com/StephenLCasper] * Cas’s Google Scholar [https://scholar.google.com/citations?user=zaF8UJcAAAAJ] * Cas’s MATS stream [https://www.matsprogram.org/stream/casper] Things referenced * Superintelligence by Nick Bostrom [https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies] * Gradual Disempowerment (Duvenaud et al.) [https://gradual-disempowerment.ai/] * If Anyone Builds It, Everyone Dies [https://ifanyonebuildsit.com/] * AI 2027 [https://ai-2027.com/] * UK AI Security Institute [https://www.aisi.gov.uk/] * Center for Human-Compatible AI (Berkeley) [https://humancompatible.ai/] * Internet Watch Foundation [https://www.iwf.org.uk/] Doom Debates episodes mentioned * Mike Israetel Returns — AI’s Gonna Kill Everyone vs. AI Will Make Everything Awesome [https://www.youtube.com/watch?v=WEMqmG1T00I] * Top AI Professor Has 85% P(Doom) — David Duvenaud [https://www.youtube.com/watch?v=mb9w7lFIHRM] * He Leads a Top AI Research Program, But He’d Hit the PAUSE Button — Kevin Zhu [https://www.youtube.com/watch?v=_UiHwZZ-P34] * Alignment is EASY and Roko’s Basilisk is GOOD?! — Roko Mijic [https://www.youtube.com/watch?v=AY4jD26RntE] * Andrew Critch vs. Liron Shapira: Will AI Extinction Be Fast Or Slow? [https://www.youtube.com/watch?v=opIvVzJF8t0] * AI Could Give Humans MORE Control — Ozzie Gooen [https://www.youtube.com/watch?v=6re47zw_6g0] * Steven Byrnes Part 1 [https://www.youtube.com/watch?v=_ZRUq3VEAc0] Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

He Leads a Top AI Research Program, But He’d Hit the PAUSE Button Today! Kevin Zhu, Algoverse Founder

Kevin Zhu walked away from a lucrative quant career to build Algoverse, one of the most productive mentorship programs for aspiring AI researchers. Can we agree on the best research path to a safe future? Let’s take a ride on the Doom Train! 00:00:00 — Cold Open 00:00:58 — Introducing Kevin Zhu 00:02:10 — From Citadel Quant to AI Researcher 00:09:14 — The Story of Founding AlgoVerse 00:12:53 — Discovering AI Safety: LessWrong & ARENA 00:17:22 — Emergent Misalignment Research 00:22:37 — Yudkowsky, MIRI & “Intellidynamics” 00:26:50 — What’s Your P(Doom)?™ 00:29:37 — Kevin’s Timeline to AGI + AI 2027 00:30:42 — Would You Slow Down AI? 00:37:44 — Coming Out of the P(Doom) Closet 00:45:49 — Should We Shame AI Company Workers? 00:52:27 — OpenAI’s Superalignment Team Collapse 00:55:01 — Riding the Doom Train™ 00:55:46 — First Stop: Instrumental Convergence 01:04:58 — Does Kevin Agree with the Orthogonality Thesis? 01:07:08 — “It’s Just Math.” Just Unplug It. 01:08:28 — “We Have a Safe Development Process” 01:11:49 — Group Dynamics & Laws Will Save Us 01:13:44 — Superintelligence Will Spare Us 01:14:25 — Is P(Doom) Just Bad Epistemology? 01:15:53 — China Will Race No Matter What 01:17:45 — Maybe Human Extinction Is Good? 01:20:36 — Wrap-Up Links Algoverse AI Research — https://algoverseairesearch.org/ [https://algoverseairesearch.org/] Kevin Zhu on Instagram — https://www.instagram.com/kevinzhu.ai/reels/ [https://www.instagram.com/kevinzhu.ai/reels/] Emergent Misalignment (Betley, Owain Evans et al.) — https://arxiv.org/abs/2502.17424 [https://arxiv.org/abs/2502.17424] Emergent Misalignment via In-Context Learning (Algoverse follow-up, ACL 2025) — https://arxiv.org/abs/2510.11288 [https://arxiv.org/abs/2510.11288] Agentic Misalignment (the blackmail result) — https://www.anthropic.com/research/agentic-misalignment [https://www.anthropic.com/research/agentic-misalignment] LessWrong — https://www.lesswrong.com/ [https://www.lesswrong.com/] PauseAI (global) — https://pauseai.info/ [https://pauseai.info/] OpenAI Superalignment — disbanded (CNBC, May 2024) — https://www.cnbc.com/2024/05/17/openai-superalignment-sutskever-leike.html [https://www.cnbc.com/2024/05/17/openai-superalignment-sutskever-leike.html] Evan Hubinger — “Why I’m joining Anthropic” — https://www.alignmentforum.org/posts/7jn5aDadcMH6sFeJe/why-i-m-joining-anthropic [https://www.alignmentforum.org/posts/7jn5aDadcMH6sFeJe/why-i-m-joining-anthropic] AI 2027 — https://ai-2027.com/ [https://ai-2027.com/] Bentham’s Bulldog / Matthew Adelstein on Doom Debates — https://doomdebates.com/p/benthams-bulldog-ai-doom-debate [https://doomdebates.com/p/benthams-bulldog-ai-doom-debate] Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

Top Mathematicians Face Irrelevance, a 7-Year-Old's P(Doom) + The “Off Switch" Debate — Livestream May 29

Liron explains the 80-year-old Erdős conjecture that a GPT model breakthrough, Scott Aaronson ponders "the last days of human relevance”, multiple live callers debate me, plus my 7-year-old son drops in to share his P(Doom)! Timestamps 00:00:00 — Cold Open 00:00:42 — First Donation & “Why Is a Duck?” 00:02:54 — Can AI Draw “Colorless Green Ideas Sleep Furiously”? 00:08:26 — Doom Debates Sponsors Less Online — Be Our Intern 00:10:43 — Who We Still Need to Get on the Show 00:26:44 — Claude Opus 4.8 & the Rising Waterline of Intelligence 00:28:48 — The 80-Year-Old Geometry Conjecture a GPT Model Cracked 00:44:50 — My 7-Year-Old Ezra Joins: ChatGPT, Minecraft & His P(Doom) 00:58:46 — How the AI Actually Beat Erdős’s Grid 01:03:50 — Scott Aaronson: “The Last Days of Human Relevance” 01:07:53 — Why the Foom Is Taking Years, Not Hours 01:10:47 — Is Ori Just a Yes Man? 01:12:41 — Does P = NP + AI? 01:23:43 — METR’s Beth Barnes: “We Are Not On Top Of It” 01:28:31 — Liron’s Vibe Coding Confession 01:31:21 — Alverin Joins: Can We Wait, Then Hit the Off Switch? 01:48:01 — AI in a Box & the Super-Persuasion Threshold 02:06:24 — Brian Joins: The Ways It Could Go Right 02:11:30 — Jack Joins: How Fast Will the Foom Be? 02:18:48 — 80,000 Hours, Inventing Erdős Problems & Holly Elmore’s Warning 02:29:42 — Wrap-Up Links LessOnline 2026 — June 5–7, Berkeley — https://less.online/ [https://less.online/] Manifest 2026 — June 12–14 — https://manifest.is/ [https://manifest.is/] Shtetl-Optimized — “Dispatches from the possibly last days of human relevance” — https://scottaaronson.blog/?p=9782 [https://scottaaronson.blog/?p=9782] Global Call for AI Red Lines (Bengio, Hinton, Harari) — red-lines.ai [https://red-lines.ai/] Order the 80,000 Hours book — Benjamin Todd (Penguin) — https://80000hours.org/book/ [https://80000hours.org/book/] Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]

Anthropic's New Hire Should WORRY You, AI is a Math Genius, Live Callers! — Livestream (May 22)

***ATTENTION: We’re looking for paid help at LessOnline from June 5-7. Apply now!*** OpenAI pushes the frontier of mathematics, Anthropic hires Andrej Karpathy to work on recursive self-improvement, live callers challenge the doom argument, and we've got a soundboard! Doom Debates Paid Gig in Berkeley! Doom Debates is proudly sponsoring The LessOnline [https://less.online/] conference this year and we got a merch table in the courtyard! We need 2 people to help represent the show — hand out T-shirts, talk to attendees, and spread the good word about Doom Debates. If you’re a regular viewer of the show, you probably know enough. Just be knowledgeable, friendly and helpful! * 💰We pay you $500 to help out during the entire Fri-Sun event * 🎟️ Plus you get FREE admission to the conference ($675 value) This is best suited for someone local to the SF Bay Area or who can arrange accommodations. If you’re interested, send an email to internship@doomdebates.com or DM Producer Ori on Discord [https://discord.gg/g2X8h5UCrU]. Tell us a little about yourself and your availability for all 3 days. Thanks! Links Less Online conference (June 5–7, Berkeley) — https://less.online [https://less.online] Doom Debates Discord — https://discord.gg/g2X8h5UCrU [https://discord.gg/g2X8h5UCrU] OpenAI — “An OpenAI model has disproved a central conjecture in discrete geometry” — https://openai.com/index/model-disproves-discrete-geometry-conjecture/ [https://openai.com/index/model-disproves-discrete-geometry-conjecture/] Timestamps 00:00:00 — Welcome + New Soundboard! 00:09:27 — Doom Debates Is Sponsoring Less Online 00:17:30 — Ben Goertzel Episode Recap 00:27:30 — Goertzel's Alignment Blind Spot 00:34:21 — OpenAI Disproves an 80-Year-Old Erdős Conjecture 00:41:00 — David Deutsch Says "Amazing" 00:50:31 — Eliezer Yudkowsky's Job Replacement Progression 00:54:07 — Anthropic's $44 Billion Run Rate 01:07:45 — Schadenfreude Week: Liron Loses $10K 01:13:04 — Teaching Ezra Prediction Markets 01:18:11 — Caller Williawa: Is the Goal Engine Argument Effective? 01:37:00 — Caller Will: Brain Rot and Idiocracy 01:55:04 — Anthropic Hires Karpathy for Recursive Self-Improvement 01:57:16 — Rob Wiblin Says Recursive Self-Improvement Should be "Illegal" 02:01:37 — Caller Henry: The Chicken Riddle 02:08:41 — David Sacks Kills the AI Executive Order 02:11:20 — The Philanthropy Tidal Wave 02:15:56 — Donate to Doom Debates Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates], or to really take things to the next level: Donate [https://doomdebates.com/donate] 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe [https://lironshapira.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4]