Doom Debates

Bryan Cantrill, co-founder of Oxide Computer, says in his talk [https://www.youtube.com/watch?v=bQfJi7rjuEk] that engineering in the physical world is too complex for any AI to do it better than teams of human engineers. Success isn’t about intelligence; it’s about teamwork, character and resilience. I completely disagree. 00:00 Introduction 02:03 Bryan’s Take on AI Doom 05:55 The Concept of P(Doom) 08:36 Engineering Challenges and Human Intelligence 15:09 The Role of Regulation and Authoritarianism in AI Control 29:44 Engineering Complexity: A Case Study from Oxide Computer 40:06 The Value of Team Collaboration 46:13 Human Attributes in Engineering 49:33 AI's Potential in Engineering 58:23 Existential Risks and AI Predictions Bryan’s original talk: Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/watch?v=9CUFbqh16Fg [https://www.youtube.com/watch?v=9CUFbqh16Fg] PauseAI, the volunteer organization I’m part of: https://pauseai.info [https://pauseai.info] Join the PauseAI Discord — https://discord.gg/2XXWXvErfA [https://discord.gg/2XXWXvErfA] — and say hi to me in the #doom-debates-podcast channel! Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates] This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com [https://lironshapira.substack.com?utm_medium=podcast&utm_campaign=CTA_1]

Thanks for everyone who participated in the live Q&A on Friday! The topics covered include advice for computer science students, working in AI trustworthiness, what good AI regulation looks like, the implications of the $500B Stargate project, the public's gradual understanding of AI risks, the impact of minor AI disasters, and the philosophy of consciousness. 00:00 Advice for Comp Sci Students 01:14 The $500B Stargate Project 02:36 Eliezer's Recent Podcast 03:07 AI Safety and Public Policy 04:28 AI Disruption and Politics 05:12 DeepSeek and AI Advancements 06:54 Human vs. AI Intelligence 14:00 Consciousness and AI 24:34 Dark Forest Theory and AI 35:31 Investing in Yourself 42:42 Probability of Aliens Saving Us from AI 43:31 Brain-Computer Interfaces and AI Safety 46:19 Debating AI Safety and Human Intelligence 48:50 Nefarious AI Activities and Satellite Surveillance 49:31 Pliny the Prompter Jailbreaking AI 50:20 Can’t vs. Won’t Destroy the World 51:15 How to Make AI Risk Feel Present 54:27 Keeping Doom Arguments On Track 57:04 Game Theory and AI Development Race 01:01:26 Mental Model of Average Non-Doomer 01:04:58 Is Liron a Strict Bayesian and Utilitarian? 01:09:48 Can We Rename “Doom Debates” 01:12:34 The Role of AI Trustworthiness 01:16:48 Minor AI Disasters 01:18:07 Most Likely Reason Things Go Well 01:21:00 Final Thoughts Show Notes Previous post where people submitted questions: https://lironshapira.substack.com/p/ai-twitter-beefs-3-marc-andreessen [https://lironshapira.substack.com/p/ai-twitter-beefs-3-marc-andreessen] Watch the Lethal Intelligence Guide [https://www.youtube.com/watch?v=9CUFbqh16Fg], the ultimate introduction to AI x-risk! PauseAI, the volunteer organization I’m part of: https://pauseai.info [https://pauseai.info] Join the PauseAI Discord — https://discord.gg/2XXWXvErfA [https://discord.gg/2XXWXvErfA] — and say hi to me in the #doom-debates-podcast channel! Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates] This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com [https://lironshapira.substack.com?utm_medium=podcast&utm_campaign=CTA_1]

It’s time for AI Twitter Beefs #3: 00:00 Introduction 01:27 Marc Andreessen vs. Sam Altman 09:15 Mark Zuckerberg 35:40 Martin Casado 47:26 Gary Marcus vs. Miles Brundage Bet 58:39 Scott Alexander’s AI Art Turing Test 01:11:29 Roon 01:16:35 Stephen McAleer 01:22:25 Emmett Shear 01:37:20 OpenAI’s “Safety” 01:44:09 Naval Ravikant vs. Eliezer Yudkowsky 01:56:03 Comic Relief 01:58:53 Final Thoughts Show Notes Upcoming Live Q&A: https://lironshapira.substack.com/p/2500-subscribers-live-q-and-a-ask [https://lironshapira.substack.com/p/2500-subscribers-live-q-and-a-ask] “Make Your Beliefs Pay Rent In Anticipated Experiences” by Eliezer Yudkowsky on LessWrong: https://www.lesswrong.com/posts/a7n8GdKiAZRX86T5A/making-beliefs-pay-rent-in-anticipated-experiences [https://www.lesswrong.com/posts/a7n8GdKiAZRX86T5A/making-beliefs-pay-rent-in-anticipated-experiences] Scott Alexander’s AI Art Turing Test: https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing [https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing] Watch the Lethal Intelligence Guide [https://www.youtube.com/watch?v=9CUFbqh16Fg], the ultimate introduction to AI x-risk! PauseAI, the volunteer organization I’m part of: https://pauseai.info [https://pauseai.info] Join the PauseAI Discord — https://discord.gg/2XXWXvErfA [https://discord.gg/2XXWXvErfA] — and say hi to me in the #doom-debates-podcast channel! Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates] This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com [https://lironshapira.substack.com?utm_medium=podcast&utm_campaign=CTA_1]

Effective Altruism has been a controversial topic on social media, so today my guest and I are going to settle the question once and for all: Is it good or bad? Jonas Sota is a Software Engineer at Rippling, BA in Philosophy from UC Berkeley, who’s been observing the Effective Altruism (EA) movement in the San Francisco Bay Area for over a decade… and he’s not a fan. 00:00 Introduction 01:22 Jonas’s Criticisms of EA 03:23 Recoil Exaggeration 05:53 Impact of Malaria Nets 10:48 Local vs. Global Altruism 13:02 Shrimp Welfare 25:14 Capitalism vs. Charity 33:37 Cultural Sensitivity 34:43 The Impact of Direct Cash Transfers 37:23 Long-Term Solutions vs. Immediate Aid 42:21 Charity Budgets 45:47 Prioritizing Local Issues 50:55 The EA Community 59:34 Debate Recap 1:03:57 Announcements Show Notes Jonas’s Instagram: @jonas_wanders [https://www.instagram.com/jonas_wanders] Will MacAskill’s famous book, Doing Good Better: https://www.effectivealtruism.org/doing-good-better [https://www.effectivealtruism.org/doing-good-better] Scott Alexander’s excellent post about the people he met at EA Global: https://slatestarcodex.com/2017/08/16/fear-and-loathing-at-effective-altruism-global-2017/ [https://slatestarcodex.com/2017/08/16/fear-and-loathing-at-effective-altruism-global-2017/] Watch the Lethal Intelligence Guide [https://www.youtube.com/watch?v=9CUFbqh16Fg], the ultimate introduction to AI x-risk! PauseAI, the volunteer organization I’m part of: https://pauseai.info [https://pauseai.info] Join the PauseAI Discord — https://discord.gg/2XXWXvErfA [https://discord.gg/2XXWXvErfA] — and say hi to me in the #doom-debates-podcast channel! Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates] This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com [https://lironshapira.substack.com?utm_medium=podcast&utm_campaign=CTA_1]

Matthew Adelstein, better known as Bentham’s Bulldog [https://benthams.substack.com] on Substack, is a philosophy major at the University of Michigan and an up & coming public intellectual. He’s a rare combination: Effective Altruist, Bayesian, non-reductionist, theist. Our debate covers reductionism, evidence for god, the implications of a fine-tuned universe, moral realism, and AI doom. 00:00 Introduction 02:56 Matthew’s Research 11:29 Animal Welfare 16:04 Reductionism vs. Non-Reductionism Debate 39:53 The Decline of God in Modern Discourse 46:23 Religious Credences 50:24 Pascal's Wager and Christianity 56:13 Are Miracles Real? 01:10:37 Fine-Tuning Argument for God 01:28:36 Cellular Automata 01:34:25 Anthropic Principle 01:51:40 Mathematical Structures and Probability 02:09:35 Defining God 02:18:20 Moral Realism 02:21:40 Orthogonality Thesis 02:32:02 Moral Philosophy vs. Science 02:45:51 Moral Intuitions 02:53:18 AI and Moral Philosophy 03:08:50 Debate Recap 03:12:20 Show Updates Show Notes Matthew’s Substack: https://benthams.substack.com [https://benthams.substack.com] Matthew's Twitter: https://x.com/BenthamsBulldog [https://x.com/BenthamsBulldog] Matthew's YouTube: https://www.youtube.com/@deliberationunderidealcond5105 [https://www.youtube.com/@deliberationunderidealcond5105] Lethal Intelligence Guide, the ultimate animated video introduction to AI x-risk – https://www.youtube.com/watch?v=9CUFbqh16Fg [https://www.youtube.com/watch?v=9CUFbqh16Fg] PauseAI, the volunteer organization I’m part of — https://pauseai.info/ [https://pauseai.info] Join the PauseAI Discord — https://discord.gg/2XXWXvErfA [https://discord.gg/2XXWXvErfA] — and say hi to me in the #doom-debates-podcast channel! Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com [https://doomdebates.com/] and to youtube.com/@DoomDebates [https://youtube.com/@DoomDebates] This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com [https://lironshapira.substack.com?utm_medium=podcast&utm_campaign=CTA_1]