Superintelligence: AGI and ASI (E.44)

Superintelligence: AGI and ASI (E.44)

Everyone debates when AGI will arrive. Fewer people ask what happens next. In this episode, we break down the difference between AGI and ASI, why humans instinctively personify AI systems, and the societal challenges that emerge when intelligence becomes abundant. We also discuss AI companionship, regulation, creativity, and what remains uniquely human in an AI-driven world. 00:00 — What AGI actually means 08:00 — Why humans personify AI 18:00 — AI companions and social consequences 29:00 — The risks of ASI and superintelligence 42:00 — Regulation, incentives, and the future of AI The biggest challenge of AI may not be the technology itself, it may be how humans choose to relate to it.

Principles of Evals: The Future of GenAI Evaluation (E.43)

LLMs are optimized to sound convincing—not to know when they’re wrong. In this episode, Deanna Emery breaks down why hallucinations are fundamentally tied to how language models work, why confidence is often disconnected from correctness, and how better evaluation strategies can make AI systems more reliable in production. We also get into uncertainty, semantic reasoning, and what humans still do better than models. 00:00 — Why LLMs hallucinate confidently 09:00 — The limits of current eval systems 18:00 — Why uncertainty matters in AI 27:00 — Semantic reasoning vs memorization 38:00 — What humans still do better than models The biggest risk in AI isn’t wrong answers. It’s wrong answers delivered with confidence.

AI FDE at Databricks (E.42)

Building a great AI team takes more than hiring smart people. In this episode, Brooke Wenig breaks down how Databricks built the AI FDE organization, why culture compounds faster than technical skill, and what separates high-trust engineering teams from teams that slowly degrade over time. We also get into mentoring, hiring in the age of AI coding tools, and why software engineering fundamentals matter more than ever. 00:00 — How Databricks built the AI FDE team 08:00 — AI cheating in technical interviews 19:00 — Why culture degrades as teams scale 31:00 — Building a team brand around specialists 43:00 — What skills matter most in the AI era Great AI teams aren’t built through rules. They’re built through people who reinforce the right standards every day.

AI for Coparenting: How AI can Deescalate Coparenting (E.41)

Most AI startups optimize speed, automation, or revenue. Sol built one to stop people from emotionally destroying each other. After a brutal divorce and years trapped inside high-conflict co-parenting, he realized the real problem wasn’t logistics, it was emotional escalation through constant communication. BestInterest uses AI to filter manipulative, hostile, and triggering messages before they reach the other parent, turning AI into a psychological buffer instead of a chatbot. AI has the potential to bridge challenging social gaps. While there is a lot of fear around this capability, this case study for navigating challenging parenting relationships showcases how unbiased AI personas can mitigate these problems. Chapters * 00:00 The Origin Story of Best Interest * 06:06 AI as a Mediator in Co-Parenting * 12:57 Designing for Impersonal yet Supportive Nature * 18:00 The Best Interest of the Kids * 22:57 Fine-Tuning Communication Expectations * 28:10 Product Insight and Differentiation * 36:52 Passion-Driven Work and Meaningful Impact * 42:37 AI and Human Communication * 49:37 AI as an Engine of Peace

How to Prevent Doomsday: Guardrails, Alignment, and Education (E.40)

AI alignment breaks the moment we assume intelligence automatically produces morality. Dr. Peter R. Solomon argues the real danger isn’t sentient AI becoming evil, it’s AI inheriting no emotional history, no family structure, and no reason to value human survival. The conversation moves from CRISPR in high schools to AI-generated writing, autonomous agents, synthetic memory, and why “guardrails” fail when systems evolve faster than institutions can regulate them. The deeper point: humans trained AI to think, but not necessarily to care. 00:00 Why science education kills curiosity 06:00 The AI extinction scenario nobody wants to model 15:45 Why static guardrails fail in production systems 27:00 The AI-written paragraph that appeared unprompted 39:50 AI as a cooperative intelligence, not a replacement The systems we’re building already shape human behavior. The question is whether they’ll eventually shape human survival.