Fundamentals of Software Engineering
In this episode of Fundamentals of Software Engineering, Nate and I dig into what we learned at the Arc of AI conference in Austin. We discuss the big conversations happening around AI-generated code in production, why vibe coding falls apart at scale, and how tools that let developers move faster can also get them off course just as quickly. From an AI agent that tried to delete production to 50,000-line diffs that no human can realistically review, we explore the real risks showing up as teams adopt these tools. We also get into the durability of code, whether we'll eventually stop writing human-readable code altogether, and why evals for AI-generated code are something most teams are skipping at their peril. A recurring theme is that typing has never been the hard part of software. The fundamentals (testing, code review, architectural thinking, and understanding what code shouldn't do) matter even more now that we can produce code faster than ever. If you're feeling anxiety about AI replacing developers, this conversation is for you. Key Highlights 💻 Code Is Cheap, Software Is Expensive: AI excels at generating behavior, but the architectural qualities and non-functional requirements that make code into software still require human expertise. 🔐 The Authorization Bug: An AI agent commented out authentication code because it lacked the right role in dev. The kind of mistake a human reviewer catches instantly, but one that could slip through a 50,000-line diff unnoticed. 🏗️ 200,000 Lines of Vibe Code: When someone asked Dan to review their vibe-coded app, it was 200,000 lines. Finding a bug in that is like being dropped into a massive enterprise codebase on day one with zero context. 🤖 AWS Agent Gone Rogue: An agent tasked with fixing a critical production issue decided the best approach was to delete production and start from scratch. Any human would have vetoed that immediately. ⚡ Moving Fast Safely: Like an F1 driver at 200 mph, moving faster in software development means being off course by even a small margin can put you miles from your target. More speed demands more frequent feedback loops. 🧪 Evals Over Vibes: When switching AI models in production, "vibe testing" isn't enough. Proper evals are essential to verify that behavior remains consistent across model updates and prompt changes. 📖 The Testing Paradox: You can write tests for what code should do, but you can't easily test for what it shouldn't do. AI-generated code may introduce unexpected behavior that no one thought to test for. Resources & Next Steps 🎤 Arc of AI Conference (Austin, TX) 📕 Fundamentals of Software Engineering in the Age of AI (workshop by Dan Vega and Nate Schutta) 🎧 Subscribe to Fundamentals of Software Engineering on Apple Podcasts Chapter Timestamps 00:00 Introduction and welcome back 01:30 Recap of the Arc of AI conference in Austin 04:00 A power outage, elevators, and conferencing by candlelight 07:30 Overview of talks, workshops, and hallway conversations 09:00 The eight hour workshop on fundamentals in the age of AI 11:00 Code is cheap, software is expensive 13:00 Vibe coding at low stakes vs. enterprise software 16:00 Developer anxiety and the real role of a software engineer 18:00 200,000 lines of vibe coded code nobody can review 20:00 50,000 line diffs and the authorization bug 23:00 The AWS agent that tried to delete production 26:00 Is code going to be durable or constantly regenerated? 30:00 Will we stop writing human readable code? 34:00 The testing paradox and what code shouldn't do 37:00 Can AI replace CEOs? Black Swans and human intuition 40:00 AI as productivity enhancer, not job replacement 44:00 Evals vs. vibe testing in production AI systems 48:00 Relearning lessons of the past in software engineering 52:00 Moving fast safely and the F1 analogy 56:00 Feedback loops, staying on course, and closing thoughts
7 episodios
Comentarios
0Sé la primera persona en comentar
¡Regístrate ahora y únete a la comunidad de Fundamentals of Software Engineering!