Episode 9: AI, Testing and DORA with Lisa Crispin

Episode 10: deep dive in AI era testing research

With developers using AI, how do quality professionals deal with reviewing increasing amounts of work to review? How can we keep pace with AI-accelerated coding? In this episode, we spoke about two different kinds of quality teams – proactive and reactive. We then explore the coordination and rework challenges that AI accelerated development poses to each of these teams. We finally end with a sobering prediction: reactive quality practices that were already wasteful in the pre-AI era will now become unsustainable. Research: https://github.com/BeyondQuality/beyondquality/tree/main/research/ai-era-testing [https://github.com/BeyondQuality/beyondquality/tree/main/research/ai-era-testing] Sources: - B. W. Boehm, "Software Engineering”, 1976 (https://selab.netlab.uky.edu/homepage/boehm-sw-eng-paper.pdf [https://selab.netlab.uky.edu/homepage/boehm-sw-eng-paper.pdf]) - Stafford Beer, "Brain of the firm", https://www.amazon.com/Brain-Firm-Stafford-Beer/dp/047194839X [https://www.amazon.com/Brain-Firm-Stafford-Beer/dp/047194839X]

Episode 9: AI, Testing and DORA with Lisa Crispin

Vitaly, Anupam, and Maryia sit down with Lisa Crispin (independent consultant, DORA community guide, co-author with Janet Gregory of the Agile Testing books). Lisa watched the industry fire testers when automation arrived in the early 2000s. The same panic is repeating with AI, but non-deterministic AI output demands more quality thinking, not less. Big batches never worked well; now they stop working entirely. Testing-after never worked well; now it can't work at all. DORA's 2025 research captures the pattern: AI is an amplifier. Whatever teams were already doing, good or bad, gets scaled up. The capabilities that correlate with high-performing AI-augmented teams are the same ones that made human teams high-performing before AI. But the industry still sells AI as a way to make individual developers 10x more productive. The result: software delivery instability is rising even as individual productivity goes up, because code reaches production badly tested and the rework drives burnout. Vitaly thinks AI is a great collaboration amplifier: the typing bottleneck is gone, freeing cross-functional teams to iterate faster, together defining why and how the software is to be built and tested. Links: * Lisa's Linkedin: https://www.linkedin.com/in/lisacrispin/ [https://www.linkedin.com/in/lisacrispin/] * BQ research: https://github.com/BeyondQuality/beyondquality/blob/main/research/ai-era-testing/analysis.md [https://github.com/BeyondQuality/beyondquality/blob/main/research/ai-era-testing/analysis.md] * DORA AI Capabilities report: https://cloud.google.com/resources/content/2025-dora-ai-capabilities-model-report [https://cloud.google.com/resources/content/2025-dora-ai-capabilities-model-report] Sponsors: 1. Qase https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy [https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy] 2. Regent https://regent.se [https://regent.se] 3. TestSolutions https://testsolutions.de/en/focus/ai-testing [https://testsolutions.de/en/focus/ai-testing]

Episode 8: AI generates code. Who carries the understanding?

Vitaly, Anupam, and Maryia sit down with Liliia Abdulina — Head of QA at JetBrains and co-researcher on the BeyondQuality AI-era testing study — who came to this topic the way most useful research starts: with a real problem on her hands. Developers at JetBrains were accelerating with AI, QA was becoming the bottleneck, and she wanted to understand why. The conversation that follows makes the case that this bottleneck isn't new — managers were already complaining "why is testing delaying us" long before AI. What AI did was make a structural problem impossible to ignore. When humans wrote code, understanding came for free: the person who wrote it carried the mental model. Now code is generated at machine speed and nobody carries it. The group calls this cognitive debt — distinct from technical debt, and harder to see. But the subtler casualty is learning. When a tester finds a bug, the developer carries that knowledge forward and writes fewer similar bugs next cycle. Agents don't learn across sessions, so the loop breaks. The industry's response has been to flood the market with AI inspection tools, because inspection is easy to automate. Prevention — which costs less — requires domain understanding that agents don't have, and nobody is building tools for that. Links: 1. Lilia's Linkedin: https://www.linkedin.com/in/liliiaabdulina/ [https://www.linkedin.com/in/liliiaabdulina/] 2. research discussion https://github.com/BeyondQuality/beyondquality/discussions/28 [https://github.com/BeyondQuality/beyondquality/discussions/28] and current artifact https://github.com/BeyondQuality/beyondquality/blob/main/research/ai-era-testing/analysis.md [https://github.com/BeyondQuality/beyondquality/blob/main/research/ai-era-testing/analysis.md] 3. https://devblogs.microsoft.com/dotnet/ten-months-with-cca-in-dotnet-runtime/ [https://devblogs.microsoft.com/dotnet/ten-months-with-cca-in-dotnet-runtime/] 4. https://arxiv.org/pdf/2603.22106v2 [https://arxiv.org/pdf/2603.22106v2] Sponsors: 1. Qase https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy [https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy] 2. Regent https://regent.se [https://regent.se] 3. AnuKrit https://anukrit.de [https://anukrit.de]

Episode 7: "On trust"

Maryia, Vitaly, and Anupam sit down with Asya Isakova — an organisational psychologist who's been studying how people function at work since 2007 — to explore her BeyondQuality research on trust in teams. The most counterintuitive finding: trust matters most in hierarchical organisations, precisely the ones that tend to destroy it with KPIs and approval chains. In high-hierarchy teams, subordinates decide what information to share based on whether they trust their manager — and if they don't, the entire decision chain corrupts. The conversation unpacks how lack of trust works like a tax: full workdays wasted on two-minute tasks, teams of 20 writing reports instead of working, $15 purchase orders going through five approvals. Why vulnerability — the core of trust — is the hardest thing for leaders to accept. And why preventive quality work like pair programming or shifting testing left simply cannot start without trust: you can't prove a prevented bug, so if nobody trusts you, nobody lets you try. Links: 1. Asya's Linkedin: https://www.linkedin.com/in/asyaisakova/ [https://www.linkedin.com/in/asyaisakova/] 2. Asya's research proposal: https://github.com/BeyondQuality/beyondquality/discussions/26 [https://github.com/BeyondQuality/beyondquality/discussions/26] 3. Organizational Justice meta-analysis: https://psycnet.apa.org/buy/2013-06399-001 [https://psycnet.apa.org/buy/2013-06399-001] 4. Trust factors and performance review: https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1081086/full [https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1081086/full] 5. Team factors and trust: https://onlinelibrary.wiley.com/doi/abs/10.1002/job.2436 [https://onlinelibrary.wiley.com/doi/abs/10.1002/job.2436] Sponsors: 1. Qase https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy [https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy] 2. Regent https://regent.se [https://regent.se] 3. AnuKrit https://anukrit.de [https://anukrit.de]

Episode 6: "on testing economics"

Maryia, Vitaly, and Anupam sit down with returning guest Mike Jarred — an exceptionally seasoned testing consultant — to unpack Vitaly’s "Economics of Testing" research and how it proposes we talk about quality. The conversation starts with a familiar problem: quality and risk are hard to "make tangible", so testing is too often reduced to outputs and activity. From there, it becomes a practical discussion of how to translate quality work into the language leadership (and other functions) actually uses: the economical language of risks, value and money. Links: 1. Mike's Linkedin: https://www.linkedin.com/in/mikejarred/ [https://www.linkedin.com/in/mikejarred/] 2. Vitaly's research on economics of testing: https://beyondquality.org/research/testing_economics/testing_economics [https://beyondquality.org/research/testing_economics/testing_economics] Sponsors: 1. Qase https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy [https://qase.io/?utm_source=youtube&utm_medium=beyondquality&utm_campaign=devadvocacy] 2. Regent https://regent.se [https://regent.se] 3. AnuKrit https://anukrit.de [https://anukrit.de]