The Vernon Richard Show

In this episode of the Vernon and Richard show, the hosts engage in light-hearted banter about football before diving into a deep discussion on QA, QE, and testing. They explore the concept of 'shift left' in software development, comparing its application in agile versus waterfall methodologies. The conversation shifts to the evolving roles of QA and QE in the context of AI's impact on the industry, emphasizing the importance of task analysis and building a quality culture within teams. The episode concludes with reflections on managing expectations in QA roles and the future of jobs in the field.

This episode is about the struggle to explain, measure, and name the work testers and quality advocates actually do — especially when traditional labels and metrics fall short. Links to stuff we mentioned during the pod: * 05:05 - Defect Detection Rate (DDR) * The rate at which bugs are detected per test case (automated or manual) * No. of defects found by test team / No. of Test Cases executed) *100 * 15:06 - David Evans' LinkedIn [https://www.linkedin.com/in/davidevansagilecoach/] * 24:57 - Janet Gregory * Janet's website [https://janetgregory.ca/about/] * Janet's LinkedIn [https://www.linkedin.com/in/janetgregory/] * 26:01 - Defect Prevention Rate * Perplexity search results here [https://www.perplexity.ai/search/ae7d7e62-f15d-4893-b2d5-e80a48b881ed] * 28:28 - Jerry Weinberg * Jerry's Wikipedia page [https://en.wikipedia.org/wiki/Gerald_Weinberg] (his books are highly recommended) * 49:33 - Shift-Left: The concept of moving testing activities earlier in the software development lifecycyle. Some resources [https://www.perplexity.ai/search/ff65175c-5f65-4de6-828f-26b7c2edff0f] explaining the Shift-Left concept (Perplexity link) 00:00 - Intro 01:11 - Welcome & "woke" testing 😳 03:15 - QA, QE, Testing… whatever we call it, how do we measure if we're doing a good job? 03:44 - Vernon’s first experience with testing metrics: more = better? 05:00 - Defect Detection Rate enters the chat 06:41 - Rich reverse engineers quality skills needed in the AI era 10:54 - How do we know if we’re doing any of this well? 12:40 - Trigger warning: the topic of coverage is incoming 😅 16:54 - Bugs in production 21:09 - Automation metrics: flakiness, pass rates, and execution time 24:29 - Can you measure something that didn’t happen? (Prevention metrics) 27:43 - Do DORA metrics actually measure prevention? 32:03 - Here comes Jerry! 33:50 - The one metric the business cares about... 36:23 - QA vs QE: whose “quality” are we "assuring"? 39:25 - What's the story behind the numbers? 48:29 - Rich brings in Shift Left Testing 50:14 - Metrics that reach beyond engineering 53:14 - Rich gets a new perspective on QE and the business 56:50 - Who does this work? Testers? QEs? Or someone else?

In this episode, Richard and Vernon delve into the complexities of Quality Assurance (QA), Quality Engineering (QE), and testing in software development. They explore the evolution of these concepts, their interrelations, and the importance of metrics in assessing quality. The conversation highlights the need for a holistic approach to quality, emphasizing that both prevention and detection of bugs are essential. The hosts also discuss the challenges of defining these terms and the future of quality in the industry. Links to stuff we mentioned during the pod: * 08:50 - Dan Ashby * We're referring to Dan's [https://www.linkedin.com/in/dan-ashby/]'s excellent post called "Continuous Testing [https://danashby.co.uk/2016/10/19/continuous-testing-in-devops/]" (featuring his famous diagram!) * 17:13 - Jit Gosai * Jit's blog [https://www.jitgo.uk/]  * Jit's Quality Engineering Newsletter [https://qualityeng.substack.com/]  * Jit's LinkedIn [https://www.linkedin.com/in/jitgo/] * 19:24 - Quality Talks Podcast * Stu's Quality Talks podcast [https://www.youtube.com/@QualityTalksPodcast] that he co-hosts with Chris Henderson * Stu's LinkedIn [https://www.linkedin.com/in/stuart-day-77155330/] * Chris's Linkedin [https://www.linkedin.com/in/christopher-henderson-344231156/] * 19:55 - The Testing Peers podcast [https://testingpeers.com/] * 22:00 - DORA Metrics: DORA metrics are a set of key performance indicators developed by Google’s DevOps Research and Assessment team to measure the effectiveness of software delivery and DevOps processes, focusing on both throughput and stability * 26:13 - A link from Episode 10 [https://thevernonrichardshow.com/10#t=28m40s] where Vern discusses Glue Work (be sure to check out the show notes on that episode) * Quick overview [https://dora.dev/guides/dora-metrics-four-keys/] of DORA metrics * 34:43 - The Credibility Playbook * A video course [https://stan.store/testerfromleic/p/speak-up-be-heard-get-taken-seriously-at-work] by Vernon as he experiments with building digital products. * Check it out and let him know what you think of it! 😊 * 46:24 - Ali Abdaal * Ali's website [https://aliabdaal.com/] * Ali's YouTube [https://www.youtube.com/aliabdaal] 00:00 - Intro 01:36 - Welcome 02:40 - Today's topic: What the hell is QA? QE? Testing? And is it all changing? 03:00 - Why is this bugging Rich? 05:11 - Fruit fly tangent 🍌🍊🍎🪰🐝🦋 06:27 - Rich's take on QA, QE, and Testing 08:31 - Vern's take on QA, QE, and Testing 11:15 - Is shift-left testing the same as QE? 13:05 - When the team tests early... is that QE then?! 16:18 - What's the big deal if we can’t define QE clearly? 19:27 - Why the Efficiency Era makes this even harder 22:55 - Trying to draw the Testing, QA, QE, Venn diagram 27:24 - Getting the QA, QE, Testing blend just right. What's the right mix? 29:52 - The kinds of work we take on as our careers grow 34:08 - What Testers get rewarded for 45:34 - How Ali Abdaal helped Vern think differently about quality 48:18 - Rich talks measurement

In this conversation, Vernon and Richard explore the evolving role of AI in quality engineering and software development. They discuss how AI can enhance quality control processes, the importance of embedding quality early in the development cycle, and the potential challenges and opportunities that arise from integrating AI tools. The conversation also touches on the need for skill development and community engagement in adapting to these changes, as well as the implications for roles within the industry. Description and Thumbnail made with AI to assess the quality, we had to! 00:00 - Intro 01:02 - Welcome and footy ⚽️ 02:15 - Today's topic: The impact that AI may or may not have on Quality Engineering 03:22 - Rich's wild idea about AI and software quality 14:10 - Vern asks a clarifying question 22:45 - Communities of excellence… for machines?! 24:03 - Vern thinks there's an obvious risk that follows from this idea... 31:31 - Rich addresses the risk (Oracles, prompts, and tester superpowers) 36:13 – Reflection: the hidden skill AI forces on us 41:40 – Shifting in all directions (not just left) 43:04 - Feeding your past self into an AI: smart or scary? 45:53 – Operation 400 subscribers (and bot listeners) 47:13 – Tony Bruce calls us out on sloppy show notes and outro Links to stuff we mentioned during the pod: * 04:18 - Shift-Left: The concept of moving testing activities earlier in the software development lifecycyle. * Some resources [https://www.perplexity.ai/search/ff65175c-5f65-4de6-828f-26b7c2edff0f] explaining the Shift-Left concept (Perplexity link) * 25:35 - Rob Bowley * Rob's LinkedIn * The post [https://www.linkedin.com/posts/robertbowley_typing-is-not-the-bottleneck-illustrated-activity-7363460996352327680-af7B?utm_source=share&utm_medium=member_desktop&rcm=ACoAAACJLG0BQTHTewSrvVgR_kl1D1z_Hk0JA_g] Vernon referred to... * ...a follow-up [https://www.linkedin.com/posts/robertbowley_yet-another-study-showing-that-ai-assisted-activity-7364190534636032003-uO0X?utm_source=share&utm_medium=member_desktop&rcm=ACoAAACJLG0BQTHTewSrvVgR_kl1D1z_Hk0JA_g] post not long after that one too! * 26:40 - Alan Page * Alan and Brent [https://www.linkedin.com/in/brentmjensen/]'s podcast [https://podcasters.spotify.com/pod/show/abtesting] * Alan's LinkedIn [https://www.linkedin.com/in/a-l-a-n/] * 34:43 - Saskia Coplans * Digital Interruption [https://www.digitalinterruption.com/] Saskia's cybersecurity consultancy * REXscan [https://www.rexscan.com/] Saskia's automated mobile application vulnerability scanner * Saskia's LinkedIn [https://www.linkedin.com/in/saskiacoplans/] (highly recommended follow) * 41:49 - Paul Coles * Paul Coles published 3 of his 4 part series "The Subtle Art of Hearding Cats [https://dev.to/paul_coles_633f698b10fd6e]" over on Dev.To [https://dev.to] Recommended reading! * Paul's LinkedIn [https://www.linkedin.com/in/paul-c-268a8a24/] * 43:09 - Maaret Pyhäjärvi * Maaret's website [https://maaretp.com/] * Maaret's blog [https://visible-quality.blogspot.com/] * Maaret's LinkedIn [https://www.linkedin.com/in/maaret/]

In this episode of the Vernon Richard show, the hosts discuss their experiences with AI tools and agents, focusing on the challenges and lessons learned from using these technologies in coding and software engineering. They explore best practices for utilizing AI effectively, the importance of context in interactions with AI, and the future of AI agents in the workplace. The conversation highlights the balance between leveraging AI for efficiency while maintaining control and understanding of the underlying processes. Links to stuff we mentioned during the pod: * 09:16 - The LinkedIn post [https://www.linkedin.com/posts/kellyvaughn_replit-deleted-a-customers-entire-production-activity-7352555110784200704-X7lW?utm_source=share&utm_medium=member_desktop&rcm=ACoAAACJLG0BQTHTewSrvVgR_kl1D1z_Hk0JA_g] talking about Replit messing with someone's production code 😳 * And the link to the thread [https://x.com/jasonlk/status/1946065483653910889] of person who went through it * The tool in question, Replit [https://replit.com/] * 13:01 - Rich's LinkedIn post [https://www.linkedin.com/posts/friendlytester_ive-spent-the-last-few-weeks-vibing-with-activity-7349052146266300416-c5dR?utm_source=share&utm_medium=member_desktop&rcm=ACoAAACJLG0BQTHTewSrvVgR_kl1D1z_Hk0JA_g] with his tips * 14:21 - GitHub Copilot [https://github.com/features/copilot] * 18:09 - VS Code [https://code.visualstudio.com/] * 29:01 - Folks at different ends of the "AI Enthusiasm Spectrum" * On the enthusiastic end * Jason Arbon [https://www.linkedin.com/in/jasonarbon/] is on the positive side and is always creating something interesting like... * testers.ai [https://testers.ai] * On the unenthusiastic end * Keith Klain [https://www.linkedin.com/in/keithklain/] has created a reading list to help get us up to speed... * Keith's reading AI reading list [https://www.linkedin.com/posts/keithklain_resources-activity-7340272047123492864-XKWa?utm_source=share&utm_medium=member_desktop&rcm=ACoAAACJLG0BQTHTewSrvVgR_kl1D1z_Hk0JA_g] * You can see his full resources [https://qualityremarks.com/resources/] list here * Maaike Brinkhof [https://www.linkedin.com/in/keithklain/] has a bunch of thought-provoking posts on the topic... * ...like this one [https://www.maaikebrinkhof.nl/llm-circle-jerking/] * and this one [https://www.maaikebrinkhof.nl/dear-software-testers-is-your-critical-thinking-ability-even-alive/] * 34:44 - Want to know what "conflabulation" means? Listen to Martin explain [https://open.spotify.com/episode/39A3aPhW8AG1upuCWunT1k?t=935] it on the Ghost in th code [https://open.spotify.com/show/6DcXFaITfElDQvjUqN1BOB] podcast (that's not a typo!) * 37:24 - What is Context Engineering [https://www.perplexity.ai/search/01c5c358-78d3-4794-87b8-d722a0c7d5ed]? Perplexity has answers! * 46:38 - The legendary Lt. Geordi La Forge [https://en.wikipedia.org/wiki/Geordi_La_Forge] from Star Trek: The Next Generation [https://en.wikipedia.org/wiki/Star_Trek:_The_Next_Generation]. * 51:48 - After recording, the very cool Paul Coles [https://www.linkedin.com/in/paul-c-268a8a24/] published his article The Subtle Art of Herding Cats: Why AI Agents Ignore Your Rules (Part 1 of 4 [https://dev.to/paul_coles_633f698b10fd6e/the-subtle-art-of-herding-cats-why-ai-agents-ignore-your-rules-part-1-of-4-5fhd], explaining the topic of Context Engineering. It’s brilliant! * 59:04 - The promises of technology [https://www.linkedin.com/posts/janwilmans_a-regular-reminder-for-people-that-worry-activity-7338821097733124097-B_Oy] over the years... * 60:50 - The always insightful Meredith Whittaker of Signal [https://en.wikipedia.org/wiki/Signal_(software)] fame, where is the president and services on its board of directors, explains the privacy and security concerns [https://www.youtube.com/live/AyH7zoP-JOg?t=3016s] with agentic technology. * Watch the clip, then go back and watch the whole thing! 00:00 - Intro 01:17 - Welcome 01:30 - TANGENT BEGINS... All kinds of egregious waffling follows. Skip to the actual content at 08:34 01:31 - Rich VS Tree Stump 01:57 - What on earth did Rich need the pulley for? 02:26 - Vern's nerdy confession and pulley confusion 02:52 - Does Rich live next door to Tony Stark?! 03:22 - What to do when you need a steel RSJ 03:35 - We admit defeat. 03:36 - Welcome to Rich's Garden Adventures Podcast! 07:25 - What has Vern been up to? 08:34 - We attempt to segue into the episode at last! 08:35 - TANGENT ENDS... 08:51 - Rich’s POC: using agents to help build AI tools 09:45 - The Replit disaster: vibe coding meets deleted production data 11:12 - Sociopathic assistants and the case for AI gaslighting 11:55 - Vernon wants his team experimenting with AI tools 12:50 - Rich explains the context for his latest AI adventures 13:18 - Rich’s bench project and “putting the engineering hat on”  15:22 - Setting up the stack and staying in control  16:53 - A familiar story: things were going fine until they weren’t  17:00 - Ask vs Edit vs Agent mode in Copilot explained  19:06 - The innocent linting error that spiralled out of control  21:16 - Stuck in a loop: “I didn’t know what it was doing, but I let it keep going”  22:11 - The fateful click: “I’m going to reset the DB”  23:10 - The aftermath: no data, no damage… but very nearly  23:33 - Security wake-up call: agents are acting as you  24:39 - You can’t fix what you don’t know it broke  25:52 - Can you interrupt an agent mid-task?  27:14 - When agents get “are you sure?” moments  28:15 - Tea breaks as a dev strategy: outsourcing work to agents  29:24 - Jason Aborn vs Keith & Maaike: where Rich sits on the AI enthusiasm spectrum  30:41 - Tip1. The first of Rich’s 6 agent tips: commit after every interaction 32:12 - Why trusting the “keep all” button is risky  34:01 - Writing your own commits vs letting the agent do it  35:26 - When agents lose the plot: reset instead of fixing  36:55 - “You’re insane now, GPT. I’m giving you a break.”  37:54 - Tip 2: Make the task as small as possible  39:59 - The middle ground between 'ask' and full agent delegation  41:12 - Tip 3: Ask the agent to break the task down for you  43:36 - The order matters: why you shouldn’t start with the form UI  44:33 - Vernon compares it to shell command pipelines  45:09 - It can now open browsers and run Playwright tests (!)  46:23 - Star Trek and the rise of the engineer-agent hybrid  47:57 - Tips 4–6: Test often, review the code, use other models  49:39 - Pattern drift and the importance of prompt templates  50:51 - Vernon’s nemesis: m dashes, emojis, and being ignored by GPT  51:48 - Context engineering vs prompt engineering  52:43 - When codebases get too big for agents to cope  53:40 - Why agents sometimes act dumber than your IDE  54:32 - The danger of outsourcing good practices to AI  54:48 - Spoilers: Rich’s upcoming keynote at TestIt  55:01 - Agents don’t ask why — they just keep going  56:42 - Goals vs loops: when failure isn’t part of the plan  58:32 - The question of efficiency: is training agents worth it?  59:47 - Rich’s take: we’ll buy agents like we buy SaaS  61:08...