Test-Talks.com

The Confidence Asymmetry Principle - Why 80% of future AI compute will be AI Verification (The Correlated Blindness Penalty Theory)

The Confidence Asymmetry Principle, AI Verification Economics, Correlated Blindness & When Code Generation Becomes Close to Zero - Confidence Becomes the Quality of the Product! For seventy years the scarce resource of software engineering was the production of correct code, and every methodology of the field was an attempt to manage that scarcity. Generative AI is dissolving it. The marginal cost of producing a plausible candidate program is collapsing toward a floor set only by the price of compute. AI is making software engineering radically cheaper to generate — but not necessarily cheaper to trust. In this episode, we explore The Confidence Asymmetry Principle, a working paper around Confidence Engineering Manifesto by Jason Arbon (2026), which argues that the real economic shift in AI-infused software is not the collapse in the cost of code, but the rising importance of the cost of confidence. As generative AI drives the marginal cost of producing software toward zero, a new constraint emerges: verifying that the generated artifact is correct, safe, reliable, and fit for production. The paper argues that generation and verification are governed by different cost laws — and those laws are now diverging. We unpack the core idea that in an AI software economy, code becomes cheap, but confidence becomes scarce. The conversation explores: * Why verification becomes more expensive as generation becomes cheaper * The three irreducible floors beneath verification cost: undecidability, execution, and interaction * How Rice’s theorem and the halting problem shape the future economics of software testing * Why “AI testing AI” can create correlated blind spots rather than genuine assurance * The Correlated-Blindness Penalty and why independent evidence matters * The Confidence Bound and the cost of reaching justified confidence * Why verification compute may become the dominant workload of future AI compute * How Confidence Engineering and AI Assurance emerge as practical responses to this shift The Confidence Asymmetry Principle the work of Jonathon Wright and Jason Arbon, in this episode reframes the AI engineering productivity revolution through a sharper lens: the bottleneck is no longer whether AI can generate software, but whether organizations can produce enough evidence to trust what it generates. The constructive consequence is not pessimism. It is that verification becomes the central discipline of software engineering, and that the discipline already has a name. Confidence Engineering — and its companion, AI Assurance — is the practice the Principle demands: justified confidence over performative coverage, decorrelated evidence over correlated convenience, continuous evaluation over static validation, a probability with an interval over a green check The next era of software will not be defined by who can generate the most code. By The Confidence Asymmetry Principle, generation is becoming free, and free things do not confer advantage. The era will be defined by who can justify confidence in what has been generated — at a known residual risk, on evidence whose independence can be exhibited and audited. When code becomes free, confidence becomes the product. The work, and the compute, and the value, follow the confidence, #TestTalks #ConfidenceEngineering #AIAssurance #AgenticAI #AITesting #QualityEngineering #TrustworthyAI #SoftwareTesting #LLM #AIConfidence #GenAI #AIVerification

World Digital Report 2026 - The Future of Quality Engineering - AI Confidence Engineering, AI Literacy & AI Assurance

AI adoption is accelerating at a pace few organizations are prepared for. In this episode, we dive deep into the findings of the World Digital Report 2026 — one of the most comprehensive global studies examining the intersection of AI, digital transformation, quality engineering, enterprise trust, and the future of work. Featuring insights and research from Jonathon Wright alongside contributors including Jason Arbon, Joel Deutscher, Alex Belotsky, Ben Daniels, Ethan Bhundia,  Milarna Parkerpayne, & Paul Gerrard this conversation explores what happens when organizations deploy AI faster than they develop AI literacy, governance, and assurance capabilities. We unpack some of the report’s most important themes: * Why AI literacy has become the defining enterprise capability of 2026 * The rise of Agentic AI and autonomous systems inside modern organizations * How quality engineering roles are evolving from execution toward AI Confidence Engineering * The widening gap between AI adoption and organizational trust * Why prompt engineering, AI governance, and AI assurance are now business-critical * The challenge of testing non-deterministic systems at enterprise scale * How digital transformation is shifting from automation toward autonomous orchestration We also explore the broader implications of AI-powered productivity, enterprise confidence, software quality risk, and the future of human-machine collaboration as organizations move from experimentation into operational dependency on AI systems. From AI-generated testing and autonomous workflows to assurance engineering and organizational readiness, this episode examines the data behind one of the largest digital transformation shifts seen in decades. Because the question is no longer whether organizations will adopt AI. The real question is whether they can do it safely, responsibly, and with confidence. #TestTalks #WorldDigitalReport #AI #AgenticAI #AIAssurance #QualityEngineering #DigitalTransformation #AITesting #TrustworthyAI #GenAI #FutureOfWork

Agentic AI - Model Learning Based Testing (MlBT) - Digital Twin Testing (DTDL) meets Context Understanding (RARE / MIRAGE)

Test-Talks.com welcomes Karl McCarron an ML Architect at Keysight within the Eggplant AI R&D labs team in Cambridge, where he defines technical solutions and researches emerging AI technologies to advance the product's future vision in AI Agentic Testing. With a strong foundation in theoretical physics, having completed his MASt in Applied Mathematics from the University of Cambridge with Distinction and a First Class Master's in Physics from the University of Oxford, his work bridges advanced mathematical concepts with practical machine learning applications. His expertise spans from quantum field theory and particle physics to applied ML architecture, bringing rigorous analytical approaches to enterprise-grade intelligent automation. Join the Eggplant AI R&D Labs team as they boldly reimagine the future of software testing in this thought-provoking brainstorming session. Exploring everything from transforming legacy test assets into intelligent models to the emergence of "model learning testing," they tackle the hard question: how do we make testing more valuable in an AI-driven world? Dive into discussions about context engineering, agentic AI systems, digital twins, and why 90% of software testing might need a complete rethink. If you're curious about where testing is headed in 2026 and beyond complete with newly coined concepts like "software archeology" and "value-informed testing" this wide-ranging conversation is essential listening for anyone in QA, test automation, or software development.

AI-Assurance - Vibe Quality Testing of AI Infused Systems (feat. SureLine.AI)

Test-Talks.com welcomes Adam Sandman who has been working in the IT industry for the past 25 years in areas such as architecture, agile development, testing and project management. Currently Adam is the Founder and CEO of Inflectra Corporation, where he is interested in technology, business and enabling people to follow their passions. At Inflectra, Adam has been responsible for researching the tools, technologies and processes in the software testing and quality assurance space.  In this episode will be a world exclusive of SureLine.AI [http://sureline.ai/] Go From Not Worrying About Your AI Agents to Knowing They are Safe, Reliable, and Effective: Ask SureLine™ to test your agent, like you were talking to an experienced test manager on your team. SureLine™ will design and implement dynamic tests, using collections of bespoke testing agents. It provides a report of the AI under test, highlighting strengths and weaknesses, risks, quality concerns, and more. SureLine™ is a whole new category of QA tool. Testing AI is not like normal testing. You can’t rely on traditional testing to really know the state of your AI. You need a quality assurance tool built from the ground up, that leverages AI to dynamically probe and test AI. You need SureLine™.

AI-Assurance - AI-Derived Vibe Quality Requirements Engineering (VARIED-Q)

Test-Talks.com welcomes Paul Gerrard legendary consultant, teacher, author, programmer, tester, conference speaker, rowing coach and publisher. He has conducted consulting assignments in all aspects of software testing and quality assurance, specializing in test assurance. He has presented keynote talks and tutorials at testing conferences across Europe, the USA, Australia, South Africa and won the most amount of lifetime achievements awards. In 2026, Context Engineering will be king, in this episode you will better understand DEFOSPAM an Vibe Requirements engineering approach to AI-in-the-loop (AIitL) to help create better requirements or stories, building on the success of the Business Story Method (published back in 2012) how can you utilize AI as a coach / mentor to help improve business requirements in 2026? #AIAssurance #automationtesting #testautomation #automationtesters #softwaretesting #functionalautomation #nonfunctionalautomation #testingcareer #e2eautomation #ai #aiintesting #autoops #aiaugmentedtesting #generativeai #AIAugmentedTesting #eggplantGAI #generativeAI