Democracy, Bias, and the Case for Ethical AI: Data has history

The Future With AI

Hallucination in AI is not always a failure to be fixed. For some organisations it is a design choice. That observation, made on stage at Brunel University of London, is one of nine tensions this conversation surfaces, examines, and poses as challenges to be resolved as we live and work with AI. This is the Season 3 opening episode of The AI Adoption Podcast, recorded live in front of an audience at Brunel's Research Festival 2026. Five panellists joined me to discuss The Future With AI: Zahra Bahrololoumi CBE, President and CEO of Salesforce UK and Ireland; Lord Tim Clement-Jones, Liberal Democrat spokesperson for Science, Innovation and Technology and co-chair of the APPG on AI; Janusz Marecki, CEO and co-founder of FractalBrain; Maggie Sarfo, founder of Meres Consulting; and Alex Dalman, Managing Partner at Faith. The theme is the future WITH AI. Not the future of AI. The emphasis matters. The panel assumes AI is already ubiquitous and asks what life looks like inside that reality, and where the tensions lie that policymakers, business leaders, and academics need to face up to. Five tensions to listen for: • Salesforce's agentic AI now resolves between 82 and 84 per cent of complex customer queries on its help platform without human intervention, saving $100 million a year with no engineers displaced. Alex Dalman observes that most organisations have no real understanding of the effort required to get there. • Janusz Marecki argues that current large language models are unreliable by design: the underlying architecture samples from a probability distribution rather than reasoning from verified knowledge, and the training data itself contains what he describes as inaccurate, harmful, and toxic content. Zahra Bahrololoumi argues that people are smarter than we give them credit for, and that enterprise AI built on curated, governed data is a categorically different proposition. • 61 per cent of people in business are already using AI without having received any formal training. The government has committed to training 10 million people by 2030. The gap between those two assertions is where a great deal of the risk currently lives. • Lord Tim Clement-Jones describes graduates encountering only automated assessments throughout the early stages of job applications, with no human contact and no feedback until they clear several initial hurdles. Maggie Sarfo argues that AI should be making us more human by releasing capacity for emotional intelligence. Both observations are accurate about different parts of the same reality. • AI, as Alex Dalman notes, is unlike every previous technology revolution in that it has no tangible form most people associate with it directly. The mobile phone was something you could hold. AI's experiential invisibility is both the source of public distrust and the condition that allows it to embed everywhere without friction. There are four more tensions in this conversation. Listen to find them. Chapters 00:00 The Future With AI: Opportunities and Risks 08:43 SMEs and AI Adoption 12:49 Creativity in the Era of AI 17:07 Societal Readiness for AI: Trust and Responsibility 30:59 Business and Societal Readiness: A Unified Approach 32:17 Assessing AI Maturity 36:17 The Rise of the Agentic Organisation 44:02 Accountability in AI Deployment 55:11 The Impact of AI on Jobs and Skills

Death by a Thousand AI Licences and other lessons from Season 2

Two seasons in, many lessons to learn. Season 3 information at the end of this post. Every organisation investing in AI is focused on the Tech Stack: infrastructure, platforms, applications. That stack matters. But there is a second stack that barely features in strategy discussions or boardroom agendas, and Professor Ashley Braganza believes it is where the real value from AI adoption is either created or squandered. The Organisation Stack has four layers: Strategy, Governance and Risk, Leadership and Transformation, and Culture and Behaviours. Each layer raises a distinct set of questions that most organisations have not yet answered seriously. In this end-of-season reflection, Ashley draws together the central lessons from Season 2. The conversations across twenty five episodes consistently pointed to the same gap: organisations are investing heavily in the technology stack and insufficiently in the organisation stack. It’s no wonder organisations are facing difficulty to extract value and ROI from AI investment. Distributing AI licences to an entire workforce is not a strategy. Layering AI on top of processes that are not fit for purpose does not fix those processes. And allowing employees to optimise their individual tasks in isolation generates what Ashley calls inefficiencies by a thousand AI licences, a pattern that sub-optimises end-to-end organisational performance even as individual productivity appears to rise. The episode closes with a direct challenge on the agentic organisation. Coordinating AI agents across departmental and silo boundaries is the next critical frontier. Organisations that can design, deploy, and operationalise agents in a genuinely coordinated way will gain a durable advantage. Those that cannot face the same coordination problem in AI that they have always faced in organisational transformation. Key themes: - The Organisation Stack: strategy, governance, leadership, transformation, culture - Death by a thousand AI licences - Agentic organisations and coordinated AI deployment - The limits of individual-level AI adoption - AI adoption maturity and what genuine strategic adoption looks like Chapters 00:00 Season 2 Reflection and Insights 03:06 The Importance of the AI Stack 06:03 Understanding the Organisational Stack 08:53 The Challenge of Agentic Organisations Season 3 launches 4 June, opening with an in-person and live-streamed panel featuring AI adoption alums from 12:00 to 13:00 UK time. Please join a live audience at Brunel’s Eastern Gateway Auditorium by registering here https://www.eventbrite.co.uk/e/brunel-research-showcase-tickets-1985263093354. The discussion will be live-streamed and the link is here https://vimeo.com/event/5944341. I hope you will join.

Your AI Use Cases and Proofs of Concept Are Not Evidence of Progress

Most banks now have impressive-looking AI. Very few have AI that genuinely transforms their customers experience. The gap between a polished interface and a transformed organisation is wider than most leaders want to admit, and it comes down to foundations that were never built properly in the first place. Moush Verma, Senior AI Lead at Santander UK, makes the case that the biggest mistake organisations are currently making is scaling AI use cases before they have scaled the underlying capability. In this conversation, Moush argues that value in banking is shifting from transaction efficiency to contextual guidance: reaching customers at the precise moment they need support, before they have to ask for it. She sets out the structural reasons why most legacy institutions struggle to achieve this, pointing to fragmented data ownership, siloed operating models, and an industry-wide habit of measuring AI maturity by volume of proof of concepts use cases. Moush argues that banks should measure the value of AI initiatives in terms of actual delivery and value to customers and the business. The conversation also covers the philosophical dimension of AI adoption that too many organisations skip entirely: not just whether AI can be deployed, but what it is genuinely for. Moush is clear that the winners in two to three years will be organisations that embed AI coherently across customer journeys, risk, operations, and service, treating it not as a set of separate initiatives but as the invisible intelligence running through everything. Highlights from this episode: • Volume of proof of concepts is not maturity. • The payment failure example: a concrete case of how AI can shift a bank from reactive to proactive customer support. • Life moments and everyday moments require different AI responses. A good AI engine knows the difference. • A polished AI interface overlaid on weak data foundations will never produce genuine transformation. • The philosophical question every organisation must answer before scaling: is AI there to industrialise processes, or to create clearer, more trusted guidance for customers? If your organisation is accumulating AI pilots without seeing sustained transformation, this episode gives you the framework to understand why, and what to do about it. This episode marks the 50th episode of The AI Adoption Podcast and brings Season 2 to a close. Thank you to every listener who has been part of this journey. Season 3 coming soon! Chapters 00:00 Transforming Legacy Processes with AI 02:14 AI's Role in Value Creation in Banking 05:40 Understanding Timing in Customer Interactions 07:16 Designing AI-Based Customer Interactions 08:56 Organisational Change for AI Transformation 10:09 Building Strong Data Foundations 11:44 Creating a Unified Purpose in Organisations 15:04 Hype vs. Reality in AI Adoption 17:23 Measuring AI Implementation Success 19:02 Scaling AI Capabilities in Organisations 21:41 Transformational Outcomes in Banking 23:59 Capturing Customer Moments 25:23 Future of AI in Banking

Agentic AI Will Always Need Constant Re-Engineering: The Future of Business Transformation

AI systems today cannot learn from a single conversation. Turn one off, wipe its context, and it will remember nothing. Every interaction has to start again from scratch. For organisations investing heavily in AI, this is not a minor inconvenience. It is a structural limitation that shapes everything from deployment strategy to governance. Babak Hodjat, Chief AI Officer at Cognizant, makes the case that while the potential of AI is genuinely substantial, the hype has consistently exceeded both the reality and even the potential. He suggests that most organisations are still only scratching the surface of what meaningful adoption could look like. Hodjat draws on decades of AI research to explain the fundamental gap between human and machine intelligence. He argues that agentic AI, systems that can take actions in the world rather than simply generate text, is the most promising near-term direction. And that introduces its own demands. These systems require constant human re-engineering. They do not resolve the core limitation: an AI agent cannot change its way of thinking as its environment changes, as a human employee would. Hodjat addresses the concentration of AI capability in a small number of commercial companies, the growing risk of catastrophic reasoning failure in long AI reasoning chains, and the pace of job disruption, which he argues is unfolding in months rather than the decades society had to adjust to previous technological shifts. Highlights • Cognizant's internal multi-agent system has handled more than 11 million employee commands and demonstrably reduced support ticket volumes across the business. • A single large language model begins to fail catastrophically at around 300 to 400 reasoning steps. Cognizant's research has achieved one million error-free reasoning steps by having agents check and vote on each other's work. • Open source AI models are roughly six to nine months behind commercial models, and the gap is closing. This matters for any organisation concerned about the concentration of AI power. • Babak distinguishes between human in the loop (every AI decision requires human approval) and human on the loop (the AI decides when to surface a decision to a human). For high-frequency decisions, the former is simply not practical. • The disruption to jobs from AI is unfolding in months. Previous technological disruptions took decades. Society, political leaders, policy makers and people do not have the same adjustment time this time. If you are responsible for AI strategy in your organisation, this episode sets out the engineering realities that will determine whether your investment delivers lasting value or requires constant remediation. Chapters Chapters 00:00 The Agentic Fabric of Business Processes 03:09 AI Hype vs. Reality 06:30 Limitations of Current AI Technologies 08:51 Addressing Limitations with Agentic AI 11:15 Use Cases of AI in Organisations 17:14 AI in Financial Services and Supply Chains 22:14 Concentration of Power in AI 25:16 Impact of AI on Jobs 27:23 Research Directions in AI 30:34 Future Predictions for AI and AGI

Democracy, Bias, and the Case for Ethical AI: Data has history

Organisations selling AI-powered election-winning services are operating openly. Deep fakes can place words in the mouths of public figures before any rebuttal reaches its audience. An estimated 95% of content on social media carries some form of AI manipulation. These are not hypothetical futures; they are the present. Dawn Butler, Member of Parliament for London’s Brent East constituency and member of the Speaker's AI Commission, makes the case that the greatest risk is not the technology itself: it’s the historical bias embedded in the data that trains it. Dawn argues that every data set carries a history, and that history in policing, health and public life is one of structural inequality. Building AI on top of that history does not neutralise it; it amplifies and automates it. She also challenges the widely held assumption that regulation stifles innovation, contending instead that an ethical regulatory framework will become a mark of quality that organisations and citizens will actively seek out. The conversation also covers what it means to be human in an age of artificial intelligence, and why teaching children to think critically is one of the most important acts of democratic resistance available to us today. Highlights • Facial recognition systems have an inbuilt bias and still produce misidentifications even at optimised accuracy thresholds. • BMI, used routinely in clinical settings, was derived from measurements of roughly 2,000 white men and was never designed for medical application. • Denmark has legal protections for citizens' intellectual property and voice that the UK does not yet provide. • Dawn argues that companies should be fined in a meaningful way to create real accountability, using the analogy of the seatbelt as a model for safety regulation that does not prevent progress. If you lead an organisation that uses data to make decisions about people, this episode sets out why the provenance of that data is not a technical detail. It is a governance responsibility. Chapters 00:00 The Evolution of Data and Ethics 02:04 AI's Impact on Democracy and Human Rights 05:50 Guarding Against Misinformation and Deep Fakes 12:26 AI in Policing: Risks and Benefits 15:45 The Responsibility of Tech Companies 20:41 The Need for Regulation and Legislation 22:25 Balancing Innovation and Regulation 23:54 The Human Element in AI 29:47 The Role of the Speaker's AI Commission