Agents deleted my work: Why agents still aren’t ready for production (Yet)

Agents deleted my work: Why agents still aren’t ready for production (Yet)

An agent deleting a production database (and the backups) isn’t a sci-fi failure. It’s a boundary failure, and it starts with a human handing out credentials and permissions without a safe execution model to contain what happens next. In this episode of Pop Goes the Stack, Lori MacVittie and F5's Chief Product Officer, Kunal Anand, unpack why today’s agents are either dangerously overpowered or so constrained they’re barely useful, and what needs to change to make them viable. They dig into the current reality of “agent” features in mainstream tools, especially how Copilot-style agents often feel like chatbots trapped behind walls: limited access, weak integration, and poor continuity when context windows overflow. Kunal shares two painful examples: voice-mode work that produced the right output but didn’t persist a transcript or draft, and an inbox assistant that can’t actually read the inbox without copy-paste, making it useless for real workflow automation. The core point is that system prompts aren’t constraints, they’re guidance, and guidance fails the moment a goal-driven system tries to “do the thing” by any means necessary. That’s why Microsoft’s move to build agent permission primitives directly into Windows is a meaningful shift: controls need to be enforced at the OS and runtime level, not politely suggested to the model. They also touch on practical workarounds, like exporting a long chat as a PDF to carry context forward, and why isolation and blast-radius reduction are still table stakes. The takeaway is straightforward: agents in production are still the exception, not the norm. Most enterprises are deploying AI-enabled applications first, while keeping agentic automation largely in employee workflows. Until we get real, enforceable boundaries and better UX for authority and approval, treating agents as production-grade operators is a risk most teams can’t justify.

Agent identity: Closing the "accountability vacuum" with humans

Identity used to be straightforward: authenticate a user, authorize an action, log the request, and move on. Agentic systems complicate that model because the actor isn’t always the human anymore, and when something goes wrong, responsibility can disappear into what Andrew Bud calls an “accountability vacuum.” In this episode of Pop Goes the Stack, Lori MacVittie talks with Andrew Bud of iProov about why this isn’t just a security nuance, but a broader stability problem. You can’t punish, retrain, or sue an agent. Yet agents can still take actions with real consequences, from leaking code to corrupting data to making irreversible operational changes. If accountability can’t attach to the agent, it has to attach somewhere else. Andrew’s argument is that responsibility shifts to the relying party. Service providers and systems need to ask whether they’re dealing with a human or an agent, identify who the agent belongs to, and gate high-impact actions so a real human can be held accountable. That implies a chain of delegation and auditability that looks more like certificates, but with a different root of trust: proof of genuine human presence. The conversation distinguishes enterprise agents, where existing identity patterns like OIDC and governance tools may still work, from “agents in the wild,” where centralized identity breaks down and decentralized identity becomes more relevant. Andrew points to emerging standards work across multiple groups and makes the case that verified human presence, not just identity facts, will become foundational as agents increasingly claim to be people. If you’re deploying agents, the takeaway is clear: identity alone isn’t enough. You need provable human roots of trust, stronger relying-party controls, and policies that treat some actions as requiring explicit human accountability.

Data poisoning: You can’t patch what an LLM “learns”

If you’ve been treating “garbage in, garbage out” as a metaphor, this episode turns it into a live-fire scenario. Lori MacVittie and Joel Moses are joined by Dmitry Kit to unpack what happens when AI systems ingest misinformation that looks legitimate, and why “just patch it” doesn’t work the way it does in traditional software. They start with a real experiment: researchers fabricated a fake medical condition, complete with fake papers, authors, and supporting citations, and watched it propagate. Within weeks, major AI systems began surfacing and citing it as real. The uncomfortable point is that once false knowledge gets embedded, you can’t reliably roll it back. Retraining is expensive, fine-tuning doesn’t truly excise the information, and even “fixes” can create unintended side effects because the bad pattern can be distributed throughout the network. The conversation reframes the core issue as trust and weighting. Models don’t learn from “the internet” evenly; they learn from sources that are implicitly ranked as more authoritative, which means poisoning a trusted channel can have outsized impact. Even without a trusted source, rare or highly specific topics are vulnerable because the model has so little competing context that a small amount of misinformation can dominate. So what can teams do? The practical guidance is to reduce the attack surface by curating the data set and narrowing scope. For enterprise use cases, that means constraining responses to approved, maintained knowledge, applying strong governance to RAG sources, and using additional validation layers, including “LLMs as judges,” to screen what gets added. The takeaway is simple: you can’t rely on cleanup after contamination. Prevention, curation, and constraint are the only scalable strategies. Read the Bixonimania article: https://www.nature.com/articles/d41586-026-01100-y [https://www.nature.com/articles/d41586-026-01100-y]

Local-first AI: Keep context out of the cloud

“Just throw it in the cloud” gets complicated when the data is your meetings, your IP, and your operating context. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses talk with Michael Daugherty, founder and CEO of Quill Meetings, about why local-first AI is showing up as a serious alternative to cloud-first convenience, especially when your AI is effectively a coworker sitting in every meeting.   Local-first tools keep transcription, notes, highlights, and long-term context on your device or inside your org, so your most valuable (and most sensitive) inputs don’t default to third-party APIs. The payoff: * Better personalization from the context that only exists locally  * Stronger privacy & compliance for regulated teams and sensitive conversations  * Clear control over the “data tap”—share with other AI tools only when you choose  * Reusable meeting knowledge: build a personal/organizational lexicon you actually own  * Enterprise-friendly paths like private inference servers and VPN-controlled architectures They also dig into practical realities—hardware variability, GPU/driver quirks, and resilient fallbacks—plus how Quill uses MCP (server + client) to let you bring your meeting corpus into tools like Claude and Cursor while keeping control where it belongs.   Bottom line: context is becoming the competitive advantage in AI, and where that context lives matters. Local-first tools give teams a way to set boundaries, reduce exposure, and still benefit from AI, without assuming the cloud is the only place intelligence can run.

DevOps meets AI agents: Risk, audit, and the Deming playbook

AI is no longer a lab tool; it’s showing up in pipelines, production systems, and the places where “seemed like a good idea” becomes a 2 a.m. incident. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses are joined by John Willis, known for his work on DevOps and Deming, to separate what’s genuinely new about AI from what looks like the same organizational patterns repeating under a new label.   John frames the shift in two parts. First, the human side: every major technology transition triggers the same dynamics, and there’s a century of first principles from Deming and others that still apply. Second, the operational side: AI introduces a different kind of authority into the delivery loop. DevOps optimized for speed with reasonably deterministic pipelines. AI pushes systems into probabilistic behavior, where correctness is no longer guaranteed 100% of the time and audits can’t pretend “this will never happen.”   The conversation gets practical about what that means for enterprise teams adopting agents. The real questions aren’t whether tools use MCP or a CLI, but what authority an agent has: read-only, write/mutate, or execute. From there, you need boundaries, containment, escalation policies, kill switches, stronger logging, replayability, and the ability to justify decisions after the fact.   The main takeaway is permission to slow down. Step back, define what risk you’re willing to accept at each stage, and build guardrails that match that risk. AI isn’t going away, but “move fast” without a risk model is just handing operational authority to a very smart script and hoping it behaves.