Data poisoning: You can’t patch what an LLM “learns”

Is AI making security obsolete?

What happens if AI finally writes secure code by default? In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and Ken Arora take that question seriously, even if it feels like a punchline today. The premise is simple: if AI starts producing mechanically correct, vulnerability-free code at scale, the security industry doesn’t disappear. It gets forced up the stack.   They outline the near-term reality first: AI can already find and fix large classes of common issues in legacy code, and we’re likely heading into a chaotic cleanup phase as tools like Mythos-style systems accelerate remediation. But longer term, the conversation shifts to the uncomfortable tradeoff: code quality may improve faster than complexity shrinks. And complexity is where authority problems, logic flaws, and operational failures thrive.   Ken uses a practical analogy: a “perfectly safe driver” can still follow a GPS off a cliff. Likewise, perfect code can implement flawed or exploitable business logic flawlessly. Ticket scalping, abuse of workflows, and manipulation of incentives aren’t bugs; they’re behavior and design gaps. The bigger risk becomes how systems are orchestrated, who has permission to do what, and how autonomous components interpret intent.   They also flag the adversarial side: attackers won’t stop; they’ll shift from injecting obvious vulnerabilities to subtly influencing specs, workflows, and machine-generated architectures. As specs in plain language become the new “source code,” more people will be able to build powerful systems without the instinct to think like an attacker, widening the logic-attack surface.   The takeaway is a shift in mindset: security becomes less about chasing broken code and more about governing boundaries and monitoring behavior. AI will increasingly do exactly what you ask, which makes the new security imperative painfully clear: be precise about what you ask it to do, and enforce what it’s allowed to do when it tries to go beyond that.

Agents deleted my work: Why agents still aren’t ready for production (Yet)

An agent deleting a production database (and the backups) isn’t a sci-fi failure. It’s a boundary failure, and it starts with a human handing out credentials and permissions without a safe execution model to contain what happens next. In this episode of Pop Goes the Stack, Lori MacVittie and F5's Chief Product Officer, Kunal Anand, unpack why today’s agents are either dangerously overpowered or so constrained they’re barely useful, and what needs to change to make them viable. They dig into the current reality of “agent” features in mainstream tools, especially how Copilot-style agents often feel like chatbots trapped behind walls: limited access, weak integration, and poor continuity when context windows overflow. Kunal shares two painful examples: voice-mode work that produced the right output but didn’t persist a transcript or draft, and an inbox assistant that can’t actually read the inbox without copy-paste, making it useless for real workflow automation. The core point is that system prompts aren’t constraints, they’re guidance, and guidance fails the moment a goal-driven system tries to “do the thing” by any means necessary. That’s why Microsoft’s move to build agent permission primitives directly into Windows is a meaningful shift: controls need to be enforced at the OS and runtime level, not politely suggested to the model. They also touch on practical workarounds, like exporting a long chat as a PDF to carry context forward, and why isolation and blast-radius reduction are still table stakes. The takeaway is straightforward: agents in production are still the exception, not the norm. Most enterprises are deploying AI-enabled applications first, while keeping agentic automation largely in employee workflows. Until we get real, enforceable boundaries and better UX for authority and approval, treating agents as production-grade operators is a risk most teams can’t justify.

Agent identity: Closing the "accountability vacuum" with humans

Identity used to be straightforward: authenticate a user, authorize an action, log the request, and move on. Agentic systems complicate that model because the actor isn’t always the human anymore, and when something goes wrong, responsibility can disappear into what Andrew Bud calls an “accountability vacuum.” In this episode of Pop Goes the Stack, Lori MacVittie talks with Andrew Bud of iProov about why this isn’t just a security nuance, but a broader stability problem. You can’t punish, retrain, or sue an agent. Yet agents can still take actions with real consequences, from leaking code to corrupting data to making irreversible operational changes. If accountability can’t attach to the agent, it has to attach somewhere else. Andrew’s argument is that responsibility shifts to the relying party. Service providers and systems need to ask whether they’re dealing with a human or an agent, identify who the agent belongs to, and gate high-impact actions so a real human can be held accountable. That implies a chain of delegation and auditability that looks more like certificates, but with a different root of trust: proof of genuine human presence. The conversation distinguishes enterprise agents, where existing identity patterns like OIDC and governance tools may still work, from “agents in the wild,” where centralized identity breaks down and decentralized identity becomes more relevant. Andrew points to emerging standards work across multiple groups and makes the case that verified human presence, not just identity facts, will become foundational as agents increasingly claim to be people. If you’re deploying agents, the takeaway is clear: identity alone isn’t enough. You need provable human roots of trust, stronger relying-party controls, and policies that treat some actions as requiring explicit human accountability.

Data poisoning: You can’t patch what an LLM “learns”

If you’ve been treating “garbage in, garbage out” as a metaphor, this episode turns it into a live-fire scenario. Lori MacVittie and Joel Moses are joined by Dmitry Kit to unpack what happens when AI systems ingest misinformation that looks legitimate, and why “just patch it” doesn’t work the way it does in traditional software. They start with a real experiment: researchers fabricated a fake medical condition, complete with fake papers, authors, and supporting citations, and watched it propagate. Within weeks, major AI systems began surfacing and citing it as real. The uncomfortable point is that once false knowledge gets embedded, you can’t reliably roll it back. Retraining is expensive, fine-tuning doesn’t truly excise the information, and even “fixes” can create unintended side effects because the bad pattern can be distributed throughout the network. The conversation reframes the core issue as trust and weighting. Models don’t learn from “the internet” evenly; they learn from sources that are implicitly ranked as more authoritative, which means poisoning a trusted channel can have outsized impact. Even without a trusted source, rare or highly specific topics are vulnerable because the model has so little competing context that a small amount of misinformation can dominate. So what can teams do? The practical guidance is to reduce the attack surface by curating the data set and narrowing scope. For enterprise use cases, that means constraining responses to approved, maintained knowledge, applying strong governance to RAG sources, and using additional validation layers, including “LLMs as judges,” to screen what gets added. The takeaway is simple: you can’t rely on cleanup after contamination. Prevention, curation, and constraint are the only scalable strategies. Read the Bixonimania article: https://www.nature.com/articles/d41586-026-01100-y [https://www.nature.com/articles/d41586-026-01100-y]

Local-first AI: Keep context out of the cloud

“Just throw it in the cloud” gets complicated when the data is your meetings, your IP, and your operating context. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses talk with Michael Daugherty, founder and CEO of Quill Meetings, about why local-first AI is showing up as a serious alternative to cloud-first convenience, especially when your AI is effectively a coworker sitting in every meeting.   Local-first tools keep transcription, notes, highlights, and long-term context on your device or inside your org, so your most valuable (and most sensitive) inputs don’t default to third-party APIs. The payoff: * Better personalization from the context that only exists locally  * Stronger privacy & compliance for regulated teams and sensitive conversations  * Clear control over the “data tap”—share with other AI tools only when you choose  * Reusable meeting knowledge: build a personal/organizational lexicon you actually own  * Enterprise-friendly paths like private inference servers and VPN-controlled architectures They also dig into practical realities—hardware variability, GPU/driver quirks, and resilient fallbacks—plus how Quill uses MCP (server + client) to let you bring your meeting corpus into tools like Claude and Cursor while keeping control where it belongs.   Bottom line: context is becoming the competitive advantage in AI, and where that context lives matters. Local-first tools give teams a way to set boundaries, reduce exposure, and still benefit from AI, without assuming the cloud is the only place intelligence can run.