Detection at Scale

Vjaceslavs Klimovs on Why 40% of Security Work Lacks Threat Models

Vjaceslavs Klimovs [https://www.linkedin.com/in/vklimovs/], Distinguished Engineer at CoreWeave [https://www.coreweave.com/], reflects on building security programs in AI infrastructure companies operating at massive scale. He explores how security observability must be the foundation of any program, how to ensure all security work connects to concrete threat models, and why AI agents will make previously tolerable security gaps completely unacceptable.  Vjaceslavs also discusses CoreWeave's approach to host integrity from firmware to user space, the transition from SOC analysts to detection engineers, and building AI-first detection platforms. He shares insights on where LLMs excel in security operations, from customer questionnaires to forensic analysis, while emphasizing the continued need for deterministic controls in compliance-regulated environments. Topics discussed: * The importance of security observability as the foundation for any security program, even before data is perfectly parsed. * Why 40 to 50 percent of security work across the industry lacks connection to concrete threat models or meaningful risk reduction. * The prioritization framework for detection over prevention in fast-moving environments due to lower organizational friction. * How AI agents will expose previously tolerable security gaps like over-provisioned access, bearer tokens, and lack of source control. * Building an AI-first detection platform with assistance for analysis, detection writing, and forensic investigations. * The transition from traditional SOC analyst tiers to full-stack detection engineering with end-to-end ownership of verticals. * Strategic use of LLMs for customer questionnaires, design doc refinement, and forensic analysis. * Why authentication and authorization systems cannot rely on autonomous AI decision-making in compliance-regulated environments requiring strong accountability.

GreenSky's Ken Bowles on Auditing Controls before They Silently Fail

Over his 15-year journey through healthcare and financial services security, Ken Bowles [https://www.linkedin.com/in/kennethbowles/], now Director of Security Operations at GreenSky [https://www.greensky.com/], has collected a plethora of practical strategies for prioritizing crown jewels, managing cloud over-permissions, and building SOCs that scale effectively. He reflects on transforming security operations through AI and intelligent automation and discusses how AI is reducing analyst investigation time dramatically. Ken also asserts the importance of auditing security controls before they silently fail. The conversation touches on the evolving role of the MITRE framework, the concept of signaling versus alerting, and why embracing AI might be the best career move for security professionals navigating rapid technological change in cloud environments. Topics discussed: * Building security operations programs around crown jewels and scaling outward to manage the most critical assets first. * Managing over-permissions in cloud environments that have snowballed across multiple administrators without proper governance. * Using AI to reduce analyst investigation time from 30 minutes to seconds through intelligent data enrichment and context. * Creating true single-pane-of-glass visibility by connecting security tools and data sources for more effective threat detection. * Training new security analysts with AI assistance to bridge knowledge gaps in SQL, SOAR platforms, and log analysis. * Documenting institutional knowledge while encouraging analysts to trust their intuition when something doesn't look right. * Understanding the limitations of impossible travel alerts and using AI to establish user behavior baselines for accurate detection. * Applying the MITRE framework as a guideline rather than gospel, adapting detection strategies to specific organizational needs. * Implementing signaling approaches that label security-relevant events without creating alert fatigue for security operations teams. * Auditing security controls regularly to catch configuration drift and ensure protective measures remain effective over time.  Listen to more episodes:  Apple  [https://podcasts.apple.com/us/podcast/detection-at-scale/id1582584270] Spotify  [https://open.spotify.com/show/6xa9t5dty4eH0UXDQXIew9?si=1df5eac89b294b14] YouTube [https://youtube.com/playlist?list=PLjYWlPBgNuD4f-hPjTyq3iPC-nT64ckFr&feature=shared] Website [https://panther.com/resources/podcasts]

FanDuel's Tyler Martin on the Bronze-Silver-Gold Path to Autonomous Security Triage

Tyler Martin [https://www.linkedin.com/in/tylerhmartin/], Senior Director of Enterprise Security Engineering & Operations at FanDuel [https://www.fanduel.com/], reflects on revolutionizing security operations by replacing traditional analyst tiers with security engineers supported by custom AI agents. Tyler shares the architecture behind SAGE, FanDuel's phishing automation system, and explains how his team progressed from human-in-the-loop validation to fully autonomous triage through bronze-silver-gold maturity stages.  The conversation explores practical challenges like context enrichment, implementing user personas connected to IDP and HRIS systems, and choosing between RAG versus CAG models for knowledge augmentation. Tyler also discusses shifts in detection strategy, arguing for leaner detection catalogs with just-in-time, query-based rules over maintaining point-in-time codified detections that no longer address active risks. Topics discussed: * Restructuring security operations teams to include only security engineers while AI agents handle traditional level 1-3 triage work. * Building Security Analysis and Guided Escalation, an AI-powered phishing automation system that reduced manual ticket volume. * Implementing bronze-silver-gold maturity stages for AI triage: manual validation, automated closures with oversight, and full autonomous operations. * Enriching AI agents with organizational context through connections to IDP systems, HRIS platforms, and user behavior analytics. * Creating user personas that encode access patterns, permissions, security groups, and typical behaviors to improve AI decision-making accuracy. * Designing incident response automation that spins up Slack channels, Zoom bridges, recordings, and comprehensive documentation through simple commands. * Eliminating 90% of missing PIR action items through automated documentation capture and stakeholder tagging in Confluence. * Shifting detection strategy from maintaining large MITRE-mapped catalogs to just-in-time query-based rules written by AI agents. * Balancing signal volume and enrichment data against inference costs while avoiding context rot that degrades LLM performance. * Evaluating RAG versus CAG models for knowledge augmentation and exploring multi-agent architectures with supervisory oversight layers.  Listen to more episodes:  Apple  [https://podcasts.apple.com/us/podcast/detection-at-scale/id1582584270] Spotify  [https://open.spotify.com/show/6xa9t5dty4eH0UXDQXIew9?si=1df5eac89b294b14] YouTube [https://youtube.com/playlist?list=PLjYWlPBgNuD4f-hPjTyq3iPC-nT64ckFr&feature=shared] Website [https://panther.com/resources/podcasts]

Live Oak Bank's George Werbacher on AI As SecOps' Single Pane of Glass

George Werbacher [https://www.linkedin.com/in/georgewerbacher/], Head of Security Operations at Live Oak Bank [https://www.liveoak.bank/], reviews the practical realities of implementing AI agents in security operations, sharing his journey from exploring tools like Cursor and Claude Code to building custom agents in-house. He also reflects on the challenges of moving from local development to production-ready systems with proper durability and retry logic. The conversation explores how AI is changing the security analyst role from alert analysis to deeper investigation work, why SOAR platforms face significant disruption, and how MCP servers enable natural language interactions across security tools. George offers pragmatic advice on cutting through AI hype, emphasizing that agents augment rather than replace human expertise while dramatically lowering barriers to automation and query language mastery. Through technical insights and leadership perspective, George illuminates how security teams can embrace AI to improve operational efficiency and mean time to detect without inflating budgets, while maintaining the critical human judgment that effective security demands. Topics discussed: * Understanding AI's role in augmenting security analysts rather than replacing them, shifting roles toward investigation and threat hunting. * Building custom AI agents using Python and exploring frameworks like LangChain to solve specific SecOps use cases. * Managing moving agents from local development to production, including retry logic, failbacks, and durability requirements. * Implementing MCP servers to enable natural language interactions with security tools, eliminating the need to learn multiple query languages. * Navigating AI hype by focusing on solving specific problems and understanding what agents can realistically accomplish. * Predicting SOAR platform disruption as agents take over enrichment, orchestration, and response with simpler automation approaches. * Removing platform barriers by enabling analysts to use natural language rather than mastering specific tools or query languages. * Exploring context management, prompt engineering, and conversation history techniques essential for building effective agentic systems. * Adopting tools like Cursor and Claude Code to empower technical security professionals without deep coding backgrounds.  Listen to more episodes:  Apple  [https://podcasts.apple.com/us/podcast/detection-at-scale/id1582584270] Spotify  [https://open.spotify.com/show/6xa9t5dty4eH0UXDQXIew9?si=1df5eac89b294b14] YouTube [https://youtube.com/playlist?list=PLjYWlPBgNuD4f-hPjTyq3iPC-nT64ckFr&feature=shared] Website [https://panther.com/resources/podcasts]

Ochsner Health's Andrew Casazza on When AI Becomes the Hammer Looking for Nails

Andrew Casazza [https://www.linkedin.com/in/andrew-c-428563232/], AVP of Cyber Security Operations at Ochsner Health [https://www.ochsner.org/], explores how healthcare organizations navigate FDA-approved medical devices running on legacy operating systems, implement AI-powered security tools while maintaining HIPAA compliance, and respond to threats that now move from initial compromise to malicious action in seconds rather than hours.  Andrew gives Jack [https://www.linkedin.com/in/jacknaglieri?miniProfileUrn=urn%3Ali%3Afs_miniProfile%3AACoAAAmvNvcBAgsSm1DUoZEsYc0NYx-V2ri87Mg&lipi=urn%3Ali%3Apage%3Ad_flagship3_search_srp_all%3BQxkNNqOUShCAJmVXVaoeVQ%3D%3D] his insights on building effective security programs in heavily regulated industries, emphasizing the importance of visibility, automation with guardrails, and keeping humans in the loop for critical decisions while leveraging AI to handle the speed and scale of modern threats. Topics discussed: * Unique security challenges in healthcare environments where medical devices run on legacy operating systems that cannot be easily updated. * Strategies for monitoring and securing systems that cannot have traditional security agents installed due to FDA regulations and medical certification requirements. * Leveraging AI and automation in security operations while navigating HIPAA regulations and protecting patient data from external training models. * Implementing human-in-the-loop approaches where AI performs initial analysis and triage while escalating critical decisions to human analysts. * Understanding the privacy and compliance implications of AI tools that may use customer data for model training and improvement. * The dramatic reduction in threat-actor dwell time from hours or days to minutes or seconds. * Building effective SOAR automation playbooks to handle repetitive cases and reduce noise while focusing attention on bigger threats. * Establishing appropriate guardrails for AI-powered security tools to prevent unintended consequences while enabling automated response capabilities. * The importance of being curious and maintaining broad knowledge across multiple domains to become more effective. Listen to more episodes:  Apple  [https://podcasts.apple.com/us/podcast/detection-at-scale/id1582584270] Spotify  [https://open.spotify.com/show/6xa9t5dty4eH0UXDQXIew9?si=1df5eac89b294b14] YouTube [https://youtube.com/playlist?list=PLjYWlPBgNuD4f-hPjTyq3iPC-nT64ckFr&feature=shared] Website [https://panther.com/resources/podcasts]