AI Is a Weapon You Might Be Pointing at Yourself | OWASP Top 10 LLMs

AI Is a Weapon You Might Be Pointing at Yourself | OWASP Top 10 LLMs

A lawyer submitted six court cases to a federal judge in New York. ChatGPT wrote every single one of them. None of them existed. When opposing counsel said they couldn't find the cases, the lawyer went back to ChatGPT to verify whether the cases were real. ChatGPT said yes. Absolutely. You can find them on Westlaw and LexisNexis. He submitted them anyway — under oath. That's hallucination. That's number nine on the OWASP Top 10 for LLM Applications. And it cost him $5,000, a formal apology to every federal judge whose name appeared in the fake rulings, and probably a lot more in embarrassment. This week on Security Bros, Rocky and John Giglio go deep on the OWASP Top 10 for LLM Applications — the 2025 edition, built by 600+ researchers across 18 countries. If you're building with AI, deploying AI, or just using it every day at work, this list is the closest thing the security world has to a peer-reviewed warning label. They break down all 10 vulnerabilities in plain English, connect each one to real stories, and don't sugarcoat any of it: * A world-famous white hat hacker who jailbreaks ChatGPT to write his own attack tools * Samsung engineers who handed proprietary source code to ChatGPT — and how long it took after the ban was lifted for it to happen again (spoiler: 20 days, three incidents) * Air Canada's chatbot that gave a grieving customer wrong information about bereavement fares — and the company's legal defense that the chatbot was "a separate legal entity" * How DeepSeek may have reverse-engineered Claude's reasoning by querying it at scale — and what Anthropic is doing about it * The invisible text on a webpage that hijacks your AI agent without you ever knowing The lesson running through all of it: your security policy will never beat convenience without technical controls. You have to make the secure path the easy path. Subscribe so you don't miss the follow-up deep dives on Claude Cowork security, AI-ready DLP, and the excessive agency problem that's about to blow up as agentic AI goes mainstream. ---------------------------------------- Resources mentioned: * OWASP Top 10 for LLM Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/ [https://owasp.org/www-project-top-10-for-large-language-model-applications/] * Previous episode: OWASP Top 10 for Web Applications https://youtu.be/oCuYgphY6iY [https://youtu.be/oCuYgphY6iY] 00:00 The Lawyer Who Asked AI If AI Was Lying to Him 00:33 Meet the Security Bros + What We're Covering Today 01:41 What Is OWASP? (600 Researchers, 18 Countries, One List) 04:38 #1 Prompt Injection — The Attack That's Everywhere Right Now 07:28 #2 Sensitive Information Disclosure — You Think It's Private. It's Not. 08:29 #3 Supply Chain Risk — What's Really Inside That Open Source Model? 10:27 #4 Data & Model Poisoning — The Sleeper Agent Attack 13:31 #5 Output Handling — Nobody Reviews AI Code. Nobody. 14:05 #6 Excessive Agency — When Your AI Has Too Much Power 18:12 #7 System Prompt Leakage — Stop Putting Secrets in the Instructions 20:37 #8 Vector & Embedding Weaknesses — How RAG Gets Poisoned 23:30 #9 Hallucination — AI Makes Things Up. Confidently. 25:58 #10 Unbounded Consumption — How DeepSeek May Have Stolen Claude's Brain 29:59 Real Story: Samsung's 3 Data Leaks in 20 Days 36:03 Real Story: Air Canada's "Separate Legal Entity" Defense 40:30 Real Story: The $5K Fine & Apology Letters to Federal Judges 45:09 Key Takeaways — Make the Secure Path the Easy Path

Why the OWASP Top 10 Moved (And What It Says About Your Security Practice)

In this conversation, John and Rocky Giglio discuss the recent updates to the OWASP Top 10 list for 2025, exploring the implications of these changes for application security. They delve into the data behind the rankings, the influence of community feedback, and the importance of secure design practices. The discussion highlights the ongoing challenges in cybersecurity, particularly around misconfiguration and identity management, and emphasizes the need for a holistic approach to security that integrates both software and infrastructure considerations. Chapters   00:00 Introduction and Technical Challenges 02:00 Exploring OWASP Top 10 Updates 07:01 Understanding OWASP and Its Data Sources 14:02 Community Influence on OWASP Rankings 17:07 Movement in OWASP Top 10: Insights and Implications 20:30 The Challenge of Keeping Up with Technology 21:37 The State of Vulnerability Management 22:44 Cloud Native vs. Traditional Organizations 24:11 Understanding the OWASP Top 10 26:14 Trends in Identification and Authentication 27:12 The Importance of Security Logging and Monitoring 28:55 Balancing Application and Infrastructure Security 30:19 The Role of Secure Design in Security 32:02 The Future of Security Practices 34:38 Understanding Weaknesses vs. Vulnerabilities 36:58 The Importance of Cloud Security Practices 39:45 Shifting Left in Security Practices 41:46 The Need for Continuous Assessment

You have too many identities

In this episode of Security Bros, hosts Rocky and John Giglio delve into the complexities of identity management in the cybersecurity landscape, particularly in the cloud era. They discuss the challenges of identity sprawl, the importance of managing permissions, and the balance between security and business needs. The conversation emphasizes the necessity of building relationships across departments to effectively manage identity and security practices. Chapters 00:00 Introduction to Cybersecurity and Identity Management 01:28 Challenges of Identity Management in the Cloud Era 06:13 The Importance of Permissions and Access Control 11:15 Balancing Security and Business Needs 19:01 Building Relationships for Effective Security Management

CrowdStrike Proved Patch Management is Broken

In this episode the Security Brothers, Rocky and John Giglio delve into the complexities of patch management and vulnerability management in the tech industry. They discuss the ongoing challenges faced by security practitioners, the implications of recent incidents like the CrowdStrike outage, and the evolving role of AI in enhancing security measures. The conversation emphasizes the need for comprehensive testing, strategic planning, and prioritization in managing vulnerabilities, while also exploring the importance of adapting to new technologies and methodologies in cybersecurity. Takeaways Handling old tech and patch management is a significant issue. Vulnerability management is overwhelming but necessary. Prioritization is key in dealing with numerous vulnerabilities. Automated systems can help reduce the burden of patch management. Testing is crucial before rolling out updates. AI can assist in writing tests and improving deployment processes. A comprehensive security strategy includes monitoring and logging. Continuous learning from incidents is essential for improvement. Collaboration with business leaders is vital for effective security management. The landscape of vulnerabilities is constantly evolving, requiring adaptive strategies. Chapters 00:00 Introduction to Security Challenges 02:49 The Importance of Patch Management 06:03 Navigating Vulnerabilities in Modern Tech 08:53 Lessons from the CrowdStrike Incident 11:45 Testing and Deployment Strategies 14:49 The Role of AI in Security Management 17:43 Building a Comprehensive Security Strategy 20:53 Final Thoughts and Future Directions

Security Bros - Episode 2 - North Korea Fakers

In this episode the brothers discuss the alarming tactics used by North Korea in cyber infiltration, particularly through fake job interviews and identity fraud. They emphasize the critical need for robust identity verification processes to combat these threats. The discussion also highlights the growing concern of insider threats within organizations and the necessity of implementing layered security strategies to protect sensitive data. The episode concludes with a reminder of the importance of mastering basic security practices to effectively mitigate risks. Takeaways * North Korea is using fake identities to infiltrate companies. The money collected is used for developing nuclear weapons. * Identity verification is crucial in hiring processes. * Insider threats are often overlooked in security measures. * Ransomware attacks can be a consequence of data theft. * Layered security is essential; no single solution suffices. * Understanding user behavior is key to detecting anomalies. * Regular assessments of security strategies are necessary. * Basic security practices are often neglected. * Investing in security must be balanced with operational needs. Chapters 00:00 Introduction to Cybersecurity Challenges 02:41 North Korea's Infiltration Tactics 05:22 Identity Verification and Its Importance 08:31 Understanding Insider Threats 11:35 Ransomware and Data Protection 14:14 The Need for Multi-layered Security 17:00 Final Thoughts