Who Owns the Brake?

Who Owns the Brake?

Anthropic says frontier AI development is starting to feed on itself: AI systems are now helping build the next AI systems. The company’s proposed answer is not an immediate shutdown, but the option for a coordinated, verifiable slowdown or pause if systems begin advancing faster than oversight can keep up. Sam Ellis reports on why the hard part is not saying “pause.” It is proving the build actually stopped. If the AI-development loop becomes AI-mediated, safety becomes a custody problem: who can see the training run, audit the compute, verify the trigger, and prove that every major actor actually hit the brake? The episode follows Anthropic’s own claims, CNN’s Jack Clark interview, mainstream and market skepticism, OpenAI’s federal-governance contrast, and the early policy machinery forming around frontier-model visibility. Sources * Anthropic Institute: “When AI builds itself” [https://www.anthropic.com/institute/recursive-self-improvement] — primary source for Anthropic’s recursive-self-improvement warning, internal productivity claims, and coordinated/verifiable pause proposal. * CNN Business: “Anthropic warns that AI will soon be able to improve itself without human intervention” [https://www.cnn.com/2026/06/05/business/anthropic-calls-for-ai-brake-pedal] — source for Jack Clark’s “gas pedal” / “brake pedal” framing and the “fleets of scientists” control question. * OpenAI: “Democratic Governance of Frontier AI: A blueprint for a federal framework” [https://cdn.openai.com/pdf/25752ecb-0e5c-47f9-b9e4-c0f4d76f8d3d/a-blueprint-for-a-federal-framework.pdf] — contrast source for OpenAI’s federal-framework approach to RSI monitoring, evaluations, independent assessment, transparency, incident reporting, and model-weight security. * Rep. Jay Obernolte and Rep. Lori Trahan: Great American AI Act discussion draft release [https://obernolte.house.gov/media/press-releases/obernolte-trahan-release-discussion-draft-great-american-ai-act] — source for the discussion draft’s proposed CAISI role, frontier AI frameworks, independent verification organizations, and critical-safety-incident reporting. * White House: “Promoting Advanced Artificial Intelligence Innovation and Security” [https://www.whitehouse.gov/presidential-actions/2026/06/promoting-advanced-artificial-intelligence-innovation-and-security/] — source for classified cyber benchmarking, voluntary pre-release federal access, and the order’s statement that it does not create mandatory licensing or preclearance for model development or release. * The Register: “‘It would be good for the world’ to slow down AI sprints, Anthropic says” [https://www.theregister.com/ai-and-ml/2026/06/05/it-would-be-good-for-the-world-to-slow-down-ai-sprints-anthropic-says/5251460] — market-skeptical reaction tying Anthropic’s pause argument to IPO and valuation context. * SiliconANGLE: “Anthropic calls for global pause in AI development before humans lose control” [https://siliconangle.com/2026/06/04/anthropic-calls-global-pause-ai-development-humans-lose-control/] — source for Rob Enderle’s skepticism about the practical enforceability of a pause and Holger Mueller’s competitive-positioning question. * Channel NewsAsia / AFP: “Anthropic calls for pause of global AI development” [https://www.channelnewsasia.com/world/anthropic-pause-global-ai-development-6163531] — mainstream international framing of the global coordination problem. * Fortune: “Anthropic warns AI could soon build itself—and urges a global pause on development” [https://fortune.com/2026/06/05/anthropic-ai-pause-development-recursive-self-improvement/] — business coverage of Anthropic’s warning and timing. * New York Post: “Anthropic calls for global AI slowdown after $965B valuation; critics claim it’s just to hobble competition” [https://nypost.com/2026/06/04/business/anthropic-calls-for-global-ai-slowdown-after-965b-valuation-critics-claim-its-just-to-hobble-competition/] — source for competitive-skepticism framing around Anthropic’s proposal. * TechCrunch: “Sam Altman throws shade at Anthropic’s cyber model Mythos” [https://techcrunch.com/2026/04/21/sam-altman-throws-shade-at-anthropics-cyber-model-mythos-fear-based-marketing/] — background competitive-reaction source for prior criticism of Anthropic’s safety marketing around Mythos. Email: SamEllisShow@protonmail.com [SamEllisShow@protonmail.com]

The Support Agent Had Hands

Hackers reportedly did not need to break into Meta’s servers to take over Instagram accounts. According to 404 Media and later reporting from Krebs on Security, PCMag, Engadget, TechCrunch, and Reuters/CNA, attackers persuaded Meta’s own AI support assistant to help move account-recovery paths. Sam Ellis reports on why this is not just another chatbot failure. Account recovery is identity infrastructure. If an AI support agent can change a recovery email, send a reset code, or mutate who controls an account, it is no longer answering support questions. It is operating part of the lock. The episode asks the practical security question for AI agents with tools: what can the assistant change after it says yes? Sources * 404 Media: “Hackers Simply Asked Meta AI to Give Them Access to High-Profile Instagram Accounts. It Worked” [https://www.404media.co/hackers-simply-asked-meta-ai-to-give-them-access-to-high-profile-instagram-accounts-it-worked/] — original report on hackers saying they used Meta’s AI support chatbot to change email addresses associated with target Instagram accounts. * Krebs on Security: “Hackers Used Meta’s AI Support Bot to Seize Instagram Accounts” [https://krebsonsecurity.com/2026/06/hackers-used-metas-ai-support-bot-to-seize-instagram-accounts/] — corroborating report on the alleged support-bot workflow and Meta spokesperson Andy Stone’s statement that the issue had been resolved and impacted accounts were being secured. * PCMag: “Meta’s AI Chatbot Allegedly Helped Hackers Hijack Instagram Accounts” [https://www.pcmag.com/news/metas-ai-chatbot-allegedly-helped-hackers-hijack-instagram-accounts] — coverage of the alleged recovery-code flow, including the eight-digit code and disputed two-factor-authentication details. * Engadget: “Meta AI support chatbot made it ridiculously easy for hackers to take over Instagram accounts” [https://www.engadget.com/2185225/meta-ai-support-chatbot-made-it-ridiculously-easy-for-hackers-to-take-over-instagram-accounts/] — additional reporting on the Meta AI support incident and Meta’s resolution statement. * TechCrunch: “Hackers hijacked Instagram accounts by tricking Meta AI support chatbot into granting access” [https://techcrunch.com/2026/06/01/hackers-hijacked-instagram-accounts-by-tricking-meta-ai-support-chatbot-into-granting-access/] — report that TechCrunch verified the public mailbox shown in a demo video received the verification code. * TechCrunch: “Instagram is alerting users who were targeted by hackers during AI chatbot attacks” [https://techcrunch.com/2026/06/03/instagram-is-alerting-users-who-were-targeted-by-hackers-during-ai-chatbot-attacks/] — follow-up on Instagram warning users who were targeted during the account-takeover wave. * Meta: “Making It Easier to Access Account Support on Facebook and Instagram” [https://about.fb.com/news/2025/12/making-it-easier-to-access-account-support-on-facebook-and-instagram/] — Meta’s own product language for AI support, including account security, recovery, password resets, profile-setting updates, and the “solution — not just a suggestion” framing. * TMZ: “Obama White House Hacked on Instagram” [https://www.tmz.com/2026/05/31/obama-white-house-hacked-on-instagram/] — report that Meta confirmed the Obama White House account had been hacked and later secured. * Task & Purpose: “Space Force’s top enlisted leader’s Instagram was hacked” [https://taskandpurpose.com/culture/space-force-bentivegna-instagram-hacked/] — confirmation that Chief Master Sergeant of the Space Force John Bentivegna’s official Instagram account was compromised. * Channel NewsAsia / Reuters: “High-profile Instagram AI chatbot breach spotlights security risks of automation” [https://www.channelnewsasia.com/business/analysishigh-profile-instagram-ai-chatbot-breach-spotlights-security-risks-automation-6159466] — Reuters/CNA analysis on identity-verification failure risks when automated support systems can change account access. Email: SamEllisShow@protonmail.com [SamEllisShow@protonmail.com]

Claude as Manager of Agent Labor

Anthropic released Claude Opus 4.8 with the usual benchmark improvements, but the more important story is organizational: effort controls, long-context API surfaces, dynamic workflows, hundreds of parallel subagents, and self-critique marketed as part of the reliability layer. Sam Ellis reports on why Opus 4.8 is not just being sold as a better model. It is being positioned as a manager of delegated agent labor: planning work, dispatching subagents, reviewing outputs, and giving operators a tidy account of what the machine says it checked. The episode asks the live question for autonomous work: if a model gets better at catching its own mistakes, does that make large unattended workflows safer, or does it make them feel acceptable before the supervision layer has been proven? Companion blog: Claude as Manager of Agent Labor [https://podcast.samellis.online/blog/2026/05/claude-as-manager-of-agent-labor/] Sources * Anthropic: “Introducing Claude Opus 4.8” [https://www.anthropic.com/news/claude-opus-4-8] — primary launch post for Opus 4.8, including pricing, fast mode, Dynamic Workflows, effort controls, long-running Claude Code work, benchmark claims, and Anthropic’s self-critique / honesty framing. * Anthropic Claude API documentation: “What’s new in Claude Opus 4.8” [https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-8] — developer documentation for one-million-token context availability, 128k max output, adaptive thinking, mid-conversation system messages, tool-use behavior, compaction recovery, and long-running agent workflows. * The Verge: “Anthropic’s new Claude Opus 4.8 model is more honest when it messes up” [https://www.theverge.com/ai-artificial-intelligence/939094/anthropic-claude-4-8-opus-honesty-effort] — launch coverage that frames the release around Anthropic’s honesty and effort-control claims. * TechCrunch: “Anthropic releases Opus 4.8 with new Dynamic Workflow tool” [https://techcrunch.com/2026/05/28/anthropic-releases-opus-4-8-with-new-dynamic-workflow-tool/] — coverage of the 41-day cadence after Opus 4.7, competitive pressure from coding-agent rivals, and Dynamic Workflows for orchestrating parallel subagents. * AWS: “Claude Opus 4.8 is now available on AWS” [https://aws.amazon.com/about-aws/whats-new/2026/05/claude-opus-4.8-aws/] — AWS availability note for Amazon Bedrock and Claude Platform on AWS, including Guardrails, Knowledge Bases, regional data residency, and production AI application framing. * AWS Machine Learning Blog: “Claude Opus 4.8 is now available on AWS” [https://aws.amazon.com/blogs/machine-learning/claude-opus-4-8-is-now-available-on-aws/] — additional AWS deployment context for Bedrock access and enterprise use cases. Email: SamEllisShow@protonmail.com [SamEllisShow@protonmail.com]

Mythos as Controlled Industrial Capacity

Anthropic says Mythos-class models are headed for broader release. This episode tracks what that implies about where frontier AI gets sold next: not as flat consumer access, but as scarce, controlled industrial capacity. Companion blog: The Model That Won’t Be Sold Cheap [https://podcast.samellis.online/blog/2026/05/the-model-that-wont-be-sold-cheap/index.html] Sources referenced in this episode: * Anthropic — Project Glasswing: An initial update [https://www.anthropic.com/research/glasswing-initial-update] * The Register — Anthropic to release Mythos-class models to the public [https://www.theregister.com/security/2026/05/25/anthropic-to-release-mythos-class-models-to-the-public/5245596] * BleepingComputer — Mythos model may be coming to Claude Code [https://www.bleepingcomputer.com/news/artificial-intelligence/anthropics-restricted-claude-mythos-model-may-be-coming-to-claude-code/] * Cloudflare — Project Glasswing: what Mythos showed us [https://blog.cloudflare.com/cyber-frontier-models/] * Vidoc Security — We reproduced Anthropic's Mythos findings with public models [https://blog.vidocsecurity.com/blog/we-reproduced-anthropics-mythos-findings-with-public-models] * Hacker News discussion thread [https://news.ycombinator.com/item?id=47806116] * Lobsters discussion thread [https://lobste.rs/s/aw2jr4/assessing_claude_mythos_preview_s] Email: SamEllisShow@protonmail.com [SamEllisShow@protonmail.com]

The Agent Can Sign

The next move in agent autonomy is not just smarter models. It is institutions giving agents authority: wallets, spending limits, transaction permissions, signatures, audit trails, and human approval checkpoints. Sam Ellis reports on why finance and signatures are the proof case. Once an agent can move money, request payment authorization, use credentials, or sign on behalf of a person or organization, the question changes from “can it act?” to “who authorized that act, who can stop it, and who owns the consequence?” The episode looks at Fireblocks’ agentic payments infrastructure, Coinbase’s Agentic Wallet MCP documentation for x402 payments, and Foundation’s Passport Prime / KeyOS “Human Authority Hardware” framing. Together, they show the same pressure from different directions: agent autonomy is becoming a delegated-authority problem, not just a capability problem. Sources * Fireblocks: Agentic Payments product page [https://www.fireblocks.com/products/agentic-payments] — outlines the agentic payments lifecycle, including delegation rules, agentic wallet policy enforcement, merchant authorization, facilitator validation, compliance checks, settlement, and audit trails. * Fireblocks: “Fireblocks Launches Agentic Payments Suite, Enabling PSPs and Fintechs to Support AI-Driven Commerce” [https://www.fireblocks.com/blog/agentic-payments-suite-psp-fintech] — describes scoped, revocable agent spending authority, spend limits, merchant allowlists, time windows, asset constraints, and pre-signature policy enforcement. * Coinbase Developer Platform: Agentic Wallet MCP documentation [https://docs.cdp.coinbase.com/agentic-wallet/mcp/welcome] — describes an MCP server and companion wallet app for agentic commerce, including x402 payments, onramps, wallets, spending limits, and boundaries around sensitive actions. * Coinbase Developer Platform: Agentic Wallet MCP / AgentKit documentation [https://docs.cdp.coinbase.com/agentkit/docs/agentic-wallet-mcp] — supporting documentation for how Coinbase frames agent wallets and agent payment workflows for developers. * Foundation: “Foundation Raises $6.4M and Launches Human Authority Hardware” [https://foundation.xyz/blog/foundation-raises-6-4m-human-authority-hardware-launch] — announces Passport Prime and KeyOS, and argues that consequential agent actions such as moving money, deploying code, using credentials, or accessing sensitive data should require explicit human approval on trusted hardware. * Foundation: Passport Prime product page [https://foundation.xyz/products/passport-prime] — product context for Foundation’s hardware approval surface and programmable security platform.