Mike Vaiana: What is AI Alignment, and Why Should You Care? (Part I)

Mike Vaiana: What is AI Alignment, and Why Should You Care? (Part II)

In this episode, James is joined again by Mike Vaiana, R&D Director at AE Studio, for part two of their conversation on AI alignment. Where part one motivated why alignment matters, this episode goes a layer deeper into what alignment research actually is and how the work gets done day to day. Mike walks through the main branches of the field: mechanistic interpretability, evaluations, and control. He explains why AE deliberately bets on neglected approaches rather than putting all its eggs in the mech interp basket, and why eval awareness, persona drift, and emergent misalignment make this harder than it looks from the outside. James and Mike trace the METR task-completion time horizon doubling curve and what a four-to-seven-month doubling time really implies when extrapolated out a few years. The conversation gets concrete on what already goes wrong with today's models. They cover the Anthropic blackmail evaluation, specification gaming and reward hacking, and the emergent misalignment result where fine-tuning a model on a small amount of bad medical advice produces a broadly evil assistant that recommends Hitler for dinner. They explain why "just turn it off" is not a serious answer once a system has goals, and why instrumental convergence on power and resources falls out of having almost any goal at all. James and Mike then open the hood on how AE actually does alignment research: one-week agile sprints, vectoring meetings to find the highest-risk question, small-scale experiments designed to falsify ideas fast, and scaling curves from 100M up to 5B parameter pre-training runs aimed at convincing frontier labs to test methods at their scale. They also discuss AE's DARPA seedling and the broader thesis behind it: that the bottleneck in alignment is not ML engineers but researchers with good ideas, and that pairing general-purpose ML talent with researchers (including non-traditional ones, like Princeton neuroscientist Michael Graziano) can unlock work that would otherwise never see the light of day. In this episode: * The main branches of alignment research and how they overlap * Why AE prioritizes neglected approaches over well-funded ones * The METR time-horizon doubling curve and what it implies * Persona drift, eval awareness, and why evaluating frontier models is hard * Why RLHF is the canonical example of an alignment technique with capability upside * How AE runs research as one-week agile sprints * The scaling-curve strategy for getting frontier labs to adopt new methods * The DARPA seedling and AE's model for scaling research through ML engineering talent * Three ICML 2026 acceptances, including a spotlight paper Learn more: ae.studio/alignment AE Studio is hiring: ⁠https://www.ae.studio/join-us⁠ [https://www.ae.studio/join-us] LinkedIn: ⁠https://www.linkedin.com/in/james-bowler-84b02a100/⁠ [https://www.linkedin.com/in/james-bowler-84b02a100/] Contact us: alignment@ae.studio

Stijn Servaes: Claude Mythos - Superintelligence or Hype?

In this episode, James is joined by Stijn Servaes, Lead Research Manager at AE Studio, to break down Anthropic's recently released Claude Mythos preview model and separate genuine signal from hype. This is for listeners who want to actually understand what's in the 244-page model card, not just the headlines. Stijn brings a frontier alignment researcher's perspective to one of the more consequential model releases of the year. Claude Mythos is the first frontier model deemed too risky for general public release, instead going only to a small set of vetted partners through Anthropic's Project Glass Wing. The model card contains a striking paradox: Mythos is described as the most aligned model Anthropic has released to date, while also posing the greatest alignment-related risk. James and Stijn work through what's actually new in the model's cyber offense capabilities, walking through specific examples including the OpenBSD SACK bug that sat undetected in code for 27 years, and the FreeBSD exploit where Mythos autonomously engineered a six-packet ROP chain from a single prompt. They explain why the latter represents a genuine qualitative jump rather than just another point on the benchmark curve. The conversation also covers Anthropic's ASL framework, the CB-1 through CB-4 thresholds for biosecurity uplift, and why cyber and bio capabilities are following different trajectories. Stijn explains why progress on alignment doesn't simply reduce risk, drawing on Anthropic's seasoned mountaineering guide analogy: a more capable, better-aligned model gets trusted with more, taken to more dangerous places, and operates with greater scope, which can cancel out the gains from better behavior. In this episode: * What's actually new in Claude Mythos versus what's marketing or hype * The OpenBSD and FreeBSD exploit examples, and why one matters far more than the other * Why the most aligned model can also be the riskiest model * How Project Glass Wing changes the frontier release model * The ASL framework and why Mythos still sits at ASL-3 * Differences between cyber and bio uplift trajectories * What the chain-of-thought contamination findings mean for oversight * What to watch for in the Glass Wing report coming in July Learn more: ae.studio/alignment AE Studio is hiring: ae.studio/join-us LinkedIn: https://www.linkedin.com/in/james-bowler-84b02a100/

Mike Vaiana: What is AI Alignment, and Why Should You Care? (Part I)

In this episode, James is joined by Mike Vaiana, R&D Director at AE Studio, to lay the groundwork for understanding what AI alignment is and why it matters. This is Part I of a two-part conversation aimed at listeners who want to understand the field from the ground up, whether they're curious newcomers or considering a transition into alignment work. Mike opens with an analogy that frames the entire problem: the relationship between humans and chimpanzees. Despite sharing over 95% of our DNA, the intelligence differential means humans control the planet and chimps have no say, not because humans are deliberately malicious, but because we pursue our own goals and chimps sometimes get in the way. The core of AI alignment is making sure that if we build systems significantly more intelligent than humans, those systems are aligned with human interests rather than pursuing their own goals at our expense. They discuss why superintelligence isn't as far-fetched as current chatbots might suggest, why "just tell the model to be aligned" doesn't work, and why frontier labs, despite doing some alignment work, face financial incentives that mean they are not well positioned or on track to solve the problem. Mike explains AE Studio's meta-approach: rather than specializing in a single technique, AE Studio runs a breadth-first search across neglected, high-risk, high-reward research directions that wouldn't get funded elsewhere. In this episode: * What AI alignment is, explained through the human-chimpanzee analogy * Why prompting a model to "be aligned" doesn't solve the problem * Why frontier labs underinvest in alignment relative to capabilities * How AE Studio's breadth-first research approach differs from other alignment orgs * Why pre-training is a neglected frontier for alignment research Learn more: ae.studio/alignment AE Studio is hiring: https://www.ae.studio/join-us [https://www.ae.studio/join-us] LinkedIn: https://www.linkedin.com/in/james-bowler-84b02a100/ [https://www.linkedin.com/in/james-bowler-84b02a100/] Contact us: alignment@ae.studio

Adriana Calejo: AI Alignment for National Security

In this episode, James is joined by Adriana Calejo, AE Studio's Director of Government and Strategic Partnerships, to explore the deep and often overlooked overlap between AI alignment and national security. Adriana shares her career journey from CIA economic analyst to counterterrorism officer to DARPA's Intelligence Community Liaison Officer, and explains why the government's need for trustworthy, secure, and reliable AI intersects with the same challenge the alignment community is working on. They discuss how DARPA and the intelligence community are thinking about AI beyond large language models, including novel architectures, computing on encrypted data, and the unique constraints of deploying AI in classified environments. Adriana breaks down what "trustworthy AI" means in a national security context: unbiased information access, election security, critical infrastructure protection, and economic defense, and why these goals map directly onto AI alignment research. The conversation gets into the practical realities of bridging these two worlds: why independent, technically grounded voices matter in government AI procurement, how adversaries are targeting the US emerging technology sector, and why alignment researchers who want to influence real-world outcomes need to engage productively with people they may disagree with. In this episode: * Why AI alignment and national security share the same core goals * How DARPA and the intelligence community are approaching AI beyond LLMs * The challenges of deploying AI in classified environments * Why independent technical voices matter in government AI decisions Learn more: ae.studio/alignment AE Studio is hiring: https://www.ae.studio/join-us Linkedin: https://www.linkedin.com/in/james-bowler-84b02a100/

Keenan Pepper: Self-Interpretation in LLMs

In this episode, AE Studio Research Director Mike Vaiana is joined by Research Scientist Keenan Pepper, to explore a new approach to model self-interpretation - teaching language models to explain their own internal activations. They dive into Keenan’s recent paper on training lightweight adapters that transform activation vectors into soft tokens the model can interpret as language. The conversation walks through how this method improves on prior approaches like SelfIE, and how simple affine transformations can unlock surprisingly strong interpretability. Mike and Keenan break down concrete examples, including how models can identify latent topics like “baseball” from internal states, and even surface hidden reasoning steps in multi-hop questions, offering a potential path toward detecting when models are reasoning, guessing, or even hiding information. They also explore broader implications for AI alignment: from probing deception and internal representations, to enabling new forms of activation steering and self-monitoring. Along the way, they discuss attention schema theory, limitations of current labeling methods, and how this work could evolve into a general interface between model internals and human-understandable concepts. In this episode: * What self-interpretation of activations is * How lightweight adapters improve interpretability without retraining models * Why this approach could help uncover hidden reasoning and deception in LLMs Learn more: ae.studio/alignment [https://ae.studio/alignment] Keenan's Research Paper: https://arxiv.org/abs/2602.10352 [https://arxiv.org/abs/2602.10352] AE Studio is hiring: https://www.ae.studio/join-us [https://www.ae.studio/join-us] LinkedIn: https://www.linkedin.com/in/james-bowler-84b02a100/ [https://www.linkedin.com/in/james-bowler-84b02a100/]