The Glitchatorio

AI & Mental Health

Could AI address the global mental health crisis at scale? And what are the risks and unknowns that go along with that? These are the questions being investigated by a working group called AIMHI (https://forum.effectivealtruism.org/posts/MrFBezseyfnQd9XmJ/seeking-feedback-an-initiative-on-ai-mental-health-and [https://forum.effectivealtruism.org/posts/MrFBezseyfnQd9XmJ/seeking-feedback-an-initiative-on-ai-mental-health-and]). In this episode, I talk to four members of the group about their field research as well as the mental health chatbot they're developing (https://stillwater.coach/ [https://stillwater.coach/]), whose focus is on serving populations with severe mental healthcare shortages (https://impartial-priorities.org/p/ai-mental-health-chatbots-for-low [https://impartial-priorities.org/p/ai-mental-health-chatbots-for-low]). * Find out more about Effective Mental Health: https://effectivementalhealth.com * Join one of AIMHI's weekly coworking sessions: https://luma.com/calendar/cal-JNJlcdItDuFEFcn [https://luma.com/calendar/cal-JNJlcdItDuFEFcn] * Read about the project's theory of change:  https://impartial-priorities.org/p/breaking-the-cycle-of-trauma-and

2 AIs Take A Session (Fiction)

What if AIs went to therapy? Would it help them to become "fitter, happier, more productive" ? (in the words of the old Radiohead song) Or would they take it as a novel type of evaluation (and maybe that's what it really is)? Note: this episode was written and recorded in November 2025, five months before the release of the Mythos Preview system card that mentions Claude's session with a human psychiatrist in the "Model welfare" section. So as weird as this episode might seem, the truth is actually stranger. https://www-cdn.anthropic.com/8b8380204f74670be75e81c820ca8dda846ab289.pdf  (See page 180 for the psychiatrist's report)

You Be The Judge

Can we trust AI to keep AI honest? Having a human in the loop is already more illusion than reality, as the task of checking and overseeing LLM outputs is increasingly assigned to other LLMs. The problem is that these LLM judges tend to be biased in favor of the answers they generate themselves — even when the answers are wrong. To understand why this is, and what we can do about it, listen to my conversation with AI safety researcher Taslim Mahbub. We'll talk about his research into self-preference bias, the surprising results of his experiments and some potential mitigation strategies, as outlined in this post on mitigating collusive self-preference: https://www.lesswrong.com/posts/nB7kAf8c4tvnvZ4u3/mitigating-collusive-self-preference-by-redaction-and-2 and this paper on mitigating self-preference through authorship obfuscation: https://arxiv.org/abs/2512.05379 As a bonus, if you're interested in Taslim's earlier research on using machine learning in service of biodiversity monitoring, here's the abstract of his paper on convolutional neural networks (CNN) for identifying bat species: https://ieeexplore.ieee.org/document/9311084

The Scratchpad Monologues (CoT part 2)

If chain of thought is a model "thinking aloud" to itself, then why does it express doubt, frustration or suspicion about the problems it's solving, sometimes for pages and pages of its scratchpad? And what does chain of thought mean for AI safety? We'll hear from Julian Schulz, a researcher who's studying encoded reasoning in large language models, about where the opportunities, risks and weirdness lie in chain of thought. Here are some links to his research: * On a model jailbreaking its monitor: https://www.lesswrong.com/posts/szyZi5d4febZZSiq3/monitor-jailbreaking-evading-chain-of-thought-monitoring * A roadmap for safety cases based on CoT: https://arxiv.org/html/2510.19476v1#S1 * His posts on Less Wrong: https://www.lesswrong.com/users/wuschel-schulz Some of the other papers we discussed include: * On the biology of a large language model: https://transformer-circuits.pub/2025/attribution-graphs/biology.html * Monitoring reasoning models for misbehavior and the risks of promoting obfuscation: https://arxiv.org/pdf/2503.11926 * How steganography comes about: https://arxiv.org/pdf/2506.01926 * Assuring agent safety evals by analysing transcripts (with excerpts from weird monologues): https://www.alignmentforum.org/posts/e8nMZewwonifENQYB/assuring-agent-safety-evaluations-by-analysing-transcripts * Stress-testing deliberative misalignment: https://www.apolloresearch.ai/research/stress-testing-deliberative-alignment-for-anti-scheming-training/ * And the "watchers" CoT snippet from the paper above:  https://www.antischeming.ai/snippets#using-non-standard-language

Chain of Thought 101

"Think step by step."  Although a simple technique in itself, the problems that chain-of-thought reasoning (CoT) addresses are complex, ranging from the specific issue of hallucinations to the general lack of explainability of AI (both in terms of understanding how it works as well as fixing things that go wrong). We'll hear from data scientist Afia Ibnath on the basics of CoT, how it can be used to evaluate the faithfulness of LLM responses, and her experiences of using it in a business context. Check out Afia's portfolio on Github: https://afiai14.github.io/ [https://afiai14.github.io/] Here's the Anthropic paper we discussed, which outlines that reasoning models are often unfaithful in their CoT: https://www.anthropic.com/research/reasoning-models-dont-say-think For a concise definition of how faithfulness is calculated, see this article: https://www.ibm.com/docs/en/watsonx/saas?topic=metrics-faithfulness