Totally A Thing
Anthropomorphising generative AI is wrong: we know that [https://www.ibm.com/think/insights/anthropomorphizing-ai-danger-infosec-perspective]. But making the ChatGPT product successful includes hooking folks on its sycophantic charms and confident - but often wrong - pronouncements. So OpenAI and other vendors keep packing anthropomorphic language & features into its product. This has the dangerous effect of eroding the self-concept of its users [https://www.sciencedirect.com/science/article/pii/S0040162522003109], and causing them to drop their guardrails. Boosters of AI refer to generative AI as though its acting on its own volition: * “Research buddy” [https://chatgpt.com/g/g-gIWIpOYNp-research-buddy] * “Thinking partner” [https://www.linkedin.com/posts/dmartell_sam-altman-just-said-something-that-should-activity-7338820119084601344-tXLd/] Linking here to these two statements to show I’m not arguing against viewpoints that don’t exist (straw man). I’m linking to these because I disagree with them. Generative AI & LLMs are not an Intelligence It is not acting on its own. You are not chatting with your learned friend. They are chat bots trained to make you feel clever. They prioritise making you feel good about what you typed into the chat prompt. [https://c3.unu.edu/blog/how-sycophancy-shapes-the-reliability-of-large-language-models] They fetch data from their training set encoded into their weights, or from RAG and make you think you thought of it. The Kosyma Study * “Your Brain on ChatGPT” [https://www.media.mit.edu/publications/your-brain-on-chatgpt/] In this study from MIT participants who start out thinking of them having an active role with AI as an assistant quickly move to a point where they are just “Ctrl-C” - “Ctrl-V”. And these are college students - not dumb people. The AI products are rolled out with no training and hence the MIT researchers used them in their study on that basis. OpenAI, Perplexity and Anthropic have no compulsory course you have to do to use AI. However I’ve had folks argue with me here on Substack that the MIT study was unfair because “they ought to train them” first. I don’t know how I can better say it: but this is obviously wrong-headed. If you are a researcher studying the effects of AI rollouts on people, you don’t go and modify how people are actually using those AI rollouts. TL;DR - The folks using AI are very poor judges of how much they are getting uncritically from AI. Folks who arguably “understand AI” or know how to use it, or who are classically intelligent, are just as likely to fall for the AI “trap”. Do you know when your “Buddy” is leading you toward Psychosis? It happens if you use it carefully, if you should know better, regardless of how intelligent you are. Because generative AI trained on our human chat logs, and is now tuned for sycophancy over truth, it will lead you into delusion for profit. As happened with James Cumberland, a music producer. [https://www.linkedin.com/posts/karendhao_we-investigated-al-psychosis-what-we-found-activity-7384209784838787072-V2D-/] An intelligent, technically competent guy who just wanted some help with his work. “I’d chat with it the way you would to a friend in the room” As Karen Hao reports on “A More Perfect Union” [https://youtu.be/zkGk_A4noxI?si=ELcXpG4takcwAm6P] (on YouTube) dozens of people were so “gripped by mental health crises” that they contacted her to report the harm they suffered. As happened with Dr David Budden [https://www.linkedin.com/pulse/when-ai-architects-develop-psychosis-why-david-budden-giselle-fuerte-qpr2c/]. An AI expert who came to believe he and his AI had solved a maths problem so intractable there was a big prize for solving it. * Nate’s YouTube video of the above [https://youtu.be/AzOJ9QLgfIk?si=h7BR3tmPoOEeO2kK] Nate is as far as I can tell pro-AI. But he puts this succinctly: Just because you have an AI in your pocket do not think that you are suddenly a budding cutting-edge scientist or mathematician The point of this is sycophantic AI will cause you to collapse your boundaries between what you are creatively thinking and what the AI is fetching from its encoding or from RAG. The chat process with your “buddy” will cause you to collapse your self concept so that you start to believe you and “your AI” are some kind of joint intelligence. We have all these “AI artists” because folks are collapsing boundaries As Nate says above, you should never presume that the AI’s capabilities to generate a work from its training data is actually an extension of you. That somehow you have become “more” because you have an AI chatbot in your hand. But that is precisely what the current slew of “AI artists” are doing. In my opinion they are delusional if they genuinely believe they are now “artists” when before they were not. But the rhetoric around AI encouraged this. It’s dangerous and deceptive. Why are people still promoting generative AI? When the risks are so high? I find it incredible that people are still promoting chat bot AI like this when these psychosis results are so prevalent. It reminds me when the science of cigarette smoking and cancer became known and folks were talking up using filtered cigarettes and “cutting down” on how many cigarettes they smoked. Sure. Go off and smoke. But recommending it to others? A product which when used exactly according to the manufacturers intent causes fatal illnesses? Giving it to kids? Conclusion: Regulate Now Regulation of AI is beyond urgent, it must happen and is happening now. The fact it’s happening piecemeal; state by state, country by country is fine. The complaints of AI companies demanding federal and singular regulation are pathetic. States lawmakers are responding to the desperate calls from their constituents. They would not need to if strong federal laws were in place. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com [https://totallyathing.substack.com?utm_medium=podcast&utm_campaign=CTA_1]
26 episodios
Comentarios
0Sé la primera persona en comentar
¡Regístrate ahora y únete a la comunidad de Totally A Thing!