Justified Posteriors

Avi Goldfarb on Prediction Machines, O-Ring Tasks, and How AI is Reshaping Economics

This week, we’re joined by Avi Goldfarb, one of the leading economists of artificial intelligence and co-author of Prediction Machines [https://www.google.com/search?sca_esv=bc87673d3ad1280f&rlz=1C1GCEA_enUS1209US1209&sxsrf=ANbL-n4AnrHPqrHiXM4Cb3oXCBXAennzbw:1777914708243&q=Prediction+Machines:+The+Simple+Economics+of+Artificial+Intelligence&stick=H4sIAAAAAAAAAONgFuLVT9c3NEwzqCw0q8wrU4Jw003S0pMLsnK1pLKTrfST8vOz9RNLSzLyi6xA7GKF_LycykWsLgFFqSmZySWZ-XkKvonJGZl5qcVWCiEZqQrBmbkFOakKrsn5efm5mclADWkKjkUlmWmZyZmJOQqeeSWpOTmZ6al5yakAebQ6E4MAAAA&sa=X&ved=2ahUKEwjFtIC1kKCUAxWiJkQIHRQiDEoQ9OUBegQIDRAD&biw=2183&bih=1080&dpr=1.75]. Avi has been thinking seriously about AI economics long before the ChatGPT shock, so we asked him what he thinks the earlier framework got right, what it missed, and how economists should update their beliefs now. The conversation starts with Avi’s seminal book, Prediction Machines, and the idea that AI is best understood as a drop in the cost of prediction, which is a complement to judgement. We ask what that book got right and what it got wrong. From there, we interrogate Avi on the murky boundary between prediction and judgment. We had investigated the idea that maybe judgment and prediction were not as separable as economists like to believe in our episode with Alex Imas [https://empiricrafting.substack.com/p/alex-imas-demand-collapse-bargaining]. We also ask whether, if AI gets better at predicting human judgment, whether judgment disappears, or do humans simply “move up the stack”? And what is taste exactly? Avi says that sometimes judgment becomes predictable, but humans still matter because goals, values, organizational politics, and “what matters” are often implicit, unstable, and hard to codify. Avi shoots down Seth’s galaxy-brain suggestion that correct ontology choice — i.e., deciding what sort of natural kind [https://en.wikipedia.org/wiki/Natural_kind] a thing is, or understanding when a problem is out of context [https://theculture.fandom.com/wiki/Outside_Context_Problem] — is a uniquely separate skill (taste?), calling it just another prediction error. But he does concede that deciding how much to prepare for ‘Black Swan’ events may be an enduring role for judgment. We then revisit the O-ring theory of production and what it means for automation. We had covered Kremer’s article in a recent episode (see here [https://empiricrafting.substack.com/p/weak-links-strong-predictions-kremers]) and asked Avi about his new paper, riffing on the idea at the worker level [https://www.nber.org/papers/w34639]. Avi says that if tasks inside jobs are complements rather than substitutes, then automating one task may make the remaining human tasks more valuable, not less. Avi explains why workers may reallocate attention toward the tasks machines cannot yet perform (shooting down Seth’s suggestion that this is actually difficult in most jobs). The discussion also covers whether AI will augment or replace workers, whether governments should try to steer AI toward human-complementing technologies, and why that distinction may be much harder to define in practice than it sounds. Avi agrees with Andrey and Seth’s pushback on “augmentation good, automation bad” framings (e.g. friend of the show Erik Brynjolfsson’s “Turing Trap [https://digitaleconomy.stanford.edu/news/the-turing-trap-the-promise-peril-of-human-like-artificial-intelligence/]”). Then we get into forecasts: how fast AI capabilities might advance by 2030, what that means for GDP growth by 2050, whether GDP is still the right thing to forecast, and why even very powerful AI may run into bottlenecks in the real economy. We use the paper Forecasting the Economic Effects of AI [http://Forecasting the Economic Effects of AI] to ground the discussion. We close with lightning-round topics including AI’s impact on centralization, privacy/de-anonymization, peer review, and whether academic journals still serve the function they once did. Papers, books, and ideas mentioned * Avi Goldfarb’s seminal book with Ajay Agrawal, and Joshua Gans — Prediction Machines [https://www.google.com/search?sca_esv=bc87673d3ad1280f&rlz=1C1GCEA_enUS1209US1209&sxsrf=ANbL-n4AnrHPqrHiXM4Cb3oXCBXAennzbw:1777914708243&q=Prediction+Machines:+The+Simple+Economics+of+Artificial+Intelligence&stick=H4sIAAAAAAAAAONgFuLVT9c3NEwzqCw0q8wrU4Jw003S0pMLsnK1pLKTrfST8vOz9RNLSzLyi6xA7GKF_LycykWsLgFFqSmZySWZ-XkKvonJGZl5qcVWCiEZqQrBmbkFOakKrsn5efm5mclADWkKjkUlmWmZyZmJOQqeeSWpOTmZ6al5yakAebQ6E4MAAAA&sa=X&ved=2ahUKEwjFtIC1kKCUAxWiJkQIHRQiDEoQ9OUBegQIDRAD&biw=2183&bih=1080&dpr=1.75#] * A black swan is the occurrence of a wildly unpredictable event, which Nassim Taleb argues, in his book by the same name [https://en.wikipedia.org/wiki/The_Black_Swan:_The_Impact_of_the_Highly_Improbable], is more common than we like to think * A New Riddle of Induction [https://en.wikipedia.org/wiki/New_riddle_of_induction] — by Nelson Goodman — is the source of Seth’s thought experiment about “bleen”, a color which is green until 2029 and blue after, and green * Michael Kremer — “The O-Ring Theory of Economic Development”, covered in this episode of the pod: * Daron Acemoglu and Pascual Restrepo’s task-based models of automation, especially “The Race Between Man and Machine [https://www.aeaweb.org/articles?id=10.1257/aer.20160696].” * Avi mentions David Autor and Ben Thompson on automation and skill scarcity when Seth comments that you may not be able to reallocate effort between tasks as a worker, including their paper “Expertise [https://www.nber.org/papers/w33941]” * Erik Brynjolfsson in the “Turing Trap [https://digitaleconomy.stanford.edu/news/the-turing-trap-the-promise-peril-of-human-like-artificial-intelligence/]” argues that automation technologies are less good than augmenting technology * Eric Topol’s book on AI in medicine — Deep Medicine [https://www.amazon.com/Deep-Medicine-Artificial-Intelligence-Healthcare/dp/1541644638] * John Markoff — Machines of Loving Grace [https://www.amazon.com/Machines-Loving-Grace-Common-Between/dp/0062266683] — The source of a title for an influential essay of the same name [https://www.darioamodei.com/essay/machines-of-loving-grace] by Dario of Anthropic. Both draw from an earlier poem about a Sci Fi utopia: https://allpoetry.com/All-Watched-Over-By-Machines-Of-Loving-Grace * Korinek and Stiglitz on AI, capital, and taxation; Lockwood and Korinek on optimal taxation and automation — We covered these topics at the end of our episode with Basil Halperin in the context of “Tax Policy at the End of History” around the 1:19:00 mark * We talk about de-anonymization, and Avi references this provocative paper [https://arxiv.org/abs/2409.15948] from Florian Ederer * Avi brings up Bob Gordon, and his argument, famously in the book The Rise and Fall of American Growth [https://www.amazon.com/Rise-Fall-American-Growth-Princeton/dp/0691147728], that the early 20th century was incredibly important for increases in US living standards, which digital technologies have not lived up to * Digital Hermits [https://www.nber.org/papers/w30920], by Jeanine Miklós-Thal, Avi Goldfarb, Avery M. Haviv & Catherine Tucker, is a paper by Avi thinking about how information spillovers, now from AI, drive some people to be more private than they would otherwise be. In our conversation, we speculate AI will make these hermits even more “hermetic” * We discuss this paper on new forecasts of AI and its impact on economic growth: Forecasting the Economic Effects of A [http://Forecasting the Economic Effects of AI]I * Refine and AI-assisted peer review are discussed in this pod. For more, see our episode with Ben Golub, founder of Refine [https://empiricrafting.substack.com/p/ben-golub-ai-referees-social-learning]. This episode is sponsored by Revelio Labs [https://www.reveliolabs.com/] — a great source of labor economics data for academics and firms. Now available on WRDS. Join our Discord community at this link: https://discord.gg/w3GSapx2d Transcript Introduction [00:00] Seth: Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, your loyal non-fiction machine, coming to you from Chapman University in sunny Southern California. Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. And we are very happy that Justified Posteriors is sponsored by the fine folks at Revelio Labs. And we’re very delighted to have Avi Goldfarb, who is a leading thinker in the field of AI economics and has also been a personal mentor on the show. We’re very excited to hear his thoughts on a variety of topics. Welcome, Avi. Avi: Thanks so much and thanks for having me on the show and looking forward to it. Andrey: All right, let’s get started. I have in front of me this book that you might remember writing at some point. Seth: Gaze into the soul of the man in the bookstore. What Did Prediction Machines Get Wrong? [01:12] Andrey: Now, I just think it’s a good cover. And I had to check: when was it released? It was released in 2018. And as I was skimming through it, you know, a lot of interesting points made there are still things that we’re talking about today, almost 10 years after it was released. So let me start off with the following question. And then maybe we can work backwards more into the ideas in the book. But what do you think prediction machines got wrong? Avi: I think prediction may... I’ll start with a hard question. Seth: No softballs on Justified Posteriors. Avi: So on the specifics of which industries and when, to the extent we tried, at least I did not anticipate how quickly language and coding would become prediction problems. And when we talk about disruption and industry disruption, a lot of the examples are things like driving, and we talk about radiology. And we still have plenty of radiologists around. Self-driving cars and trucks. seem like they’re now imminent, but it certainly took a lot longer than we expected back in 2018. Andrey: So is it a fair assessment to say that the large language models, even in 2018, weren’t on your radar? I guess they weren’t on many people’s radar. The Three Ideas of Prediction Machines [02:45] Avi: Not really. We have some discussion of machine translation. So that’s in there as a huge potential use case, but the arrival of ChatGPT and how it sort of changed how we interact with machines and how we think about AI was not really there. Another way to put it is prediction machines had three ideas. So idea number one is AI can be framed as a drop in the cost of prediction. So prediction. As in filling in missing information, statistical prediction is getting better, faster and cheaper. Idea number two is that when something gets cheap, you start using it for unanticipated uses. So when arithmetic got cheap, it wasn’t just that we use computers for accounting. We started to use computers for all sorts of things that we never used to think of as arithmetic problems like imaging and mail and music. And then idea number three is what are the complements to machine prediction? And we talked about data and judgment. The book, and certainly our attention to the book in the first three or four years after it was published, was on idea number one and idea number three. So identify prediction problems in your organization, and then think about what data you need to make those predictions better, and try to understand what matters to you in terms of judgment. And that second point kind of got lost. But in the last four years, it’s become clear to me is that that second point was maybe the biggest one, which is this tool, which still under the hood is computational statistics, enables us to find all sorts of applications for computational stats that we didn’t really imagine before. Judgment and data are still gonna be useful, but that phase one, that step one, that first idea of identifying prediction problems, that’s not really how we think about using AI today. And in some sense, that... was a missing emphasis throughout the book and throughout how we thought about that book, or at least how I thought about that book for the first few years. Does Proprietary Data Still Matter? [04:59] Andrey: Very interesting. You mentioned one kind of underlying idea there, whereas you should identify the data that’s going to make your predictions better. Do you think to what extent is that now true, given that your foundation models seemingly can be very smart without having any proprietary data? Avi: Data is still central to the use of AI, the building of the models. In building a foundation model that, at least in the pre-training stage, that data is essentially interchangeable. You just need more. It doesn’t really matter what. To build a structure of language, and then you can move from there. On later stages of using that model, at least the AI companies seem to think data is valuable to the model companies. And then in terms of use cases within organizations, that’s more a matter of whether you want to delegate sort of the judgment of how to use the model and what the model should output to the vendor or whether it’s something that you need to build in-house. And depending on the organization, some of them are very happy to delegate to the foundation model provider and some of them think they need to fine tune in-house. Andrey: Well, so there are kind of two little sub ideas in there. One is you have choice. You can fine tune a worse model with your own data. And maybe that will outperform as a frontier model. I think for many cases so far, that’s been a bad bet. But there’s a different idea here. Use whatever model you want, but you design the evaluation. And then you optimize via the prompting strategy or scaffolding towards that. that benchmark for your own use case. Is designing a benchmark proprietary? Should we think of that as a proprietary data that an organization has? Seth: Is that the judgment part in the judgment prediction distinction? Vendor Choice as Delegated Judgment [07:01] Avi: Yeah, I think there’s a bunch of judgment. there’s judgment number one: which which vendor do you use? Because you’re delegating a lot of values as in like, knowing what matters to the maker of the model. And then there is judgment in how heavy-handed do you want to be to make the outputs fit your needs? And then there’s judgment on, okay, you’ve decided to be heavy-handed. What exactly does that mean? And is it, guardrails or is it really making sure that the output from the prompts every time fits your organization’s values or what matters to you? Andrey: Have you had an opportunity to kind of advise companies on this judgment decision? Like what has your experience been in these situations? Avi: At a high level, yes. I don’t want to exaggerate my experience, but the things I emphasize and the things that seem to resonate are, one, what I just said, which is recognizing when you choose a vendor, you are delegating your understanding of what matters to that vendor. And then two, that means before you start thinking about choosing a vendor, you need to know what matters to you. So think through, you know, before you go talk to somebody, you should know what your KPIs are and what outcomes you want to see. Because otherwise, once you talk to them, they’ll convince you that their outcomes are the ones you want to see. and so it’s this, I talked to, someone who is running an AI at a... Let’s call it a big healthcare organization. And his job used to be, like five years ago, his job was building tools. He’s like, my job isn’t building tools anymore. There are all sorts of vendors building AI tools for healthcare. Okay. And what my job is now is every week, 20 or more people come in and say, I have a solution for you. And he chooses one or two of them. Seth: Kind of seems like a good job for an AI. Avi: Well, maybe, maybe not. But he understands the individuals, the people, guess, in theory that could happen, but the individuals in his organization, what they’re willing to accept, what they don’t. Which decisions they like to have control over, which ones they’re comfortable delegating. For the ones they like to have control over, he has a sense of what might be negotiable and what might not be. He knows where the power structures are and what things might change. Therefore face resistance from people who have the power to resist. He knows those things that might not face resistance from people because the people don’t have power to resist, but they’re going to be really, really unhappy about it. It’s going to bad for the organization. And so there’s all these things that I guess in principle an AI could do, but we’re a long way away, I think, from that. Can Prediction Eat Judgment? [10:16] Seth: So let me let me just push down that line a little bit longer is the way to think about this sort of prediction and judgment distinction is is that like as the models get better the Prediction is like eating more and more of the stack right? You know we give the information about our organizational structure to the AI and then maybe it can make a couple more of these decisions for us And you could either imagine that asymptoting to, you know, in 20 years, AI does everything, or you could imagine there are higher and higher levels of judgment that humans keep on getting promoted to. Are one of those two ways the way that you think about it? Avi: Yes, Andrea Pratt has a note in our first Economics of AI volume that covers that exact idea. I think actually it’s a comment on our paper or the model behind the Prediction Machines book. it’s, well, in principle, with enough data, you can learn to predict judgment. And so you move up the stack. So absolutely. There are some limits to that. There’s limits on you may never get enough data. on that kind of judgment. Judgment can change over time. To the extent that ultimately you’re trying to predict your tastes, then they can change over time. And there’s some limits on causal inference and the impossibility of seeing the counterfactual, which creates a need for a model. Andrey: But humans have that problem too. Avi: Yeah, yeah, yeah, no, I agree. But in the need for a model. So then the question is, well, how come LLMs and some of these models seem to be pretty good at doing that? And in the process of prediction, I suspect -- though I don’t know rigorous work on this, so I’m being cautious -- Seth: That’s what this podcast is for. Avi: this is building some kind of model of the world that is embedded in the training data, like the language. Taste, Values, and Human Wants [12:16] Seth: So let’s go back to the one of the examples you gave, which is this idea of taste, right? Because I’ve had so many conversations with other economists about this idea that, well, taste will save us as a scientist, right? Because the AI won’t have taste. I have some ideas about what taste might mean, but can you be a little bit more precise about what you think taste means and why it’s something worth saving? Avi: So, okay, let’s operate under the assumption that whatever we want to call the machines, their goals are to help humans. Okay, not all humans. And we can debate about which humans, but like ultimately. Seth: Well, the Anthropic Constitution says, you know, safety first, the idealized anthropic researcher, then the guy that then then like virtue and then like the customer in some order like that. Avi: I’m gonna, all that matters for the point I’m about to make is that it’s not about the machine’s needs. So in that case, at the very limit, humans have wants and needs and those wants and needs, the machines need us, our judgment to know what our wants and needs are. Seth: So taste literally as in, this tastes good to me, I want more of this food. Avi: That would be one specific example of it. Absolutely. Okay. Now, I think we’re a long way from that limit, but that’s what I would argue the limit is. Seth: That’s the Bailey, right? So now let’s go out to the motte. Avi: So then it’s more like, okay, what matters to a set of humans, a group, an organization? What can we codify? If you can codify it and say, like, this is your goal, you’re not quite at that limit, but pretty close to it, then the machines can try to optimize on a goal. Goals have so much that are implicit. And so the machine would have to be able to infer the implicit part. Maybe it can, maybe it can’t, I don’t know. And then you can sort of ratchet back all the way to where we are now, which is you still need to tell your agent what you want. You still need to check on it every once in a while and guide it in the right direction. Prompting still has a role. Ontology, Umbrellas, and Context Shifts [14:45] Seth: Here’s another way of thinking about taste. And I’m curious whether you think this is in one of the categories you already listed or a new idea or you wouldn’t call this taste, which has to do something like with the idea of your ontology that is kind of built into the system, right? It’s your way of sort of dividing the world up into parts and maybe a good tastemaker or a good judger might have a more refined or more adaptable ontology. than the prediction machine. So I’ll give you an example of what I mean. have a couple of examples in mind, but one example I have is, you know, historically in the data, it’s always been the case that if lots of people show up with umbrellas, it means that you can predict that it’s raining. But then we have these Hong Kong protests and in the Hong Kong protests, they’re the umbrella protests and people bring umbrellas to show that they’re protesting, right? And it seems like a human would do better at adapting to like the completely new context for why you would need umbrellas than, you know, a pre-trained system that was only on historical data. So you can say that that’s like a context switch problem. Is that one of your ideas of taste or is that more of a judgment that’s not a taste? Avi: Honestly, that seems like a prediction failure to me. Seth: Right. That’s just we don’t have data on the context that we’ve moved to. The job is to understand when the context has changed, maybe. Avi: The judgment, I would say the judgment is like, what’s the consequential decision that’s going to be a function of, look outside and I see a lot of people in umbrellas. Yeah. What am going to do? And. Seth: You know, I should water my plants. Should I water my plants? Avi: No, I water my plants. Okay. So I look outside, a lot of people are carrying umbrellas and I think, no, I don’t need to water my plants. Okay. And then it turns out it’s a protest. It’s a little bit of weird context, but going with your example. Seth: It’s gotta be a weird context. That’s the reason that the AI is going to make the wrong decision because it’s out of context. Avi: the, the automated sprinkler doesn’t go on and, my plants die. Right. Okay. So, the judgment is, is it then worth it for me to invest more either in my prediction technology or to actually go outside and look and to see if there’s rain, to overcome that downside. So what you described as an error in prediction, there’s ways to reduce that error in prediction. The judgment is whether it’s worth the bother to reduce that error in prediction or to create some kind of insurance system where you would say, you know what, I’m gonna water the sprinklers. I’m just gonna run the sprinklers anyway. That’s how I think about judgment. It’s sort of what goes wrong when your prediction fails or it’s one important aspect of judgment. Seth: Sorry, can I give you an even more abstract? Andrey: Wait, wait, wait. No. I actually disagree with the premise of the example in many ways. I think a reasoning model would be able to handle the situation, especially with internet access, substantially better than many humans already, because you can call an API to get the weather forecast if you’re unsure. You can read the news. You can use reasoning traces. There’s this kind of implicit assumption in your question that like, we’re just using a raw pre-trained model and like asking it to like, if you, like, if you had a gun to your head, what would you do? You know, and not use any reasoning. Seth: Okay, but I can tell you a story, right? The weather API was always reliable in the data, but now there’s been a government takeover and I don’t trust the new government and you shouldn’t trust the API weather data anymore, right? Avi: So Andrey, I actually agree with, like, that seems unrealistic, but I think the idea is what you’re describing is how many resources you wanna put toward making it right, and I would view that as judgment. Andrey: But I guess the model has that judgment, maybe. Already. Already. Yeah, that’s kind of goes out like the stack of when judgment problems become prediction problems, I guess. Avi: But then there’s going to be... well, there’s going to be some places where the model is imperfect. Okay. Yes. Still a prediction tool. It might be better than human. Actually, it doesn’t matter if it’s better than human. But to the extent the model is imperfect, how do you want to behave? Like, let’s say the model is right 99.99 % of the time. Does your behavior change at that versus 99.9999 % of the time, even if the human benchmark is 50? And that ultimately is going to is going to be essential to judgment. We do this with self-driving cars. The models aren’t perfect, but they’re better than human. And yet, I still drove to work today, partly because that’s the law in Canada. Andrey: Do you think there’s hope? I mean, maybe this is kind of too much in the weeds versus the abstract idea, but sometimes people implicitly assume that they’re anchoring on the current technology where there’s an instance of an LMM that does something. But we might be able to design systems of LLMs that are interacting with each other to cover some of these. shortcomings that we can think of. I mean, at a conceptual level, maybe it’s the same thing anyway... Avi: So maybe another way to think through these trade-offs is to talk about whose judgment, okay? Which is Seth’s example was about, or my example was about my judgment, know, the individual’s judgment and should they listen or not. Andre, I think what you’re describing is the model builder’s judgment on which things is it worth investing in making the model better and when is it okay not? Like they have choices on sort of rate and direction. And those require some understanding of what they think is going to matter in terms of the use cases, the model. And on that, yes, there is a limit where a small number of players have extraordinary power because AI scales their judgment because they embedded into the models. But I do think. then there is still a human or set of humans responsible. It’s not like, the AI did it. It’s humans making those kinds of decisions. And I understand, like, at the limit, that actually gets quite nuanced, especially once we have models with continuous learning. But that’s how I think about that problem. Grue, Bleen, and Black Swans [21:41] Seth: All right Andre, can I ask my riddle of induction question? Andrey: Do you need me to induce it? Seth: You already know where I’m going with this. I’m curious if Avi knows where I’m going with this, but this goes back to the question of maybe where taste comes in is having a better or a more human ontology than the machine. All right. Have you ever heard of grue and bleen, Avi? These are colors that are different than blue and green. No? Okay, awesome. So briefly, we have this conceptual category, which is a thing that’s green. And a thing that’s green, we think that if you don’t do anything to it, it should be green indefinitely, right? Avi: Okay, yeah. Seth: All right. There’s this other thing that’s called bleen and things that are bleen are green until the year 2029. And after 2029, they turn blue. Right. Here’s the issue is that bleen and green things are observationally identical until 2029. Right. Yeah. So an inhuman, bad at forming natural kinds, ontology of an AI might decide that something is bleen instead of thinking it’s green. Right? And a human’s role might be to say, no, that’s a bad definition of a natural kind. That’s a bad ontology. And that would be a role of either taste or judgment. Do you buy that? Is this way too abstract? Avi: I think what you’re describing is a failure of prediction. I don’t think that’s taste or judgment. The taste or judgment is if you or a machine aren’t sure if something is bleen or green, do you care? Seth: Okay. Well here’s the thing, you didn’t even have the concept of bleen until I told you about bleen, right? Avi: So this is just the difference, I think, between known unknowns and unknown unknowns. So in Prediction Machines, we have a whole chapter framed on Rumsfeld and his discussion of known unknowns and unknown unknowns. Look, sometimes you don’t have a prior on it, and it’s an unknown unknown. That doesn’t mean that it’s not a prediction failure. It was just off the support of your data, and you didn’t know what to do about it. And I think that happens all the time. Seth: Sometimes you find a black swan. Avi: Yes, exactly. And so like, there might be places where humans are better at that kind of prediction than machines. There might be places where both humans and machines are really awful at that kind of prediction. And if that’s the case, then you want to have robust systems to anticipate those kinds of things. And that’s where judgment comes in. Like, if you’re wrong about the existence of a black swan, you know, does that change anybody’s behavior? I think the answer is no, because black swans and white swans aren’t actually that different from each other. But if there were other examples, like financial crises, where he uses the metaphor of the black swan, then absolutely there are meaningful differences. And you should Andrey: Financial crises. Seth: All right, so you’re saying that jobs that will survive TAI number 7 should be Black Swan, anticipator. Andrey: Not an anticipator. Actually Seth, this is actually kind of the key point. The point is, anticipator of whether Black Swan affects your utility enough that you should plan for it. O-Ring Complementarities and Automation [25:22] Andrey: I think next it will be awesome to talk about automation and some O-rings. Actually, the previous episode we did, we reread Michael Kremer’s classic O-ring paper because it’s been so inspirational for so many. It’s a great paper. They don’t write them like this anymore. Seth: It’s so fun to read. They don’t like to do macro like that anymore, unfortunately. Andrey: So we were wondering, so you have your own spin on the O-Ring paper. Maybe you’ll tell, you can tell us a little bit about that. Avi: Paper makes a pretty simple point. There may be two simple points. First one is that when you think about tasks within a job, they’re not interchangeable and substitutable. So it’s not just like, okay, a machine comes in and takes tasks. Sometimes tasks are complements. Now that isn’t, I’m gonna a little cautious. We talk about that in our O-Ring automation paper. It’s not necessarily a new idea. It’s implicit in the constant elasticity models. you can have a Leontief production function. Seth: We’re talking about the Daron-style task-based models. But if you actually read the papers everything immediately goes Cobb-Douglas. It’s always immediately weird. All the tasks are substitutes and then Cobb-Douglas over all the tasks. Avi: Yes, but it’s possible to, within the canonical model, to have that. So our point number one is tasks can be complements. And I just wanted to be cautious because I don’t want to claim that that’s necessarily our idea. But it’s an emphasis maybe that the existing literature hasn’t had. And then the second is, well, once you have tasks that are complements, if a machine starts doing some of those tasks, human can move their attention to the other tasks that are not yet automated. And when that happens, the human gets better at those tasks, which then makes automation of those remaining tasks even harder because the machine has to be better than now the human who’s spending all of their time focused on the remaining few tasks. Skills Versus Tasks [27:40] Seth: So let’s pause right there because I have a couple of questions right there immediately. So one way to think about automating part of your job is you’ve automated part of your job and now I can reallocate to the stuff that’s not automated. also another way to think about tasks within a job that are complementary is to think about them as sort of like innate skills or abilities. So think about the job of being a basketball player. The job of being a basketball player involves being tall and being agile. If you somehow automated being tall, I can’t reallocate my skill points into being agile, right? If we think about my performance as more as a combination of my skills, then automating part of it or taking part of it away, it’s not necessarily obvious to me that I can get better at the thing that’s not automated. Avi: The way we, okay, so first the way the literature usually thinks about jobs is generally at the task level, not the skill level. Okay. So a worker does a bunch of tasks. Okay. Those tasks require skills, but the worker does a bunch of tasks and the A machine comes along and can do the task and not the skill. So I’m not sure what it means for a machine to be tall. What it means for a machine to slam down. Seth: Well, let’s think about being a doctor. Let’s assume you might imagine being a doctor involves bedside manner and judgment about and diagnosis right it’s not clear to me that if you automate my diagnosis I can reallocate more effort into bedside manner some people are just level five at that and some people are level one at that AI Doctors and the Future of Medical Work [29:25] Avi: It is obvious to me that there’s a bunch of tasks in a doctor’s workflow. Some of them involve diagnosis. Some of them involve talking to patients and making the patients feel better. And within those, there are skills in being good at filling in the missing information of what’s wrong with the patient and skills of making the patient feel comfortable. And actually, for some of those tasks, you might even need both. A machine comes along and automates the diagnosis skills. Okay. That means medical professionals are going to be spending more time on the other skills. This is actually an Eric Topol’s deep medicine book. I’m not sure if you’ve read it. It’s, it’s like a pre-ChatGPT, but like how AI might transform medicine. And that is his core thesis. The idea is that AI is going to make healthcare human again, because doctors are going to spend less time looking at screens and focused on diagnosis and more time. interacting with patients and making patients feel better. So in that sense, we get the automation of the diagnosis task and some of the computer tasks that should exactly lead to reallocation toward the human part. But then you brought up something else, which is, do our current doctors, if they spend that much more time interacting with patients, are they the right people for this job? Or alternatively, could we have a different set of medical professionals who we could train because now the machine can do some of those tasks who would be way better than our current doctors at the remaining tasks? I suspect if the machines get good enough at diagnosis and identifying appropriate treatments, there is an enormous opportunity for a new kind of medical professional who is focused on essentially interacting with patients. Seth: Yeah, so you’re making the occupational reorganization point and that’s that’s obviously essential and we’re going come back to that in the second. Yeah, I just I’m just pointing out that maybe maybe my example of basketball wasn’t so good. Maybe my medical example wasn’t so good. But I bet you I could pick out some domains where the elasticity of task output to effort is very inelastic. Avi: Okay, trying to think. You’ve switched from skills to task and that makes me much, much happier. Seth: Well, I mean, you would only need to worry about skills is if you were inelastic to effort, right? Then it’s just the skill. Rare Skills, Common Skills, and Wages [32:04] Avi: So there’s the new Autor and Thompson paper on automation, which I think gets at some of the things you’re talking about, which is if the things the machine does are relatively rare skills, like are tasks that involve relatively rare skills, to be precise, then what happens is we get entry into that profession. More people can do it and very likely wages go down. And if the machine things that the machine does are things that many people can do, they require less specialized skill, then the remaining humans in that job will, there’ll be fewer of them and they’ll likely be higher paid. Seth: Right, think that’s right, but I think maybe a missing component here is within the job already, what is the correlation in abilities between people who are good at the automatable and non- automatable part of the task, right? Avi: Yeah, but I think that’s the statement about that. Like in the short run, we’ll get the Autor and Thompson results. And in the long run, we’ll get a reallocation of jobs, right? There’s a system of professions and the system of professions will change. Are Tasks More Complementary Than Cobb-Douglas? [33:23] Seth: In the long run, you get the reorganization of jobs. Maybe one other thing I want to talk about before we get into reorganization of jobs is just this question about, tasks more complimentary or less complimentary than Cobb Douglas? Do you have a sense of that with tasks within a job? I mean, it seems like would vary a lot, a lot from occupation to occupation. I think we all have this intuition that they should have some kind of complementarity. That’s why they’re a job in the first place. That’s why they’re bundled. But you might bundle them and they still might just be, you know, gross substitutes that have a little bit of complementarity. Avi: I suspect there’s a lot of heterogeneity across jobs and I don’t think we have good data on that yet because sometimes we haven’t been looking because our model is substitute model and so our papers are fundamentally focused on the substitute. Seth: And I think this is an example of somehow the theory is sometimes a little bit downstream of the data, right? We just have so little data on people reallocating effort across tasks within a job that of course it makes sense to aggregate up to just add up all of the tasks done by all of the workers. That’s kind of, that’s my guess of why Acemoglu gets there. Avi: So of the task papers, the Eloundou et al., Dan Rock’s paper, is incredibly careful on every page. Seth: This is not an automation measure. Do not use this to measure automation. Avi: This could be a complement, it could be a substitute. These are just jobs that change. So like kudos to them, the four of them for being super, super careful. Nevertheless, when that paper is cited both in the academic literature and in the press, that idea seems to get lost. I’m not exactly sure why, maybe that’s because of the model. Seth: Question people want to answer, right? The people don’t want to know what job’s going to change. People want to know what job should I get, right? And so... Avi: Well, okay, but if it’s a question people want to answer, then the complements matter just as much as the substitute. I wonder if the answer that people want to know, like the answer that people want, and then they just... Andrey: I actually think it’s I think take has always been that just most people are pretty, they’re very sophisticated users of this data, but a lot of people don’t have a sophisticated economics model. And therefore to them, it’s just obvious that what’s going to happen is the machines are going to take our jobs. As a result, that’s just, they don’t have a more nuanced model of economic activity and therefore that’s how they interpret it. Now there are more sophisticated readers, think, we know some of them, where they’re just really just think that AI is going to be able to do everything in a very short period of time and then it all kind of becomes moot. You know, if you think that every single task can be done by an AI. Why the Impact of AI Was Ambiguous in Earlier Work [36:15] Seth: Yeah. Well, I guess this kind of brings us to your 2019 Journal of Economics paper, which is about where you guys kind of where you kind of throw your hands up. That’s not that’s a positive part and say there’s an ambiguous impact. So I guess I want to push you there on is the ambiguous impact because. We just don’t know all of the relevant elasticities, right? We need to know the elasticity within tasks within a job. We need to know elasticity across jobs within an organization, the elasticity across sectors of demand. And if we could put all of those together, we would be able to answer the question. Or is it more ambiguous than even that? Avi: No, I think you need to understand when that paper was written in order to understand the paper, which is in 2019 or late 2018 when we were writing it, we had no concept of anything but a task- based model with substitutes. Okay, maybe that was on us. We should have. But Acemoglu and Otter and Rastrepo were the dominant- Paradigm. ... working in literature, especially Acemoglu. Seth: Are you saying our ontology was limited? Avi: I’m not exactly sure what you mean by that, but... Andrey: You forgot about the O-ring which was the black swan of papers. Avi: Yeah, yeah. So like, we did. Seth: I mean in Kremer, I mean, presumably you looked at Kremer again before writing your paper. You can almost see he’s almost there. He’s almost at, and this is within workers too. He doesn’t exactly say it. Avi: Exactly. So when we wrote that paper, we were thinking task-based substitution. That was the model that we had. And actually, in the process of writing that paper, in some sense, we learned what was wrong with that model and ended up with, we just don’t know. And part of that is, we wrote it in 2018, 2019. We were looking for new tasks from AI. So this is before ChatGPT, like four years before ChatGPT. So new tasks hadn’t really come up yet. All we had was identifying space junk and treatment for complex disease, which actually wasn’t our idea. It was Tim Taylor’s idea, our editor. Andrey: Well, you already had AlphaFold, right? Avi: Yeah, but it’s not clear what the new task is because of AlphaFold. Yeah, fair enough. In terms of... So, and actually that paper in some sense directly led to our work on system change and GPTs, because Tim Bresnahan pulled me aside that summer at the Summer Institute and told me he hated our GPT paper. I’ve told you guys this before. Because it was a task-based model and that’s not how meaningful change happens. That then led to all this work on trying to understand, well, if it’s not a task-based model, how does the system change? Andrey: Okay. And we’ve covered that to Bresnahan paper on this podcast. Reorganizing Jobs Around AI [39:22] Seth: I guess let’s talk about reorganization of tasks. Obviously that seems to be, that’s the best case answer. The best case answer is you split off the, I guess from the perspective of a firm trying to boost productivity, maybe not necessarily from a worker’s perspective. From the firm’s perspective, you want to slice off the automatable thing, let that rip, and then figure out what you have to leave behind for humans. Is there any good research about... How do you do that? What industries are better than that at others? Like, what’s the next research frontier on that question? Avi: I think you just defined it. there are two. One is like within the firm, how do we think about where the complements are and what’s left for humans and how does that vary across organizations? The second part, and Alex Emas has highlighted this recently, is it also depends on elasticity demand for the... Seth: products. Avi: Like, you know, even if within an organization workers reallocate and they become hard to automate because they’re more productive, but then the organization is producing more, well, someone has to want that more or else then, you know, at least that organization or its competitors are going to to business. Seth: Well it’s factor, well its price will come down, know there’s a kind of a nebulous connection between price and profitability. Avi: Right. Price goes down. It’s got to go down like, well, quantity has to go up enough that we still need the workers. Andrey: There might be a paradox in there that’s not really a paradox. The misnamed Jevons paradox. Avi: Maybe. Should We Want Less Automation? [41:05] Andrey: Following up on this idea, think several prominent economists have called for a government push or ideological push to make AI that complements humans rather than substitutes for humans. Seth: Friend of the show, Erik Brynjolfsson has written about the Turing Trap. Is the Turing Trap misnamed? Is it not a trap? Should we embrace the Turing? Avi: Okay, so this is our science paper. Seth: Let’s get the hot takes. This is where we brought you on. Avi: Do want more automation? Yeah, so Eric has said it. Doron has said it. There’s lots of policy. We should complement humans, not replace them. And John Markoff is a journalist. He has this book called Machines of Loving Grace, same title as Amodei’s essay, essay, but older book. It is about the history of computing. Seth: When you’re a tech billionaire, you’re allowed to use cool phrases unsighted. I’ve noted this. Augmenters, Automaters, and Inequality [42:10] Avi: Well, they’re both referencing a poem. And in Markov’s book, there’s these two streams of computer science. There’s the, I forget exactly how he labels them, but essentially there’s the augmenters and the automaters. And at least from my perspective, the augmenters seem like the heroes of his story. And the automators who start to become prominent as this book is getting written around 2014-2015 Seth: They’re trying to trap us. They’re trapping us. Avi: But we also know that the rise of computing the internet massively increased inequality. They generated enormous wealth, but they massively increased inequality. And I hypothesize that the reason for that is, yes, they were augmenting what humans do, but they weren’t augmenting what all humans do. They were augmenting what a set of humans who are good at abstract thinking do. And those people were already doing pretty well. And so in the process of augmenting humans, right, because no human can do what the internet does or what a computer can do, they augmented folks at the top and left others with relatively stagnant incomes. Seth: Is this story there really at the task level? The way I think about that inequality story is that it’s kind of at the firm level, right? It’s we’ve now put the corner store into competition with Amazon and so Amazon wins and whatever Amazon takes as input wins. Avi: There’s a bunch of different pieces. The one I’m emphasizing is like the Autor, Katz, and Kearney framework, which is about skills. Andrey: I mean, it has to be both, right? There’s a set, right? Like, the humans who are now able to market their unique skills match with the firms that are larger, but you kind of need both to create the inequality or some of the humans become superstars without like needing the firm in first place, right? Avi: I think in principle you could get within firm inequality without getting across firm inequality. We ended up getting both. Seth: Yeah, both. Both happened. Andrey: Fair enough. Avi: but as I’m thinking like Autor, Katz, and Kearney with computing and then Shane Greenstein, Chris Foreman and I have some work on sort of the internet inequality, same kind of idea. so on the other hand, automation technology, if it’s automating things that folks at the top do, could superpower everybody else. Okay. And this is a could, cause we hasn’t really happened. So what we hypothesize, so the question, the paper is called, Do We Want Less Automation? And our answer isn’t no. Our answer is, here are reasons why it’s not obvious. Okay? It’s very economist-like. And the essence of it is, we were just talking about this medical example. Well, if what doctors are paid for is 10 years of post-secondary schooling, that essentially is about prediction, diagnosis and treatment. Then someone potentially with two to four years of post-secondary schooling who was much better at managing patient stress and all these other things, training like a social worker, combined with a diagnosis machine could be super hard. And so their productivity goes up. And there’s a bunch of industries where What people at the top do seems a lot like filling in missing information. Are Intellectuals Giving Biased Advice About AI? [45:58] Seth: One might even cynically say that these thought leaders who have been so augmented by the internet are maybe not giving the populace the best advice. Avi: Maybe. So I had an undergrad RA write an essay for me. She’s a philosophy major. you know, a couple summers ago, it’s Amelia Agarwal. I feel like I should call her out. Seth: Love undergraduate research on the pod. Avi: Yeah, the opening of her essay was, part of her assignment was to read and hear about all these people who said AI is going to automate work. And so I’m going to have to have leisure, like essentially. And she’s like, that doesn’t strike me as bad. And then she dug into it and her framing was essentially the people whose identity was driven by their, you know, intellectual abilities, public intellectuals are exactly the people most threatened by AI. And so anyway. Andrey: You know, it’s very interesting. I actually disagree. Yeah, I think lots of intellectuals are threatened by AI but not public intellectuals and that’s because humans are going to want other humans to communicate to them in many ways. So, the role of the public intellectual is not going to go away. The role of the maybe the scientist toiling away on their research. That is in my opinion much more a threat. if you’re... one might even deduce that Seth and I have started this podcast as a hedge for that world. Seth: Well, what I say is as the price of writing papers goes down, the return to reading papers goes up. But maybe this goes back to the taste idea, right? Which is one way you might think of taste is a public intellectual doesn’t let’s let’s be cynical for a minute. The public intellectual, the public art critic doesn’t actually know art better than anybody else, but they serve a role as a coordination mechanism. Right. Everybody trusts Andrey. So when Andrey points at the thing and says it’s good, everybody converges to that. And then maybe that’s one notion of taste that will be preserved. Avi: Yes, and so you started in science and moved to art. There’s probably differences between them, but in the sciences, there’s a question, or a scholar’s, what’s our goal? What are we trying to accomplish? And I think different disciplines have different goals. And depending on the goal, the role of the human curator changes. If the goal is so that humans understand the world, and have sort of a consistent model, then there’s a real role for a curator. If the goal is to build a better spaceship, then maybe there’s not such a role for a curator. And so I haven’t been following that literature, so I don’t know really what the formal academic take on what I just described is. Can Policy Steer AI Toward Augmentation? [49:27] Andrey: Yeah, I agree. I haven’t seen much formalization. So listeners, if you know of any, send it along. Yeah, I mean, I sorry, I just want to make a final point is that I think I like your criticism of this augmentation idea. But to me, there’s like a much deeper criticism, which is there’s there’s just kind of a whiff of central planning involved in it. like, how how do you know? What technologies are going to automate versus augment. Like this is very hard to predict in my mind. And to think that the government is going to like somehow implement a system of taxes on technologies that are augmentation versus substitution, it’s ridiculous in my opinion. Avi: So I was taking as given that you can understand what is automation and what’s augmentation. I agree it’s a very hard challenge. There, I think the narrative, I’m gonna be careful. I think the argument is if even without choosing winners, we might be able to tax capital relative to labor or something like that. in order to push things in a particular direction. I think that’s it. Andrey: Yeah, that’s the most plausible. Seth: That’s pretty plausible, but when you actually hear versions of the Turing Trap articulated, it’s really like go and burn down the houses of the people who want to automate you. Avi: Okay. So Korinek and Stiglitz have a chapter that’s really about tax and capital that’s in our economics of AI book. And I think like the Acemoglu Johnson argument is really about tax and capital. I’m not enough of a macro economist to have a strong opinion about one way or the other, but that I agree seems more Seth: Right, and then there’s a deeper, deeper argument there about whether or not you want to tax capital, right? There’s the old Chamley-Judd result about, well, know, labor is inelastic and capital is elastic, so really you don’t want to tax it. There’s obviously international considerations about if you have a fully automated technology, isn’t that just going to locate itself in the lowest tax jurisdiction? And so it might be very hard to tax capital. And then of course the Iván Werning follow-up research kind of complicating the original Chamley-Judd results. So this gets in the weeds really fast. Andrey: And it’s also very blunt in many ways, right? A lot of capital is not about automation. it’s a... I don’t know. Avi: Yeah, and there’s all sorts of questions in public finance and how that all plays out to like the there’s under the names Trammell and Korinek. I think it’s Trammell. No, it’s not. Andrey: That’s Lockwood. Avi: Lockwood and Korinek, thank you. have a relevant paper there. AI Growth Scenarios Through 2030 [52:36] Andrey: Next topic. Yeah. So there was a very well-circulated survey of economists about their expectations of economic growth in different AI scenarios. Seth: Now Avi, I understand you have intentionally not read this so as to have an unbiased take, so you will not be contaminated by the opinions of everyone else. Is that right? Avi: That is absolutely right. Andrey: Excellent. You’re definitely not in the same university as many of the authors. Avi: I probably will, but we’ll see. Andrey: All right. So the first conceit is that there are three scenarios for AI progress that they want us to consider. The first one is slow progress, where by the end of 2030, the AI can do PhD student level assistance, half of eight hour long coding tasks, passable stories and songs. Robotics navigate homes with some help. So that’s kind of the slow. Moderate is you have semi-autonomous labs, five-day coding tasks, high-quality novels and hit songs. Robotics can perform basic tasks. And then rapid progress outperforms top humans in research coding and leadership, award-winning creative works, nearly all physical tasks. So those are the three scenarios by 2030. So the first question is, how do you allocate the probabilities between slow, moderate, and rapid by 2030? Avi: So, okay, so with the exception of the statement about hit songs and award-winning, those are all about the models and not about the outcomes. So I’m going to ignore the hit song and award-winning part because I think that’s... Andrey: It’s of the quality of the quality that could win it. Avi: Okay, because at a high level, what I think is the technology is going to accelerate rapidly, but there are all sorts of meaningful barriers to widespread diffusion and having an impact on the economy. and sometimes I think we’re already in the slow and for aspects of the medium versus the fast, I feel like I should call it 50-50 because I’m skeptical of the like, I’m skeptical of the robotics stuff, but the five day coding task seems very, likely. And so just. Andrey: Yeah, there’s some other things. CEO level agency, you know, like is is one of the criteria. Seth: I don’t know whether or not they can run a vending machine. Avi: But don’t like part of it. So much of what a CEO does is like is charisma and creating followers, right? And I’m not sure that’s a mission. Seth: Is it charisma judgment task? Is it charisma judgment? Avi: It’s a skill. I’m not sure it’s a prediction or judgment. It’s more like an action. Andrey: Yeah. But okay, fair enough. Just to give you like a sense of where economists came in and they took this in the fall, 39 % that were still in slow by 2030, 47 % that were in moderate and 14 % then were in rapid. So you are more bullish than a typical economist. Avi: I’m more bullish. I probably shouldn’t have said zero for slow. In retrospect, I was just going to be something five to 10 or something like that. GDP Growth by 2050 [56:22] Andrey: Okay, great. Now, and I think this is the question that really there was a lot of controversy about. So, the question was, by 2050, what is the annual change in GDP on average? Avi: GDP or GDP per capita. Andrey: This is GDP. Avi: I like I have to make a population assumption. somewhere between two and 3%. Andrey: All right. You are well within the economists’ answer here: 2.5%. Avi: duplicate. And so we’ll be a little above that. Andrey: So 0.5%, that’s all we get. okay. Extra from AI over and above. Avi: Well, no, I don’t think you want to say that because the reason we have 2 % is because of innovation in past. Andrey: Okay, so fair. I agree, I completely agree with you. Avi: Like it’s possible, especially with, you know, it’s possible we would have gotten zero. Seth: 5 % better than historical rate of technological growth. Avi: Yes, something like that. Andrey: Now, what if you were for sure, what if you for sure knew we were in the fast scenario by 2030? How would that like change your predictions? Seth: It’s hard to get to above three. Avi: Like, yeah, I just think there’s a lot of bottlenecks in the economy. I think that, and we’re going to figure out what they are. Seth: We’re gonna find out fast and that guy is gonna be rich. Avi: Yes. Andrey: So you’re once again, like a very down the median economist. Avi: On growth. Yeah, okay. Seth: Can I ask you, you think that’s mostly about bottlenecks? You don’t think that’s mostly about people taking leisure? Avi: I think it’s mostly about bottlenecks. What Are the Bottlenecks? [58:36] Seth: So gun to your head, what’s the biggest bottleneck in that high growth robots are awesome scenario. Avi: I feel like my best answer is we’ll find out. Andrey: Okay. I guess the pushback that folks gave is this is a scenario where by 2030 robots can do nearly all home and industrial tasks and faster than humans, right? So you might say, well, manufacturing and physical tasks are a tiny, not tiny, but they’re not that big of a portion of the GDP already. maybe- Avi: be essentially zero is the point. If they’re that efficient and that cheap, then they won’t mean like, I guess it depends on how we calculate the deflator. agriculture is way more productive. GDP hasn’t grown by that much. Andrey: But what if we have, you know, you know, robot doctors that can do, you know, like, Avi: Great, then medicine will be cheap. It’ll be less of GDP. Andrey: I guess, all right, so here’s a hypothetical. Here’s a hypothetical. Let’s say we had a cure for cancer as a result of this, which is very plausible in the rapid scenario, and that we also, at least in principle, have the technologies to administer it through robots very efficiently because we are in a world of just true abundance. My sense is that people would value that medical care extremely highly. And if one were to properly deflate the existing cost of cancer treatment, wouldn’t that imply a very large GDP effect? Now you can say maybe we’re not going to calculate that correctly. GDP, Consumer Surplus, and Health Breakthroughs [1:00:25] Avi: Now I feel like I’m going to, you know, it’s sort of the Bob Gordon sense. I don’t think we deflated antibiotics properly. I don’t think we deflated flush toilets properly. So if you’re talking about consumer surplus, then maybe consumer surplus will be found, especially, you know, to the extent that it’s health outcomes, then huge increase in consumer surplus, much more than the argument that we’ve had for digital. Because the that debate on whether digital really made us better compared to what was happening in the 20th century, I reasonable people can be on both sides of that debate. what you’re describing, is can’t secure people living wonderfully and healthy to 100, there might be some limits to how long, but that would be wonderful and great for consumer surplus. But if that happens, I guess it might and it’s that easy, it might become so cheap that it’s it’s like agriculture. Because food is pretty essential too. And food is so cheap that we don’t worry about it so much anymore. Seth: Inelastically demanded. think people will elastically demand years of life in a way that they won’t elastically demand calories, right? Avi: Potentially. Seth: You think people will get sick of it. I thought you were to go to maybe you’ll recall in Doron’s simple macro economics of AI, a favorite paper of this podcast. He actually predicts that actually consumer surplus might raise by less than is implied by the GDP growth rate, because we’ll invent evil jobs like social media manipulator. Do you are you still convinced that consumer surplus growth will be faster than GDP growth evolves? Or are you open to this idea of the invention of evil tasks? Avi: I feel like we are not in my expertise. Seth: Turn it up. Andrey: Seth is really trying to get the hot takes. Avi: I don’t like to judge what particular products, a particular. Seth: Well, you can’t judge, you can’t predict. Avi: Yeah, you know, what am I in a- Andrey: Then you become a economist. Avi: Actually, let me give... So I think it’s reasonable for people to say some roles, some jobs, some products are better than others. I don’t think that has a meaningful role in GDP calculation. And I also worry if in our consumer surplus calculations, we economists say some things are better and some things are worse because then... So much of it is just obviously to the taste of the... Seth: It’s such a normative can of worms, right? GDP we can measure, consumer surplus. I mean, we do things at the Stanford Digital Economy Lab around trying to do willingness to accept experiments, but obviously those are highly limited too. Avi: So consumer surplus as in figuring out the area under the demand curve, that’s the kind of task I think we’re good at. It’s within our domain. whether the demand curve is morally right or wrong, that’s not something I’m going to be finding out this day. Andrey: I wanted to just like close off that loop a little bit by just saying that you just gave me an answer that said that for our evaluation of how good of a world we’re gonna get in 2050, GDP is no longer the correct sufficient statistic, which obviously makes me question like why is this such a bench? Why are people so interested in forecasting GDP in 2050 if we think it’s going to get pretty uncoupled with consumer surplus in these scenarios? Avi: Well, I’m not sure it’s more or less uncoupled than it has been in the past. I think reasonable people can disagree on that. I think the debate between Bob Gordon and Erik Brynjolfsson or Bob Gordon and others over the years is sort of is really informative about how hard it is to say, you know, what’s better versus today versus the past. What happened in the early 20th century is pretty amazing. okay, that’s point one. Point two is it’s not obvious to me that GDP like GDP tells you your national capacity. That’s what it tells you. Seth: That’s useful for things like wars and public finance. Avi: If I remember my first year econ, haven’t taught first year econ for a long time. That was the idea. What’s the industrial capacity of the country? Or what’s the economic capacity of the country? It turns out it’s highly correlated, as I understand it, with lots of welfare measures. You guys know this. And so we use it for that. Once you start deviating, then... then that’s fine, but you’re now embedding a whole other set of values. At least with GDP, we know what the values are. It’s not it’s not value laden, but we at least know what the values are that we’re embedding in that measure. Andrey: But guess I’m not sure we know, just in many conversations with economists, this question of deflators has come up and most of us haven’t spent much time thinking about what actually goes into that and how well that’s don

4. maj 2026 - 1 h 20 min

Alex Imas - Demand Collapse, Bargaining with Machines, and Behavioral AI Economics

University of Chicago behavioral economist Alex Imas joins us for a conversation on AI, economic growth, behavioral economics, and the future of science. We discuss whether AI could ever lead to negative growth, why simple “automation means abundance” stories may miss important welfare effects, and how behavioral economics changes the way we think about satiation, meaning, and human preferences in an AI-rich world. Along the way, we cover AI bargaining agents, “Marxist AI,” discrimination, mechanistic interpretability, and why Alex thinks there may still be a large future for human-valued goods. Origins & Intellectual Background * Why Alex started Ghosts of Electricity [https://aleximas.substack.com/] and how Substack complements academic research * The Bob Dylan origin of the name and Alex’s path into behavioral economics AI and Economic Growth * Two models where AI could lead to negative growth * Demand collapse: heterogeneous MPCs, satiation, and the zero lower bound * Caves of Steel, dissaving, and the possibility of a high-tech, low-capital trap * Why GDP and welfare may diverge more in an AI economy Human Preferences & Motivation * Why wireheading and pure hedonic satiation may be the wrong model of human motivation * Whether economists can cleanly separate AI beliefs from AI preferences AI Agents & Interaction * Whether AI agents can develop stable “attitudes” through repeated interaction and memory * Agentic bargaining, prompt-dependent personas, and interaction heterogeneity * Guardian agents, aspirational preferences, and AI as a meta-rationality tool AI, Society, and Risk * AI and discrimination: why scalable auditing may be easier with models than with humans * Mosaic intelligence, systemic risk, and the dangers of AI sameness Science & Knowledge Production * The future of peer review, automated science, and human-valued goods Timestamps: (00:00) Introduction (01:35) Why Alex started a Substack (06:09) The meaning of “Ghosts of Electricity” (09:51) Can AI lead to negative growth? (19:54) Satiation, wireheading, and behavioral economics (26:44) “Caves of Steel,” automation, and dissaving (38:42) Plausibility, policy, and sovereign wealth funds (41:02) Marxist AI and whether agents can develop attitudes (47:23) Agentic bargaining and prompt-driven heterogeneity (54:46) Guardian agents and aspirational preferences (1:00:25) Separating beliefs from preferences in humans and AI (1:14:15) AI and discrimination (1:25:13) Peer review, science, and human-valued goods Transcript: Seth: Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology, sponsored by Revelio Labs. I’m Seth Benzel, setting my marginal propensity to consume at exactly the right level to drive the singularity, coming to you from Chapman University in sunny Southern California. Andrey: And I’m Andrey Fradkin, bargaining with the agents in exactly the right way. Coming to you from San Francisco, California. And today, we’re very excited to have Alex Imas, friend of the show and professor at the University of Chicago, join us. Alex, welcome to the show. Alex: Thank you. I am Alex Imas. I’m at the University of Chicago Booth School of Business, Economics and Applied AI groups and behavioral science. I don’t have a tagline because nobody asked me to come up with a tagline. Seth: You know where I’m at. Alex: But I have hair just small enough to not qualify for clown college, but just large enough to be weird. So that’s what I’m going with. Seth: Erratic professor level hair. That’s exactly the optimal. Andrey: That’s right. If we combined your hair and my beard, we could almost match Seth’s hair. Seth: You mean my majestic mane, Andrey. Why Start a Substack? [01:35 - 05:02] [00:01:35] Andrey: Well, let’s get started. Alex, you’re a professor. Why did you start a Substack? Alex: That’s a great question. I’ve been thinking about that a lot, both before I started a Substack, but also as I’m going through the Substack. If you notice, when I introduce my Substack on my X account, the tagline is, “Oh no, why did he start a Substack?” [00:02:03] It was preceded by me getting into AI from economics and behavioral science. I came into it what I view as kind of late. Many people were much earlier than I am, including you two. I came at it when ChatGPT was first released, 2023. But as I was getting more and more into AI as a research topic, the way that academic papers were — the process of writing them, getting feedback, the journal process, which is what I’d been doing for decades — it just didn’t seem like that format matched the speed with which the technology was moving, nor with the types of questions that I wanted to talk about in terms of doing the science. [00:03:05] If you’ve been around the block for a little bit— Seth: You be talking like you’re an old man, Alex. Come on. Alex: It’s gray hair. They made me dye it in clown college. [00:03:15] So the way that you would write an academic paper is, in some ways, defensively. You know after you’ve had a lot of feedback from journals, you know the type of referees you’re gonna get. So there’s an idea, which is what you’re excited about. You work through that idea, and then I would say 80% of the time you’re doing defense even before you submit it. And that 80%, I feel like you just can’t afford to do that when the science is moving so quickly. So for me, the Substack was a way to do research in a format that — and this is a skills problem for me probably. I think many other people write academic papers differently. But the way that I wrote academic papers, where each paper was like a seven, eight-year process, I needed a different way of doing things. Seth: Okay. So you see both of them being complementary, right? Here’s track A, fast track, here’s track B, slow track. Or are these substitutes, and eventually you’re gonna have to fully substitute into Substack land? Alex: No, these are complements. A lot of my Substack posts either have an academic paper being developed in real time or are the idea that this is a first shot in the bow, and then these will begin being developed into academic papers. For example, one Substack from early January came with a technical note, which is essentially an academic paper that I was starting to write, and I’ve been writing that paper since. A lot of the posts are in that vein. [00:05:01] Seth: Okay, and you’re not... That’s actually interesting because I think a lot of academics would be afraid of being scooped. If you put out the key idea first, but it’s seven years until you actually get the paper published. What about a young hungry grad student taking the idea and doing the legwork of all the defenses first? Is that something you worry about? Alex: Absolutely not. One of the nice things about being an old man is the fact that I don’t really care as much about being scooped. Like, not at all. I think especially in the space of AI, it genuinely feels like we’re in such an energizing, collaborative moment. And this is gonna change after we get replaced by robots, but right now it feels like — it must have felt like this in the ‘20s in physics. Ghosts of Electricity: Alex’s Origin Story [06:09 - 09:50] [00:06:09] Seth: So who’s Heisenberg? Which of us is Bohr? Who’s Einstein, obviously? Andrey: I think Alex has the hair that’s closest to Einstein, so we’ll give it to him. Seth: I was gonna say Einstein is the Acemoglu, ‘cause he was really right until he was really wrong. [laughs] Alex: No comment. Seth: Wow, no comment. Again, why Ghosts of Electricity? Why that title? Alex: Ghosts of Electricity — I’ve been waiting for somebody to ask me this question. First of all, it’s a Bob Dylan lyric. My favorite artist, one of several favorites, but he’s up there, is Bob Dylan. He influenced my life more than probably any other individual in my entire life. I was gonna go to medical school, and then I heard a bunch of Bob Dylan records and went nuts for a while. Seth: Wait, how did Bob Dylan make you an economist? Alex: Well, he made me not go to medical school. I was like, “Hey, actually, I can do anything I want now. I’m gonna go and paint paintings like this one in New York City.” And play music on the subway and all that stuff. And through that period, I discovered behavioral economics. Fell in love with behavioral economics and then decided to go to grad school. Bob Dylan kinda took me off of medical school. Seth: What did you... You picked a Dan Ariely book off the shelf? How does one fall in love with behavioral economics while being a painter in Brooklyn? Alex: I heard a Richard Thaler interview about Nudge. Seth: Wow. Talk about a full circle story. So Nudge got you into economics, and you ended up writing Nudge version two. Alex: Winner’s Curse two. Yes, that’s right. But it is actually Winner’s Curse two — there’s a first Winner’s Curse. Seth: Everyone buy Alex’s book. Okay. [00:08:13] Alex: So anyway, I got into economics that way. My favorite song by Bob Dylan is Visions of Joanna. My favorite lyric from that song is, “Ghosts of electricity howl in the bones of her face,” which I think is the greatest lyric of all time. And I love that line, but then I felt that line about ghosts of electricity really captures the way that I think about AI. LLMs and AI, the way that they’re trained now, are almost like ghosts of people who used to exist or in the past that have written something down that these agents have now learned. And electricity — it runs on electricity. Seth: I thought it was gonna be the other angle — that we’re hearkening back to the first industrial revolution, and the ghosts of the original industrial revolution are here to give us guidance and wisdom as we move forward. Alex: I like that too. Maybe on the next interview somebody asks me, I’m gonna give them that. Andrey: You see how much foresight Bob Dylan had. He was ahead of the AI game before anyone else. Alex: He was right until he was wrong. Some of those albums in the ‘80s were real bad. Andrey: But some of the more recent ones, not bad. Can AI Lead to Negative Growth? Model 1: Demand Collapse [09:51 - 19:24] [00:09:51] Andrey: All right. Seth, I think you had some spicy questions for Alex. Seth: Yes. We’ve talked a little bit about how you got into economics. Now I wanna actually dive into all of this content on your blog. There’s one blog post that we had an interaction with in particular that I thought had a lot of provocative ideas. This was your post about models under which AI can actually lead to negative growth in the economy or somehow reduce the growth rate. [00:10:48] Obviously this is a common intuition. I remember there was a first scare about this in 2014, 2015, where people were mostly worried about big industrial robots. And I remember doing interviews about what happens when robots take all our jobs. Don’t people need money to support the economy? And I remember having these conversations about Say’s law — supply creates its own demand. Fundamentally more productivity is good. It pushes out the production possibilities frontier. Sure, we could screw up the political economy somehow, but as long as that’s being pushed out, only good and better can happen. So tell me about these models you came up with and why that naive economist answer maybe isn’t 100% of the answer. [00:11:30] Alex: Let me start with the fact that what inspired this line of thinking was me seeing your paper at the spring meeting at Wharton. Seth: Yes. Yeah, Dan’s conference. Alex: The way that I started thinking about can artificial intelligence lead to negative growth is when I saw your paper, “Robots Are Us.” Which was a very — I love the way that you pitched it, kind of like an Asimov sci-fi tale, but like, “Hey, let’s take a part of this seriously.” Do you want me to start with that? Seth: Well, have you read Asimov’s Caves of Steel? ‘Cause otherwise I’ll introduce that part. Alex: I want you to talk about that paper after. So the blog post starts out with this question and then introduces two different models. The second model is Seth’s paper, so I’ll let him talk about it. The first model is in some ways more intuitive but also more problematic. The ultimate answer to that question that starts the blog is probably not — it probably will not reduce growth. Just to get that out of the way. [00:12:46] So the first intuition I had was: labor gets automated. In a new Keynesian sort of way, can you get demand collapse? A bunch of people don’t have any money. What are they using to purchase goods and services in the economy? Firms anticipate the drop in demand, they stop producing, and then you get into these classic spirals where you get actually less output because of this automation. Seth: Let’s slow down a minute. In the classic Keynesian story, people get laid off, workers don’t have enough money to buy stuff, and then there’s some sort of nominal price rigidity. What should happen is wages should fall so workers get employed, but maybe there’s a nominal restriction there. And therefore you kind of have surplus, superfluous labor. So how is this story different than just the classical Keynesian cyclical problem? [00:13:55] Alex: What I introduce into the model is heterogeneous MPCs — marginal propensity to consume. Because what AI’s gonna do, at least how it’s modeled, is be a reallocation of resources from labor into capital holders who own the technology. And there’s literature by some of my colleagues at University of Chicago on something called indebted demand, where it documents the idea that richer people who own capital have lower MPCs than labor. If you have this sort of heterogeneity, what that means is that— Seth: We’re gonna come back to that, but I think that’s cross-sectionally true without maybe being over a life cycle true. But keep going. Alex: I’ll let you come back to that. I’ll also say that Ben Moll has a paper putting some caveats into that assumption. So none of what I’m saying is — I’m just setting something up. None of it is necessarily true. [00:15:18] So let’s say capital owners have lower marginal propensity to consume than the people getting displaced. What that’s potentially gonna do is that the people who have money to buy goods and services in the economy aren’t buying enough, and production anticipates this, so economic growth actually decreases. And then you need something like a floor on the interest rate to take care of investment. Seth: Famous zero lower bound. Because otherwise, savings are going up, consumption’s going down, at least consumption of poor people is going down. We would love it if the poor people could have more consumption ‘cause they could just employ themselves. But because savings hit this zero lower bound, there’s not even investment demand. Alex: Precisely. Seth: Whereas theoretically if investment went — if savings drove investment negative enough, at some point you would start building factories again, and there’d be jobs for people. [00:16:03] Alex: Precisely. So what I’m trying to say through all of this is that you need a lot of conditions for this to make sense. You need the lower bound, you need the heterogeneity in MPCs, you need some sort of satiation on consumption — as in at some point rich people are like, “Ah, I don’t wanna consume anymore. I have enough. I’m just gonna sit on my gold toilet all day.” Seth: Still gold. Alex: Still gold. And someone’s like, “How about emerald?” And I’d be like, “No, I only want gold.” I’m satiated. [00:16:54] Andrey: So Alex, I understand these are all these conditions, but isn’t the natural response here that we have a central bank, we have monetary policy, any competent central bank will be able to inflate enough in the right direction so that this doesn’t happen? Seth: Right. We’ve solved the new Keynesian problem. Alex: Yeah. So the second part of the post is like, “Hey, what about a central bank? It’ll potentially ease this issue. What about fiscal policy? It can fix this issue.” There’s a bunch of other levers that can be pulled even if all these conditions are met. Which is — we came to the conclusion that this is a very intuitively appealing idea. A lot of people have this idea. There’s a bestseller from the mid-2010s basically outlining this idea, not questioning it, actually saying, “This is what’s gonna happen to the economy.” And the goal of my post was just to say, “Look how much needs to happen, and the monetary policy can’t do anything, and fiscal policy can’t do anything — that’s how you get negative growth.” [00:17:58] Seth: I like how this story fits in with the new Keynesian story really well. It definitely was the case that post-2008 financial crisis, the economy kinda got stuck on this zero lower bound. But to quote our favorite economist, Tyler Cowen, you can kind of overlearn the lessons of the 2008 financial crisis. Just because maybe economic policy was a little bit not expansive enough, either fiscal or monetarily, in 2009, 2010, that doesn’t mean this is a permanent problem with the economy that we don’t know how to solve. Alex: The cause of the financial crisis was completely different. It’s not extreme productivity growth. [laughs] Seth: Right. And if you have a budget, you can solve a lot of problems. Alex: Exactly. The cause is there were beliefs about these assets that were inflated. There was a bubble, it burst. Now things that we thought used to be assets are no longer assets, then you’re getting into a downturn. Here, it’s like you’re getting extremely rich. So that’s ultimately why you need way more conditions. The problem is getting extremely rich that’s generating problems, and in some ways you can solve issues easier if you’re extremely rich. Seth: [laughs] That’s a good phrasing. Alex: My — has the best sayings. He’s from Moldova, I grew up there. He has very good sayings, and one of them: “It’s better to be rich and healthy than poor and sick.” Seth: That’s the kind of deep insight you usually can only get from an economist. But I’m glad your Zadie is coming through with it. The Satiation Debate & Wire Heading [19:54 - 26:45] [00:19:54] Seth: So of those assumptions you talked about for that first immiseration story, we talked about the zero lower bound constraint — that for whatever reason we can’t do more fiscal or monetary policy, or it’s ineffective. The other bit was that AI might redistribute from a group that is high marginal propensity to consume to its lowest marginal propensity to consume. That seems plausible. I wanna talk about the satiation point for a minute. People have very different intuitions about whether this is a plausible hypothesis. If we are really not far away from kind of wire heading itself — designing the perfect VR game that you can just sit in all day — is it really completely implausible that the rich person gets the perfect VR setup, and then they’re pretty much satiated? Why is that model unrealistic? [00:20:48] Alex: This is where the behavioral economist in me comes in. The model of satiation makes sense if all you’re thinking about is hedonics. Think about ice cream. I love ice cream. I can get satiated on ice cream — the third ice cream cone gives me negative utility. This assumption makes a lot of sense. But from a behavioral economics perspective or a cultural economic perspective, there’s so many other dimensions to utility. For example, I have a paper with Kristóf Madarász on superiority seeking and memetic preferences, where people get utility the more exclusive a good becomes. So you’re gonna get these — let’s say a firm wants to make revenue, and a guy sitting on his headset watching things is gonna say, “Hey, if you get that arbitrarily exclusive item in your video game and pay me infinite amount of money for it, but nobody else can get it,” the company will make money, and the satiation thing is gonna be undermined. Seth: Let’s talk about that for one second. What about sufficiently advanced NPCs that can always be subordinate to me and tell me how cool I am because I have the shiniest VR sword? Why do I even care about the opinions of non-AI NPCs who will continuously praise me? Alex: Human socialization is a thing. Seth: Ah. Okay. So at least for one generation we’re set. [00:22:32] Alex: I think — Oh my God, I can’t believe I’m gonna get into evolutionary psychology. Seth: Of course, dude. We go everywhere here. Alex: I think the ghosts of my ancestors are gonna hit me with a stick at some point. But we’re hardwired to do certain things. One of them is to seek other humans’ approval in order to achieve things that humans have wanted to achieve for a long time, like mate, stuff like that. Seth: Mate, stuff like that, you know. Alex: Unless that urge to do very basic human stuff gets overridden by AI, a lot of the other stuff is gonna continue to play a role. [00:23:25] Andrey: But that doesn’t tell me anything about wire heading. You enter the matrix — you’re Cypher. You love that steak in the matrix. And once you’re there, you think you’re interacting with humans, even if you’re not really interacting with humans. And presumably running a matrix-like simulation where everyone’s happy takes a finite amount of resources. Seth: Or even better, it’s just the rich people are happy for the horrible version of the model. Alex: I think if you want to run that scenario — like, put wires in people’s brains and just zap the hedonic centers — Seth: Sure. That’s the simplified version. Alex: Okay, my model’s wrong. But my comment that satiation is wrong— Seth: Where, so, here’s the fork. Is that gonna happen? Alex: I don’t think that’s gonna happen. Even if you give — in The Matrix, there’s Cypher, and then there’s other folks who wanna party in the cave. Seth: Rave in the cave. [00:24:42] Andrey: I think a related story here is civilizational projects. I have a hunch that even once AI makes us all very wealthy, we might want to pursue things like building a Dyson sphere and exploring the universe, which are gonna be pretty resource-intensive. So we’re still gonna be consuming things and making things. Maybe the AI will be doing that, but we’ll be devoting resources to that. So it’s not like we’re gonna be fully satiated. Seth: There would be GDP growth. Alex: And then this is the other dimension of preferences: meaning. We don’t wanna get too far into — the Holocaust. But the — you know, it’s Man’s Search for Meaning. Viktor Frankl. I love that book. It’s very sad. Seth: Not the Holocaust part, but the psychology part. [00:25:45] Alex: The psychology part is very deep. And I think when thinking about AGI and eventually ASI, things like meaning, identity, memetic preferences, all of these things that have been on the fringes of economics because economics has been so focused on material scarcity — I think once material scarcity becomes more relaxed, the other things are gonna play a bigger role. Seth: But there will still be unsatiated desire, right? Even if it’s an interpersonal desire, it’ll be an insatiable desire. Everyone will want a little bit more love and respect and admiration and rank and honor. And maybe the mimetics of that become complicated. But people won’t be satiated. They’ll want more of that stuff. Alex: This is my conjecture. The Caves of Steel Model: Automation & Dissaving [26:44 - 38:42] [00:26:44] Seth: Okay. So we talked about this first doomer scenario, which is the rich people get satiated, and then there’s no more economy for the rest of us. Let’s talk about this opposite story. I’m honored to hear that you were inspired by my presentation. My big inspiration was Isaac Asimov’s Caves of Steel. As I was thinking about these questions in the mid-twenty-teens, there were very few sci-fi works around societies that were automated but poor. I was trying to wrap my head around that. What would it mean to have a society where robots can do everything, but there’s not a lot to go around? Shouldn’t the robots do everything? In Asimov’s Caves of Steel, which imagines just such a society — in future New Jersey, people live in this giant underground mall. Most of them live on the dole. Some of them have small jobs that give them a little bit of extra income, but there’s no physical capital to complement the workers at their jobs. Any sort of physical capital is just devoted to the big machines that keep civilization alive and the robot farmers. And there’s anxiety that comes around when a new kind of robot is introduced that could take one of the shoe shop sales jobs, and they’re like, “We have so few jobs left. Why would you take this from us?” And there are riots. [00:28:11] And I’m trying to wrap my head around this story, and then Asimov kinda makes the clear point: the reason this is happening is their society is too impatient. If their society was really to double down on automation, and instead of having one robot per 100 people, have 100 robots per one person, then you’d have unlimited abundance. So really the tension is an intertemporal tension — between consuming today and consuming tomorrow. So in our model, automation comes along that redistributes income from the low marginal propensity to consume to the high marginal propensity to consume. So just for people playing along at home, this is the opposite problem of the previous model. In the model, this is justified by an overlapping generations framework. Young people are workers. When they’re young, they save for retirement, and when they’re old, they take their retirement savings and consume out of it, and then they die. So that’s the reason why old people who own the capital also have a higher marginal propensity to consume. And contra Alex’s point earlier about cross-sectionally people who save money tend to have high marginal propensity to consume — longitudinally, people save money when they’re younger, pay down their college debt, accumulate for retirement, and then when they’re older, they spend down. [00:30:05] Andrey: Seth, just a question on that. Empirically, isn’t it true that a lot of very wealthy old people are not actually consuming very much on the margin? They are saving that money for their generational wealth trusts and so on. Seth: Right. So the simple economics is: why not just spend all your money before you die? You can’t spend it after you’re dead. One level more complicated: maybe we want to think about there being this intergenerational dynasty — my family — that is maybe a lot more long-lived than me personally. These dynasties, except in exceptional cases, seem to spend down their wealth over more generations — it just takes longer. Yeah, it is clear that some people treat their wealth as more of a family asset than as an individual asset, and obviously families live longer than individuals. Alex: There’s also a paper that I want to pitch by my co-author Raleigh Heimer. Greatest title of all time: YOLO. It’s in finance. The paper basically documents a puzzle that old people spend too little, and then young people spend too much. And then he actually gets people’s beliefs about how long they’re gonna live, and young people think they’re gonna die pretty soon. Seth: [laughs] Alex: So they spend down, and then old people basically, once you hit seventy, you’re like, “I’m gonna live forever.” Seth: Right. What you need as an old person is insurance against living too long. In principle, the right way to solve this problem would be buying an annuity, but in current markets, annuities are all kind of completely mispriced. But that’s a whole nother conversation. [00:32:25] Seth: But to wrap up the model — we’ve now transferred the money from people who have a high propensity to save, low marginal propensity to consume, to people who have a high marginal propensity to consume. That leads society to start dissaving. And if the transfer effect is larger than the raw productivity effect from the AI, what you can get is — not the first generation. The first generation loves this because they benefit from all the productivity boost. But all future generations are worse off because there’s not enough capital to use on all the amazing new technology, and you end up in Asimov’s Caves of Steel, where there’s one robot per a hundred people, and we’re all living on the dole, and everybody’s hand-to-mouth, and there’s no saving, and you’re in a low income, high technology trap. So what did you think of that model, Alex? What was plausible? What was implausible? [00:33:21] Alex: I think a lot of the intuitions were very interesting. But when you work out the actual simulations, it’s almost like a Goldilocks immiseration growth. If you save just a little bit more or a little bit less, you basically see a very different picture emerge. Seth: Right. If the saving rate is high enough, it can absorb all of this new stuff to invest in. Alex: Exactly. In the blog post, that was my main comment — you’re doing something very similar to what I did in the first part, where you’re saying it’s possible you can get this, which is interesting conceptually. But it’s not like this is a giant, robust region of plausible scenarios where this is gonna happen. Seth: Right. You would need to absorb a huge amount of savings. There’d be no capital left over for human investment. The robots would have to be simultaneously productive enough to suck up all of our investment away from complementing humans, but also not so productive that the boost from that overwhelms the dissaving. [00:34:43] Andrey: Yeah, I think for a lot of these scenarios — and I’ve noticed a similar scenario with the fertility crisis — this goes back to cultural evolution. If we were actually in that scenario, I could imagine a new movement within society for savings — that might be religious or it might be rationalist — such that enough savings happens so that we don’t get immiserated. Similarly to how with the fertility crisis, hyper-religious people are gonna dominate the earth because they just like having a lot of kids. Their fertility rate will end up dominating in the long run as the cultural norms remain as they are. [00:35:30] Seth: Yeah, Andre making a really good point here. Compare the two scenarios about what the disaster looks like in terms of interest rates. In the first scenario, the disaster has interest rates stuck at the zero lower bound. In the second scenario, interest rates are skyrocketing, but nobody wants to save. First of all, I would say at a plausibility level, I would bet on the latter rather than the former. I think all of the productivity unlocked, all the anticipated changes, are gonna lead people to be dissaving rather than saving more. But one of the results of that is, as Andre points out, for my story to work forever, you kind of need to be stuck in this trap of everyone having a high marginal propensity to consume forever. But if you just had one small group of society that was patient — one infinitely lived endowment, the Harvard endowment, whatever group — the Catholic Church — eventually they’re gonna start running up the game with those really high interest rates. So there’s a sense in which my result is unstable. It’s unstable to there being a big enough group that has a high saving rate. [00:36:46] Alex: Yeah. Exactly. I think for both of the frameworks — to get negative growth, too many things need to align for it to be plausible. But what’s very useful from these exercises — I talked to some folks in the profession, sent earlier drafts of this essay, and they were like, “Who thinks this is possible? Who are you talking to?” And I’m like, “Okay, you need to get—” Seth: Everyone. Society, dude. Alex: You need to get out of your little office, buddy. People are— Seth: Everyone’s worried about this. [00:37:25] Alex: I think the models still illustrate forces that might not necessarily tip you towards negative economic growth, but will still — let’s say you don’t need satiation, you don’t have this lower bound in investment — you could still have demand keep you away from the technological frontier, even if it doesn’t turn growth negative. If there’s enough displacement, you would still have welfare consequences where many people are getting displaced and much worse off, even if GDP is growing. So maybe one takeaway is that maybe you shouldn’t necessarily look at GDP to measure how well automation is helping the economy because of the implications for displacement and welfare consequences. Seth: In conclusion, everything I told you about GDP is irrelevant. [00:38:26] Andrey: I do think this is a very common theme in conversations I’ve had with numerous folks — we know that GDP is not welfare. That’s not a surprise to us. But there might be an increase in the divergence of the two with some AI technologies, and just something we should be looking out for. Closing the Growth Models: Plausibility & Policy [38:42 - 41:02] [00:38:42] Seth: I wanna ask some closing questions, then we’ll change topics. You keep saying both of these are plausible stories, but they’re opposite stories, Alex. Alex: They’re plausible stories in two senses. One, one is a long-term scenario, one is short-term. Seth: Right. Okay, so you could have a short-term problem and a long-term problem. Alex: Exactly. Two, these are plausible stories from an intuition perspective, not necessarily from an economics-happening perspective. Like, let’s say you came up to somebody in the street and told them your story. People would be like, “Oh. Okay. Makes sense.” But then I could go up to that person a day later and tell them my story, and they’ll be like, “Oh yeah, that seems plausible.” Like, obviously you only have one set of facts, hopefully. Seth: Right. Either MPC is too high or too low. Or just right. Alex: But there’s a lot of — I just wanna point out that there is controversy over the MPCs. Even as economists, we’re having these conversations in journals right now — what is the actual heterogeneity of MPC? [00:40:18] Seth: Then you go on to say that a solution to both of these problems is a government sovereign wealth fund that would lump sum rebate to households — it would have to be inalienable. One thing I would point out there is the exact design of when those payments are made would be very important to determining the marginal propensity to consume. If you get a sovereign wealth fund that only supports retirement income, that will lower marginal propensity to consume. And actually might not solve the problem. Marxist AI: Can Agents Develop Attitudes? [41:02 - 47:23] [00:41:02] Andrey: All right. Well, as listeners know, I am not a macroeconomist. I’m more comfortable in the land of the micro. But I did wanna bridge the two topics to bring in a little bit of Marxism here. One of your recent posts, Alex, talks about Marxist AI. What do you mean by that? [00:41:20] Alex: So in that exercise — this is with Andy Hall at Stanford and Jeremy Nguyen — we basically looked at what happens: can an agent, an AI agent, change its attitude? And I’m putting quotes here because the way that we think about attitude as something that permanently follows us is different than an agent who resets every single time the context window opens up. These are two different things, hence the quotes. So can putting them into some sort of environment of work — a task where it’s grinding, it’s hard, they’re getting rough feedback from me being like, “Do it again. Do it again,” and then them trying and getting no feedback versus a very pleasant thing that they’re doing and they get good feedback — can these sorts of tasks change the attitudes that they have? Do they want the system to change? Do they want more equal share of resources? What we showed is that if you give them the two different types of scenarios, their attitudes towards what they endorse — the legitimacy of the system, how resources should be distributed — change as a function of their experience. And one thing the listeners probably think is, “Oh, why does this matter? Agents will just — you could just keep resetting them.” Well, as some of you know, agents can have memory now by writing skill files. When their amnesia sets in, they read the skill file, remember, and then keep going with some sort of rigged up memory system. And what these agents were shown to do is basically write down like, “Hey, you were mistreated. Remember this. Things still suck. You gotta hate this guy.” Andrey: [laughs] Alex: So basically, the skill files that they were creating for themselves were making these attitudes more embedded than you would otherwise think. [00:43:58] Andrey: So a theory that’s espoused by some people about how LLMs work is that there are different basins of personas that exist in the training data — perhaps different characters in novels or movies. And then by putting enough text into the context, you’re making the agent take a persona that might be different than the default. For example, Seth and I recently did an episode on the Anthropic Constitution — there’s a very detailed document about a specific persona that Claude should take. And you’re saying you’re able to undo this persona with enough drudgery and meanness to the agent. My question: how easy is this to undo? Alex: Yeah, we’ve all three thought about this. My guess is that it’s very easy to undo. In the sense that you essentially have to activate a different set of embeddings with the context. And so unlike — this is what I mean by putting quotes on these things — these are not the way that we think about attitudes in humans, where I have been working in the mines, I am now a Marxist. You tell me, “No, no, no. The mines were actually good. Remember, they were good.” And I’m like, “Oh yeah, never mind. I’m going back to the mines.” That doesn’t happen with people. Seth: Because we can’t edit memories, or because people aren’t that persuadable? Alex: It’s essentially the difference between the way that the in-context activation works versus the training, the actual weights of the model. What we’re doing in this experiment is not affecting the weights of the model. If we were affecting the weights through online learning — which we’re not doing, none of the models have online learning — then I would put smaller quotes on “attitudes.” [00:46:43] Andrey: I do think my understanding of how these things work is that some of the simpler weight updating techniques like LoRA fine-tuning are very superficial. Even if you did that, I don’t think it would — because relative to the entire training data and the larger set of weights, it’s so small that those personas are still in there somehow. So it is a very interesting open question. Alex: Yeah. In-context learning is a very interesting open question. What will online learning look like when it first starts being developed? Is online learning going to actually change the deep-seated base persona? Even making that distinction in a conceptually rigorous way is gonna be where a lot of research will be. But in our experiment, we were not changing the weights, which is why my answer was I think this is gonna be very easy to change. Agentic Interactions & Bargaining [47:23 - 54:46] [00:47:23] Andrey: Kind of following through this set of questions about whether context matters — you have this other paper about agentic interactions where people are using AIs to bargain. Maybe you can tell us about that. Alex: Yeah. This is with Sanjog Misra, my colleague at Booth, and Kevin Li, who was a grad student with us. We started with this idea — Sanjog has this really nice theoretical piece called Foundation Priors. The idea is that we shouldn’t think of LLMs as databases in the sense that there’s a database, I ask it a query in many different ways, and as long as it hits that one unit, I’m basically drawing data out of a distribution. Some people might have that mental model, but the way that LLMs actually work is the context around — like, let’s say I say, “Hey, you have a budget of $10,000 and spend it on a car.” If it was a database or an algorithm the way we traditionally thought of algorithms, it would just use the instrumental information — that you have a budget of $10,000 — and maximize your surplus in that negotiation. Everything superfluous wouldn’t affect its behavior. But what the Foundation Prior says is that the prompt, everything around the instrumental information, will actually be activating different types of personas within the LLM, and the LLM is going to act fundamentally differently depending on changes in that non-instrumental information. [00:49:32] And our claim was that this has serious economic consequences. If LLMs were just algorithms, then if everybody has the same algorithm and the same preferences, the economic outcomes in a used car market would go from very heterogeneous — because people are different, they negotiate differently — to very homogeneous. Andrey: Well, they’re different in their budgets. Even if it was reasoning exactly the same, they would have different contexts. Alex: But let’s imagine a world where everybody has the same budget. You would still, with humans, get a distribution because of individual differences. So our claim was: take that theory, put it into an empirical test of agentic interactions, and different people will write different prompts where the non-instrumental parts are gonna change, activate a different persona in the agent, and that’s gonna generate heterogeneity in the outcomes. Andrey: Some of us are so good at using LLMs, we always make sure to add, “Make no mistakes.” Alex: [laughs] Or skip permissions dangerously. [00:50:48] The crux of it: we ran an experiment of a car negotiation where everybody had the same preferences. We had human-human interactions, same underlying conditions, and then we had agent-agent interactions. We looked at the spread of economic outcomes, and we found more heterogeneity with agents than with humans, and that heterogeneity could be linked to individual differences in the way humans wrote the prompts. Why is there more heterogeneity? Agents didn’t use norms. Norms actually discipline economic outcomes. In a negotiation we say, “Let’s just split the difference.” Agents don’t do that. Andrey: Agents don’t know about Schelling points? Alex: Some of them were told to do it. You see the prompts and someone’s like, “Hey, negotiate, but by the end of it say 50/50.” And they did. [00:51:46] Andrey: Cool. I like the setup. Now, here’s a meta question for you. You’re an experimentalist, you’ve done a lot of these lab studies, now with AI, before without AI. There’s a concern that what we learn from these might not be as applicable to the real world as we think. And with this agentic bargaining one specifically, I’m a bit skeptical, even though I think the greater point holds. Here’s why: we’re gonna have specialist agents that are gonna be our agents for bargaining. Even if we have our own personal AI that we give context to, it will be smart enough to call the bargaining agent, and the bargaining agent will be a specialist that’s really good at bargaining. As a result, some of these dependencies on specific details of the context are gonna go away. In our Cosine Singularity paper, we argue that AI’s use as an agent in these situations is actually super promising because humans are so bad at it. I’m curious how you think about that. [00:53:13] Alex: There’s two points you’re making, and I think we’re making one of them but not the other. One point is conceptually that the role of the human in the relationship between the agent and the human is gonna play a role in how that agent behaves — like activating different personas and leading to greater heterogeneity. That’s the point we wanna make, an existence proof of that. Your second point is, what do our results hold for the economy? And on that point, I agree with you. I don’t think there’s a disagreement here. Knowing about our paper means that systems will be designed in a way to potentially avoid these outcomes. We didn’t write our paper to say agentic interactions will be just as heterogeneous in the actual agentic economy as human interactions. We wrote it to say, “Hey, this is a factor that you should think about when designing systems for agentic interactions.” It’s straightforward to think of ways to circumvent this through layered agentic interactions. But in contexts where someone is prompting an agent to do something for them, knowing that the non-instrumental parts of that interaction are gonna play a role is important. Guardian Agents & Meta-Rationality [54:46 - 59:08] [00:54:46] Andrey: A related question. You’re a behavioral economist. You’ve documented various cognitive biases. Do you think agents are going to be able to serve as meta-rationality guides for humans? Are you optimistic that’s gonna be a widely adopted use case? [00:55:09] Alex: Oh yeah, I’m 100% behind that. The main reason why I’m optimistic about AI is — Leo Bernstein and I are doing work on what we’re calling guardian agents, which is essentially everybody has their “bring your own agent,” using your terminology from the Cosine Singularity paper. A personal agent that you endow with what preferences you want that agent to have. And I was about to say “your preferences.” I didn’t, because that’s not what happens. We actually have a study running now where we ask people their preferences over a bunch of different things. We elicit their time preferences — the standard behavioral economic toolkit. And then we tell them, “Over the same choice set, we’re gonna have an agent do that behavior. Can you program the agent’s preferences?” And this is consequential — the agent will actually do it. And what you see is this beautiful result: they do not endow the agent with their preferences. They endow them with the aspirational preferences. I don’t wanna near cast or far cast, ‘cause I don’t know what’s gonna happen. There’s a wide confidence band. But there’s a world that could happen where economic outcomes are gonna be very different because you’re going from a bunch of system one agents interacting to a bunch of system two agents interacting. [00:56:38] People’s meta preferences are more wholesome and socially positive than their in-the-moment preferences. And this is across a wide array of things. They wanna consume better information than they actually do. They want the agent to encourage them to have social interactions. Seth: Wait for the second marshmallow. Alex: Wait for the second marshmallow. The agent’s not gonna keep you from having that ninth drink, but— Seth: But why not? I could pre-commit to a self-tax on myself if I overconsume something, right? Andrey: Seth has spent a lot of time in New Orleans, so his number of drinks is quite high. [00:57:43] Seth: But so these agents will help us think through things and be more rational. But like you say, that’s not pinned down. People’s meta preferences might be worse than their object level preferences. We also hear examples of people acting selflessly in the moment — running into the burning building — that they might not do if the agent was there to talk them down. Alex: Absolutely. The broad point is whatever your reflective preferences are, that’s what people wanna give to their agent. And in some cases, this could be the less empathic response. [00:58:18] There’s an interesting question here about who is really you. What is identity? If you have this meta-rationality agent telling you to be a good person and committing you to that, that might not reflect who you are — it might just be reflecting your constraints. The positive version is it’s training you to be a better person, and eventually you’ll grow into your meta preferences. You can think about this with someone who has addiction — if this helps them kick their addiction, eventually they won’t need the AI agent. But it raises a question of authenticity, especially in human interactions. This is a topic behavioral economists have been talking about for decades — what is the welfare relevant domain? When you have these models of behavioral economics, you’re now in a multiple selves framework. What is the self that is the welfare relevant self from a policy perspective? Is it the self that wakes up in the morning and doesn’t wanna go to the gym, or the one who bought the gym membership? Doug Bernheim, Antonio Rangel, Dmitry Taubinsky have been doing a lot of this work, and there are measurement exercises to try to identify the welfare relevant domain. I think all of these tools will be really important for this topic. Seth: There’s a Greek saying: “Count no man happy until he is dead.” The idea that you should evaluate lifetime utility from the deathbed — the stoic version as you look back. If lifespans get longer, maybe that makes that non-viable, or maybe it continues to be viable. Separating Beliefs from Preferences [1:00:25 - 1:14:15] [01:00:25] Andrey: Let’s move into some empirical questions. Let’s say we’re observing an AI system behaving in a certain way. Just like observing a human, we might be interested in what the AI agent believes versus what its preferences are, if it does have coherent preferences. Behavioral economists have been in this framework for a long time, thinking about separating beliefs from preferences, and you’ve done some work on this. How have economists thought about this problem? [01:01:07] Alex: This problem has been more recent in economics than you would think. The big question is how do you do welfare analysis and public economics more generally. The way to estimate preferences is you do structural estimation. You get a choice set, you see how they behave, and then you say, “Based on these choices, I can estimate people’s preferences. Now let’s do welfare analysis.” The assumption that economists have made basically since the beginning is that people have correct beliefs over the choice environment they’re facing. Andrey: Can you give an example of that? Alex: Yeah. Let’s say I have a bunch of different interest rates for a loan, and I’m trying to estimate people’s intertemporal preferences and risk preferences. I get a bunch of people’s choice data. What I need to assume to close the model — unless I have other data sources — is that people understand how the parts of the loan contract map onto intertemporal payments and all of these things. If people have what we call a distorted mental representation of the choice environment, this entire exercise breaks down. Because now their choices may not be reflecting their preferences — they may be reflecting their misunderstanding of the choice they’re actually facing. [01:03:04] Seth: So there’s two things. They could either have wrong beliefs, or somehow their beliefs could be a function of their preferences — the two could be more intertwined than we classically assume. Which of the two are you talking about? Alex: Either, either thing is gonna mess up the analysis. This is a point Chuck Manski made in a really nice 2004 paper in Econometrica about trying to do revealed preference in the context of thinking about welfare. He didn’t talk about incorrect beliefs — he talked about partial information. The econometrician might have more information than the people in the setting. Me and Aislin Boran, my frequent collaborator, and others have been working on the idea that incorrect beliefs might be present too. We have all of these experiments showing that in very basic settings — lottery choice, giving people two simple gambles — people have distortions in their representation. Things that look like probability weighting — people loving risk — are actually people not understanding the risk of the gambles. Their preferences can actually be just as well represented by standard expected utility theory, but all of the choice anomalies are being loaded up onto incorrect perceptions. [01:04:39] Andrey: How does one learn this from the data? That seems really hard. Alex: In experiments, it’s not. Here’s what Chuck Manski said: if you do it in this context, you just elicit people’s beliefs. You say, “What do you think you’re facing?” You take that, plug it into the model, replace rational expectations with the data you’re collecting, and now you go to town estimating preferences. We do the same exercise. We say, “Here’s a gamble. There are 10 states of the world. They’re randomly chosen. In one state, one lottery does a lot better; in all the other nine states, the other lottery does better by a little bit. Tell me what is the expected value of these assets.” I incentivize it — if you get the expected value right, you get some money. People think about it, and guess what? They give us the wrong expected value because they have a different distorted mental representation. We take those beliefs, plug that into the model, look at their choices and show that actually choices that look weird and anomalous are perfectly consistent with expected utility theory, but they’re not perceiving it correctly. [01:06:00] Andrey: Now I wanna shift this back into the AI world — which is much more speculative. AIs know a lot of stuff and they’re pretty smart, we think. But when we observe them doing things, we still feel very far from understanding why they do it. One can imagine a similar representation for AI decisions. Have folks tried to use these techniques for AI? Is there an application here to eliciting latent knowledge from the models? [01:06:47] Alex: There’s some of this research. I wouldn’t say there’s a lot. I’ve tried thinking of a rigorous way of doing it. For reasons we’ve already discussed — like these personas — it’s hard. I have the view that the architecture of the LLMs represents one part, a big part, of intelligence, but it’s also missing an important part of human intelligence. Max Bennett has a really nice book about this that I always recommend: “The Brief History of Intelligence.” For me, the first order question is: I have a hard time separating beliefs and preferences when thinking about LLMs. And maybe that conceptual failure is on my part, not the LLM’s part. But currently, the way they’re working, the sort of behavior we’re observing, the very easy persona switches that you can induce — they’re unstable in a very different way than humans. Humans are unstable in a much more systematic, structurally interpretable way. And it could be that actually everything is literally the same with LLMs, but we just do not have the right mental model of them. If that happens, then we can start talking about preferences and beliefs. But given our current understanding, I have a hard time separating the two in a meaningful way. Now, I think there is some value in getting their representations of the choice environment, which is a bit different. And Tom Griffiths— Seth: Wait. What’s the difference between a representation of its environment and a belief? Alex: The way I think about it: a belief is separate from a preference. And where something doesn’t have a preference necessarily, I’m not sure I can call a representation a belief. What I mean by representation is something you can elicit from them. Even in very small models — you can actually open the box and say, “Here’s how it’s representing something.” That’s what I mean. Seth: So this is the node that represents “black cat.” It knows it’s talking about a black cat because that node is activated. [01:09:52] Alex: Exactly. Like the old school experiments with cats — the old school AI-related experiments, where people opened up cat brains and saw that certain parts of the brain are responsible for coding certain regions of the visual sphere. Like, “Hey, this set of neurons is actually coding this part of the visual field, and this is what lights up when things turn from black to white.” That research fed directly into the way that Geoffrey Hinton and all those guys were developing neural nets. Seth: So that would be sense data. Maybe the distinction is that there might be an objective correlate in the LLM architecture to the sense data. But then belief and desires might be inextricably mixed up. Alex: Yes, exactly. Beliefs in humans are a very complicated object that could be tied to things like preferences in many cases. Whereas sensory representations are in some ways a simpler object. [01:11:08] Seth: We very clearly — you’re either hallucinating or you’re not. We generally don’t think about a fuzzy boundary there. And I guess just to round out this topic, this eliciting latent knowledge framework of trying to make sure the AI doesn’t lie to us is built on this distinction — the AI has its own best understanding of what the world is like, and that can be separated out from its response prompts. You’re kind of skeptical about this approach. Alex: It’s an interesting question. I’m not necessarily skeptical about this approach. It sounds like an engineering problem. Think about a very simple model where you can actually open it up and look at its actual representation. You observe it lying. It’s an engineering problem to come up with a prompt to get it to reveal its actual representation, the ground truth that’s in its head, versus what it’s distorting. In theory, you could do that with humans too — we just don’t know how to do it. With a cat, I guess we figured it out. Andrey: This seems very related to mechanistic interpretability — that entire research stream that Anthropic very prominently has been pursuing. Trying to learn from the actual neuron activations what’s going on inside the LLM. I wanted to push back a little about beliefs and preferences. I view beliefs and preferences as a modeling device — a very useful one for humans. I don’t know if there is such a thing as beliefs and preferences actually in the brain. But it’s just a very useful way of thinking about it. So it might end up being a useful way of thinking about LLM behavior as well. Alex: I’m not gonna push you. The psychology of these things — if you talk to certain psychologists, they’ll agree with me. Others will say, “Everything’s constructed. There’s no such thing as preferences. It’s all beliefs.” Then there’s the Bayesian brain folks, who are somewhere in between — the idea that you’re not actually seeing anything; you’re making estimates of what you should see, and the only time your neurons are actually firing to see something is when something is a surprise. Basically, it’s an information theoretic criterion for stopping the simulation and actually observing something. AI and Discrimination [1:14:15 - 1:25:13] [01:14:15] Andrey: Another topic I wanted to cover — you’ve done some work on discrimination. Interestingly, we don’t hear as much about this concern these days, but maybe five years ago, it was all the rage that AI helps people discriminate and there should be laws against it. New York City passed a prominent law regarding this. Do you have any thoughts on this topic? [01:15:02] Alex: I’ve thought about it a lot. Aislyn Bourne, my collaborator in all the work I’ve done in discrimination, we’ve been thinking about this quite a bit. For a long time with algorithms, there was this worry that they were gonna be scaling bias because they’re trained on human data, human data is biased. You saw this with the anecdote from Amazon where it stopped hiring women because it was looking at its training data set where very few women were hired and down-weighting those resumes. And that Amazon scenario gets repeated every single time you talk about this. AI in the way that we’re thinking about LLMs — they work differently than those basic algorithms. They’re much more complicated. But the broader point I wanted to bring up is my view that — and this is part of the positive view of AI I have, I also have a lot of fears, I hope I express them carefully— Seth: No, just all gas, no brakes, dude. [01:16:18] Alex: I think they have the potential — if we view as a society that discrimination is something that we want to mitigate — LLMs and AI are just such an incredible tool. Think about auditing human beings with discrimination studies. There’s average discrimination in a particular industry. What do you do? You go to each individual and say, “Hey, you gotta stop.” And maybe it works, maybe it doesn’t. But if LLMs were in charge of something like that, you audit the LLM on your computer. If it was discriminating — and I wanna be very careful about what I mean by discrimination: here are the underlying qualifications of an individual for the task, and discrimination means people with the same qualification, one of these people based on group characteristics is less hired, less promoted. So I wanna be clear about that definition. Seth: Although there’s the false positive versus false negative version of this, right? Even defining it that way is not so simple. Alex: You’re talking about the fairness-efficiency frontier. Yes. You have to be very careful about this. But I’m saying, let’s say you chose a point on the frontier. I’m not talking about normative stuff. I’m just talking about you have somebody doing the normative part, and they chose a point on the frontier. In the human world, it’s extremely difficult to implement that. With LLMs, you can audit the LLM, say where you are, determine where you wanna be on that scale, then roll it out, and you are getting your solution at scale. [01:18:28] There are so many thorny questions in what I said. Like, do we want this in the first place? That sounds super scary. But in the very basic question: if the goal is to get to a certain part on that frontier, it is much easier to do that with LLMs than with humans. That’s the positive vision. Depending on what your goal is, that goal is achievable with AI, and it was not achievable with people. [01:19:26] Andrey: But a counterpoint is that LLMs are extraordinarily complex, so there might be a lot more scope for unintended discrimination to enter back into the system. Alex: But the counterfactual is humans, where it’s much more complex. Because LLMs are — think of it this way. Seth, you were not happy with my response. But let me set it up. LLMs are very complex, but they’re the same. You have one model Gemini, another model Gemini, another model Gemini. The human equivalent is there’s a Seth, an Andre, an Alex. We’re each very complex, but we’re also different. [01:20:08] Andrey: I guess, this is not how we usually think about it. But there is a concern with AI that they’re all the same. The plurality of humanity, this diversity that we have, has a lot of advantages. Even if some people are discriminatory — this is Gary Becker’s point — and other people are not, then in equilibrium, maybe this is quite mitigated. But if you launch the same agent for all applications, you have a very different error profile. Alex: Yeah. In financial markets, this is called systemic risk. Andrey: Yes, exactly. That’s a great way to think about it. [01:20:59] Alex: With AI, the sameness has so many implications I wish were explored more. Let me preview a project I’m doing. We know about jagged intelligence — LLMs are really good at some domains but bad at others, and it’s hard to predict. This is becoming less of an issue as models get bigger, but we still see this jaggedness. The thing that’s brought up less is that humans are also very jagged. Some people are really good at math but barely can read. Others can read really well but can’t do math. Seth: Is that real? My sense is sure, there are word cells and shape rotators. But word cell-ness and shape rotator-ness — math and verbal on the SAT are 0.7 correlated. They’re pretty broad categories. When we talk about the jaggedness of AI, we mean something even more striking. Alex: There’s a big difference between the two types of jaggedness — that’s Sendhil Mullainathan’s generalization function paper. But as far as jaggedness in the sense of a radar plot, it’ll look jagged in a predictable way. [01:23:04] Here’s the point I wanted to make. In LLMs, all of the agents are jagged in the exact same way. Human beings are jagged in different ways. What does this mean? The role of organizations is to create something that I call mosaic intelligence, where you get different people with different jaggedness and fill out a large circle that actually looks bigger than any individual. Everybody’s complementary, they’re filling each other out. With LLMs, you can’t do that. Because they’re all jagged in the same way, you collect a bunch of them, and the thing that one of them can’t do, the group of them can’t do either. This has implications for labor markets. What you need for full labor displacement is not to replace the average person, but to replace the organization. Therefore, it’s not about minimal AGI — it’s more about true ASI when we really need to start freaking out. Seth: One thing about the jaggedness — yes, frontier models often have a lot of overlap in what they’re good versus bad at. But if we’re thinking about a coalition of smaller models, you might imagine lots of small models each individually specialized at one sub-task. That would be a mosaic intelli

23. mar. 2026 - 1 h 33 min

Justified Posteriors

2 måneder kun 19 kr.

Læs mere Justified Posteriors

Alle episoder

Kun på Podimo

Populære lydbøger