Online Learning in the Second Half

EP 42 - Einstein: How a "cheating bot" started the conversation we needed with developer Advait Paliwal

Einstein: How a "cheating bot" started the conversation we needed with developer Advait Paliwal In EP 42, John and Jason talk with Advait Paliwal, the young entrepreneur behind "Einstein, "an (OpenClaw) AI agent that went viral for claiming to autonomously complete college coursework. Together, they unpack the intentional provocation behind the product, the legal fallout that followed, and what it all reveals about the urgent need for higher education to rethink assessment, relevance, and the human role in learning as agentic AI reshapes the world. See complete notes and transcripts at www.onlinelearningpodcast.com [http://www.onlinelearningpodcast.com/] Join Our LinkedIn Group - *Online Learning Podcast [https://www.linkedin.com/groups/14199494/] (Also feel free to connect with John [https://www.linkedin.com/in/jnash/] and Jason [https://www.linkedin.com/in/jasonpauljohnston/] at LinkedIn too)* Guest Bio: Advait Paliwal is a a 22 year old founder who is passionate about building technology that makes a meaningful impact on people's lives. He recently created the Einstein bot at companion.ai [http://companion.ai] Connect with Advait https://www.advaitpaliwal.com/ [https://www.advaitpaliwal.com/] and https://www.linkedin.com/in/advaitpaliwal/ [https://www.linkedin.com/in/advaitpaliwal/] Resources: * Michael G Wagner: The Einstein AI Panic * https://www.theaugmentededucator.com/p/the-einstein-ai-panic [https://www.theaugmentededucator.com/p/the-einstein-ai-panic] * USDE Meta study on Online Learning: * https://www.ed.gov/media/document/evaluation-of-evidence-based-practices-online-learning-meta-analysis-and-review-of-online-learning-studies-revised-september-2010-107159.pdf [https://www.ed.gov/media/document/evaluation-of-evidence-based-practices-online-learning-meta-analysis-and-review-of-online-learning-studies-revised-september-2010-107159.pdf] Theme Music: Pumped by RoccoW [https://freemusicarchive.org/music/RoccoW/contact] is licensed under an Attribution-NonCommercial License [https://creativecommons.org/licenses/by-nc/4.0/]. Transcript We use a combination of computer-generated transcriptions and human editing. Please check with the recorded file before quoting anything. Please check with us if you have any questions or can help with any corrections! [00:00:00] Jason: So, tell us who are you and why are you single-handedly trying to destroy education? [00:00:07] John Nash: I'm John Nash here with Jason Johnston. [00:00:09] Jason: Hey John. Hey everyone. And this is Online Learning in the second half, the online learning podcast. [00:00:14] John Nash: We're doing this podcast to let you in on a conversation we've been having for the last three years about online education. Look, online learning has had its chance to be great, and some of it is, and a lot of it still isn't. [00:00:27] Jason: Mm-hmm. [00:00:27] John Nash: Jason, how are we going to get to the next stage? [00:00:31] Jason: That's a great question. How about we do a podcast and talk about it? [00:00:34] John Nash: I think that's a great idea. And what do you want to talk about today? [00:00:39] Jason: Well, today we have a guest, and we'll introduce our guest in a moment. Maybe we'll have Advait say hello. Just so people know that we're not just making up this guest advent's actually with us. Hey, Advait, how are things today? [00:00:55] Advait: Thanks for Great. Went on a run had some food, did some work, and yeah, ready to have this conversation. [00:01:01] Jason: Alright. Yeah, that sounds like a good day so far. And John, it's good to see you again. You know, any day I get to have a conversation with John is a good day. We had a lot of conversations this last weekend because we were working on a writing project together. The deadline happened to be yesterday, and we were working on it quite a bit yesterday, so, so, but it's nice to see you again as well. so, tell us a little bit about you. It sounds like you have an interesting educational background. Just love to hear more about it. [00:01:26] Advait: I'm Advait Paliwal, I've been in many schools throughout my life and I've transferred. In and out of many schools as well. I was born in India ended up moving to Japan when I was two. That's where I did my kindergarten. And I moved back to India in a different city in Mumbai where I did my first through seventh grade. And in the middle of seventh grade, I ended up moving to Texas and where I did my eighth grade to a bit of ninth two different schools and. the middle of my ninth grade, I moved to Michigan where I completed my high school. Then I went to Wayne State for my first year of college. Then I transferred to Michigan State for my, last two years where I graduated early and I started at Brown for master’s from where I ended up dropping out. But between my graduation and my start date at Brown, I spent a lot of time at Stanford. For a month crashing at my friend's place. Then I spent a lot of time at Harvard and MIT, just spending a lot of time around teachers, students, educators, researchers, and just trying to understand what separates the best and the like the most high, highest caliber educational institutions from what we perceive as not to be and what can they offer that we can learn from and can AI democratized the the learning experiences between different institutions. 'Cause every place I went to it was a completely different experience. I had to transfer credits. People just wouldn't understand me. I had different ways of speaking, and it really caused lot of harm and. And I think that I was okay because I ended up coming out on the brighter side, but most people don't get that chance to. I still have friends from Wayne State, for example, which is not a great college, I would argue, but they asked me like, hey, do you have a job opening or do you have any other people that can offer me jobs? But the issue is that just weren't told to create projects. They weren't told to use ai. So now they're kind of stuck with like a very specialized skillset. Now they're banking on their degree, whereas Stanford, they have a pretty high pedigree. So, anyone can get any opportunity in the world, but just a Stanford pedigree. So how do we create the same learning expectations or experiences that Stanford has but give it to every single institution in the world? is the ideal future. And I really want to that, but the issue is people just don't understand. The change in progress in technology. So, creating some sort of way for people to hear about it and create emotion is the best way to incite change. [00:04:06] Jason: That's great. [00:04:07] John Nash: Yeah. [00:04:07] Jason: \ And Advait, we connected on LinkedIn and feel free to confirm or deny any of the facts that I put out here right now. And just bringing people up to speed that maybe are listening in and maybe they didn't follow this story, but it was just less than a week ago. I saw in an email somebody was talking about this bot. Called Einstein. That was, and this is a quote off of the page. It says, Einstein is an AI with a computer. He logs into Canvas every day, watches lectures, reads essays, writes papers, participates in discussions, and submits your homework automatically. Okay. So, I saw that. I saw that in an email that somebody sent to me and I was like, oh wow. So, I checked it out to make sure it was actually true, that it was there. I looked at it, started reading a little bit more about Einstein. I went ahead, I made a post on LinkedIn as sometimes I do just kind of trigger finger, I just did a screenshot and posted something. What I posted was, it's certainly unethical, it's real, maybe it should be illegal or at least blocked question mark. Plans start at $40 a month with a link. [00:05:17] John Nash: You didn't hold back. [00:05:18] Jason: I know. It was just a, it was a knee jerk, right? It was a knee jerk. I'm going to admit to this. And it was before I'd had any connection with Advait at all. Right? So, it was just this kind of random Einstein bot that was out there. All right. But then I looked you up. And asked to connect on LinkedIn, and I was so happy, and this is true. I, this is out of my heart Advent, is that I was so happy to then to get a message from you and you messaged me and you said, hey, I saw your post. I'd be happy to discuss the underlying tech, what schools can do about it, and how it'll affect the future of education. And I thought, this is wonderful. This is a, this is exactly the kind of conversation I love to have with people. And to be honest, you're reaching out completely. Flipped my feeling about things. And we can get into talking about the ins and outs and the ethical approaches of Einstein, but not that my feelings necessarily changed about that, but all of a sudden there was a person behind it, right? And it was like, hey, this is somebody. We could have a conversation. I would be really curious and all of a sudden flipped me from being defensive to actually being really curious about what you were thinking. So, this was like Tuesday last week. By Wednesday there was a new description up, and then by Thursday I visited and there was a 4 0 4, which for those non-techies means it, this website does not exist anymore. It's permanent. And I messaged you and you told me there was a cease and desist. So, I was like. Wow. Fascinating. Let's chat about this. You agreed to come on the podcast, and so, thank you for joining us. I know there's lots of details to fill in, but was, is that all a pretty good summation of the week? Okay. So that's all correct. Okay. So, [00:07:01] Advait: Yeah. [00:07:02] Jason: So, let's go back to the beginning if you don't mind. All right. Let's start from the beginning and, [00:07:07] John Nash: I say that we're recording this on a Monday, so it's [00:07:11] Jason: yeah. [00:07:11] John Nash: been less than seven days. [00:07:12] Jason: That's right. Yes. The rise [00:07:14] Advait: released on Sunday of last week. So [00:07:17] Jason: that [00:07:18] Advait: I'd say it's been about eight days since it was introduced to the world. [00:07:22] Jason: okay. So, eight days, the rise and the, and should we call it a fall? Does that sound too depressing? The rise and fall of Einstein, [00:07:32] John Nash: No. He's phoning in from Dog Patch. I mean, he's wearing failure like a badge, like a stripe right there on his jacket. So. [00:07:42] Jason: John he may not, you know, he's younger than you are, so maybe he hasn't been through all those all those ups and downs of what it looks like to start and stop things. [00:07:52] John Nash: Oh [00:07:54] Advait: I have been to many of these failures as well. This is just one of many. [00:07:59] John Nash: Excellent. [00:08:00] Advait: I would say arguably it succeeded in the sense that it created a sense of urgency and discussion among educational communities and, that was the main goal of the project. [00:08:11] Jason: Huh. Yeah, absolutely. I would not disagree with that at all. So. [00:08:16] John Nash: Well, that, that's interesting Advait. you say that the goal was to start a discussion and so say a little more about that. [00:08:26] Advait: I can go far back, but let's just start with, the goal was never to create a cheating tool. Me and my friend, we were hacking around with this new open-source tool called OpenClaw. OpenClaw is this AI that has access to a computer. When you download it, it just runs as if a human being was inside the computer. For example, I can command it to do things, and it would just do it. Granted that there are limitations that an AI model would have, but surprisingly it was quite effective at doing things for people like coding or responding to emails or summarizing news. And my goal was how can we give this AI a sandbox computer? Because what we saw, was that people were downloading the OpenClaw tool locally on their actual main computer or they were buying a New Mac mini to put it on. And the understanding was that if this AI has a home that it can feel free to do as it pleases and you can command it without repercussions to your main device. Obviously, Mac Minis are expensive, and I wouldn't understand why people were buying hardware when you can just spin up quick servers online on the cloud. So, my friend and I, we started hacking on this tool, which was just supposed to be us provisioning computers online for people to host OpenClaw. And this was a need that we identified in the market and started solving for it. However, my friend, he's in college, and he was spending a lot of times on assignments, and he was taking away from the work that he was doing with me. And randomly as a joke, I was like, "Hey, why don't you just get this AI to do it for you? 'Cause apparently Open Clock can do anything." And we didn't really expect it to do the assignments, but it didn't he tested on an assignment that he had completed two days ago, and it just did it. And that moment to me was, in a sense, incredible and also threatening because I've been a student. I've all my friends. Literally the day before one of my friends had called me and he was like, Hey man, I'm so worried about AI. I have this current job, it's remote. I feel like I can get fired anytime I have this degree that I paid money for. I have loans. And it gave me this sense of fear for people especially students. I started questioning the entire system from the start to finish and I started questioning about what it means for the world to have an automated AI completing the jobs, what it means for educational institutions to train these specialized workers that come out into the world with a promise of a return on their investment of time, money, and, and I felt like there was a disconnect from the responsibility of the institution and the human being, because no one's consistently checking in on the student. They've just become a number. No one was checking in on the student asking, hey, do you have a job yet? How can we help? We have these alumni. And that felt quite unfair to me. And I also saw this post from Canvas that said, oh, we created this AI grader for teachers, so teachers can focus more on the in-person learning, because clearly group rating seems like a chore to do, and the dots quite just connected. And I was like, okay, the world needs to hear about this and take this seriously. And the entire phrasing of the website was very intentional because if my goal was to build a cheating tool, it wouldn't be advertised as such. was Very intentional. It was to provoke rage. And the goal was can we use the same logic that institutions have against them? Students see assignments as a chore. They should be focusing more on the in-person learning. The jobs will be automated, which means education should also be automated. Like, if any, I can recently do the work being demanded in class, then it can do real work in the world. That's what's being promised and that's what's going to be delivered. Release that and yeah, I didn't expect it to be so viral. I thought, yeah, it would provoke some thoughts. It would, maybe some people would write articles and maybe some professor would see it and be like, yeah, okay, let's create change. But I didn't think it would be that of like rage. Because to me it feels like educators are people that are supposed to question the norms. They're supposed to engage in Socratic discussions. And what I was seeing in the world was quite the opposite. It felt like rejection of the system instead of the Socratic questioning, which we are engaging in right now. And I received many emails that were quite frank. I didn't expect the languages to be used by teachers. And then among those many emails, one or two of them, we're genuinely questioning, like out of curiosity, how can we help? What can we do? Is this something will stay on? How can we build a better education system? And I have engaged with the people that are trying to spark discussion. The others I feel like are at risk at automation because the people that ate the system are usually the ones not as effective in it. [00:13:43] John Nash: This is fascinating. You've arrived. Oh, I have so many thoughts here. The trajectory of arrival of agentic AI in this moment, and then the presence of Einstein is fascinating because there has been also sort of a bit of a corner of rage and then, a thoughtful discussion around, for instance, when Comet started to come out I built a fake Canvas course and I asked Comet to take it, and it did it quite well. It’s kind of, "Einsteined" actually. And our colleagues, and Jason will affirm, and we've talked to them on this show, and we even talked to Canvas about it, saying agentic AI could be a problem for the things you've identified. And so, I think when Einstein arrived, and if it arrives as you note, as your interest in trying to spark rage in a way but in the direction of trying to show that the emperor has no clothes, I think that's an, that's kind of an interesting take. 'Cause I think most people were feeling like, Perplexity's out in San Francisco they're sitting up in their boardrooms, in their glass ensconced rooms with their marketers saying, "Hey, let's ship this thing that will let kids not have to take their courses. And then Einstein arrives and it just seems like a piling on. And so, when did you think, you would start to reveal that you really didn't mean to have a cheating tool although we all fell for it? [00:15:13] Advait: So technically it is not a cheating tool in the sense that it's a tool. [00:15:18] John Nash: Mm-hmm. [00:15:19] Advait: So, what I built was a tool and how the user uses it is how the tool is presented. Example, you can use a knife to cut food and make delicious meals, or you can use it to kill someone. I feel like this tool was branded as a knife to kill someone, whereas the tool at hand was still, let's say a knife in the sense that people can still use the product, to pull from their course Canvas, for example and create these beautiful presentations with flashcards or interactive diagrams to learn better. But that part was quite missing from what I noticed. And I'll give you the context on why I phrased it like a cheating tool instead of a helper tool. 2023 that was when AI was kind of starting to take rise. January was when I founded this company called YouLearn. an AI tutor company and our main goal was how can we empower students to learn better in classes? And we clearly added instructions that would prevent users from getting answers. If they tried to, they would engage them in discussions, and it would help them learn better. the next step for me was obviously this tool needs to be in schools because those are the people that are. Dictating what the educational curriculum looks like and what the tools the students have access to. So naturally, I just scraped every single email of professors that I could find in my university and outside in the Michigan area. And I just cold emailed them. I was like, "Hey guys, I have this really cool tool. I think it's going to help you guys. Let's work together." I received very few responses. And they were all skeptical in every class that I would go to, my first question would be, what's your gen AI policy? And again, I could feel the skepticism from professors that, yeah, current AI's capability is just not there yet in order to teach students or provide them with it just feels like a gimmick at that point. That's what the major sentiment was just because it was so early and I've realized that over time as I spoke with more people. They said educational institutions have the slowest failure cycles know in the industry. It's very hard to sell. It's very hard to adopt. And even if you do, you have to convince 10 layers of management. I, I quite decided that this is if our goal is to move at the pace of technology, then integrating it would probably be harmful actually to, to people in general. 'cause now you're not adopting with the pace of technology and now people will be left behind. So, we decided to just spin off and build our own AI tutor company. And so far, it's been going well. It has a few million users. All of them are using it to learn in, in their classes. But when this new AI agent came out, the OpenClaw that just has access to your entire computer, it made me question who the apex predator is in this scenario. And, what I mean by that, is throughout history, humans have been at the top of the intelligence chain. We have always created tools that have made us smarter, better, faster. However, what's happening is now we're not creating a tool anymore, we're creating a new form of being, like a new species that's just better than us at everything. This AI has judgment. This AI has agency. This AI has a stable execution environment to do things in, and it's just better. Now it's very hard to convince someone to, to try to keep up because eventual tool will end up replacing all of knowledge work. And then humans will be very skilled in a very like sliver of the world and question their place in it. And I think that the, that's what I was trying to expose with Einstein, is that what's happening is colleges are over preparing people for the industry in a very tiny sliver, and they're not integrating AI into their assignments, into their coursework as deeply. And if I did try to pose it as a learning tool, it wouldn't cause a rapid change as a cheating tool would. And obviously we know the internet. It loves negativity, it loves spreading hate. So, in this age of attention, you need to play to that. And I feel like that was pretty effective in the sense that now people know the harms and they will realize that actually this has a lot more potential than what it says to have. [00:19:56] Jason: Yeah, I thought it was interesting just that, as John said, like it sounds like from the capabilities of this, it's not something new. It's something that we've had in the last five or six months, and that we've tested out. A lot of people knew about it, but that's what was really fascinating. And even Perplexity, to be honest, it had been advertising themselves as a cheating tool. Here's your 4.0, secret in those kinds of things, right? As they're trying to push out perplexity to students. And yeah, it was interesting that fervor that arose out of the response. So, at least right on the front end, you had a free trial of it, but you also were selling subscriptions to it. I'm curious did you have people subscribe and pay for this? What happened with that? [00:20:45] Advait: Yeah, so the main goal was building an AI with a computer, and the product that I made just had a website. The core product is no different in the sense that it's still an AI with a computer. And Einstein was just one of the use cases, for example, and yeah, it's had a free trial for a while actually. But what ended up happening was. So many people signed up that, we were losing money in the sense that I'm like one guy. I'm not a big company. I don't really have the resources of a big company to, to sell products for free. And the issue was that each provisioned computer instance costs money. Each one comes with a file system. Each one comes with its own network policy. It's all on like AWS, there's a ton of like security around it. It's just expensive. It's not a simple chat bot because chatbots are just stateless. You send them a message, they respond, but each individual person was getting a stateful execution environment and that costs a lot of money. [00:21:50] John Nash: Each person was getting an agentic claw bot [00:21:54] Advait: Yes, correct. OpenClaw. [00:21:56] John Nash: Yeah. OpenClaw. Thank you. [00:21:57] Advait: Hosted on a private secure sandbox. And yeah, that does cost a lot of money. So, when people signed up, the curiosity brought them to the website, but the emails that I was getting where people were asking me, "Hey, can this help me with job search? Can this help me draft a resume? Can this help me learn new skills?" Like I, I feel like what you see in the public is not what actually happens internally. [00:22:20] John Nash: Right. [00:22:21] Advait: The users we were getting. We’re seeing the possibilities that their mind was open, and we actually had quite a few PhD students very interested in it doing research for them in the sense that it would go out find papers surface it to them saying, Hey, this maybe suits your interest, and it would work overnight while they were sleeping. So, these kinds of emerging capabilities are very hard to see for people, I would say. [00:22:47] John Nash: It is almost as if what you released was a, "red herring" is not the word I'm looking for, but I mean, as an entrepreneur who's looking to create software solutions that might involve AI for the world and who's being thoughtful about what the role of AI might be, if you were to take a page from the Silicon Valley playbook where the founder has to get out of the office, go talk to users, look at unmet needs, it sounds like you discovered a ton of unexpected, unmet needs that your um, adjacent solutions might help be, you know, 'cause "Can it help with my job search? Can I do research with it?" That you would never get if you asked directly, but by dropping this, bomb that, that taps into, and Jason helped me figure out what I'm asking here, 'cause it's, it, what Einstein did was tapped into, people may have an unmet need or a struggle to make progress on achieving getting information or getting something that AI can do, that OpenClaw can do. And it doesn't have to necessarily be what Einstein does, like to cheat on exams or cheat on courses, but to make progress on struggles I have that are that what is it though? Is it they, have they lack bandwidth? Or do they lack knowledge? Or do they, but whatever it was Advait you, you, then got all these people knocking on the door. Probably seeding your mind with a ton of new software ideas too. I bet. [00:24:19] Advait: Surprisingly, a lot more professors signed up. And they're all actually, so your question was, did you make money, but you know, everyone was on the free plan and most of them used dummy debit cards or they ended up canceling, so I. of them, they just signed up to see what the craze was about, and then they ended up canceling. Yeah, most of them were professors. They were just testing out the product. Some of them made videos. The most interesting thing for me was when I looked online at what their thoughts were, I immediately realized that there's a lack of AI awareness among the general public. There's a huge delta between what people think AI can do versus what it actually can do. And I have a feeling that this product may have ended up closing or reducing the delta at least for people to understand, "okay, this is where we're at in the general public now. Like this is what the capabilities are, this is what students know because they're at the forefront of technology, they're looking for these things actively, whereas professors may be more lenient about it." [00:25:23] Jason: Yeah. So, picking up the story then a little bit, and this is great conversation. It's really interesting to hear all the background. So, you've got some people working the free trial. You had a shifted to a paid subscription to keep up with bandwidth. Talk to me about changing the language then on the front page. [00:25:41] Advait: Good question. So, we were receiving a lot of letters of cease and desist from different universities, different companies, and I wasn't sure how to deal with that mainly. That was mainly my inexperience 'cause didn't think it would cause, like, legal issues. I wasn't aware that Einstein's name was under a trademark as well. [00:26:05] Jason: Mm-hmm. [00:26:06] Advait: that was just my lack of experience on how to deal with legal issues. [00:26:10] Jason: Mm-hmm. Yeah. You. [00:26:12] Advait: up in the page just being I just took it down because I didn't know how to correctly show it because there were just so many different things that people could pick on and create lawsuits, and I felt like the main goal was to spark discussion. And that had happened in the two days that had passed. Which led to that decision. [00:26:32] John Nash: The multiple cease and desist orders from not just companies, but also universities. [00:26:38] Advait: I'm not sure if they're based in reality in the sense I, and maybe a lot of these are scare tactics. [00:26:43] John Nash: Right. [00:26:43] Advait: at the end of the day, I didn't want to cause any legal troubles. [00:26:47] John Nash: No, of course. That's what had me curious is like what standing did some of them have actually to have [00:26:53] Advait: Yeah. [00:26:54] John Nash: and desist, but. [00:26:55] Advait: And most of them really don't because this is not a tool that does things for you. It's literally a computer. It's like, how can someone prevent a student from buying a computer downloading and just like it versus. Spinning up com spinning up a computer in the cloud. 'Cause these resources have become more accessible to people. It's actually cheaper to spin up a computer on the cloud than buying a new one. If anything, uh, this is akin to buying a computer and there, there is no basis in that. [00:27:30] Jason: Yeah. And so if we can move on from even your initial reach out to me. So, we've talked a little bit about the technology, how it started up, how it. How it ended. And one of the things that you had suggested was, thinking about the rise of agentic ai, I think , we're at an inflection point right now, so, you know, John and I have been talking about it for five or six months, but right now, I think in a public awareness, we're at a little bit of an inflection point. I think it just hit, as John said, in that right way. We've got new AI agents, obviously Claw Bot, Molt bot, Open Bot have all come out in the last month or so here. So, what do you think that schools can or should do about it? Like if you could, from your perspective, you're at least half my age, maybe a little bit less. And so, you've more recently been to college than I have. You're looking forward into your life, both in terms of career and from your generation. what do you think that schools can or should do about it? [00:28:30] Advait: Great Question. So, I feel like what's ended up happening in the last, let's say decade or more, is that schools are becoming less... and by schools, when I say schools, I mean. The average school, because I would argue that schools like Stanford are on top of it. They understand the latest technologies. Their courses are constantly evolving. They've got the funding they can keep up. But the average person doesn't get to go there. So, for the average school, it's very hard to do those things. And I have a feeling that what's going to happen is higher ed may actually end up becoming irrelevant as AGI comes around. And by AGI, I mean, I just mean an AI that is as capable, if not more than a human being at doing tasks end to end. And if that happens, this world is going to be filled with an increase in supply of knowledge. Schools are basically an input to create higher output. Students are spending four years of their lives learning things, understanding how the world works, so they can come out and create some sort of economic output. If an AI agent creates more economic output, or a company, why would they not just use that? So, there's economic incentives around using AI agents, which in turn will force the institutions to evolve. Because human being is shaped by influences and educators have such a high influence, especially at a young age, up to older ages. So, what we need to do is figure out can we make this human being good at creating more output in the world? And a lot of the times, more input helped because it would increase the knowledge because again, knowledge was in low supply. But now when knowledge is in high supply, what they can do is provide students access to AI tools, do more project-based learnings, do more apprenticeships, where they send students to maybe a company, get them to work there, create some actual incentive alignment between the company and the schools. I feel like we're going to see a rise in that. And and maybe move off more digital learning into more physical, there could be chips. To do with lab-based testing. I feel like liberal arts will become far more important because now you have to think about governance of AI. What are the ethics? How do you verify what the AI did? It is, it's a lot more non-technical work to be able to delegate to an AI agent. I think these things are going to become so important in the near future. [00:31:07] John Nash: Did you have a great teacher in your past? [00:31:12] Advait: Yes. I've had had this teacher, she. She would let me do whatever I wanted, like that. And that was the best thing because I was actually keeping up in class. In fact, I was exceeding as a math teacher in high school. There was a chemistry teacher too. He was incredible. He would let me stay after class and just finish my homework while everyone was gone. I feel like teachers that give more freedom to students and actually there was a teacher who caught me cheating in an assignment as well, and he gave me slack and I would say these are the teachers that. I love. And there's one professor who actually listened to me when I asked, what's your GenAI policy? And he started questioning, yeah, what is our gen ed policy? Why don't we build a gen ed tool together? So, I built an AI tutor in their course that would pull from their course assignments of students answer. It would replace, it, replace all the discussion boards at that point. 'cause the AI was just so good at answering questions. Uh, But the teachers that. Gave me more, a little more freedom. They kind of understood where I was going at. They integrated ai. I think those were the best teachers, the ones that were moving at the student's pace rather than enforcing walls around. [00:32:20] John Nash: And I that's really cool. And I'm very excited that you had those kinds of people in your life. And I ask, because as you were describing what the world may become, I'm wondering were your great teachers so good that no agentic AI could do better than them? They're still [00:32:39] Advait: Yes, [00:32:39] John Nash: still [00:32:39] Advait: so. [00:32:40] John Nash: necessary. How might we, then ensure we have good human teachers going forward? [00:32:47] Advait: Very very good question because I think the human is so important in that because are mostly like memetic creatures. We look at people, we try to copy them, especially people in positions of authority. I think hu and we get inspired by people. We don't get inspired by chat bots. Right. And I think the role of the human is. To facilitate discussion and to facilitate the AI. My ideal education would be an AI tutor that every student has an AI teacher grades assignments, and the humans are just interacting AIs are just doing the grading. They're doing the work based on the student's explanations and. The entire goal is to create some sort of Socratic discussion. Just engage in discussions, maybe even do labs so the student can get the AI create things for them. I feel like the world is going to reward more being able to delegate, being able to assess what's going on internally. I think those are two qualities that would become very important compared to being able to do the work and memorize things. It's like. Right now, I would argue that people that can calculate numbers really fast aren't that useful, marginally useful compared to a person that can use a calculator really fast. They're the same people. So, we need to understand how the world would end up in order to create these educational experiences. I've actually I've had great relationships with educational institutions where I didn't have to, I wasn't forced to do anything in the sense that I crashed at my friend's place at Stanford for a whole month, and I was just a student there. I would attend classes; I would engage in discussions with professors. I would even I even had research, like open opportunity for me, the falling fall. Right after that, I went to Boston and I was crashing at a friend's place at Harvard and I was helping. but research at MIT and these experiences around incredibly talented people and professors. It like 10Xd my curiosity, I would say, because I wasn't forced to learn anything, I could just attend any class and engage in discussions out of my pure need for curiosity. And I think people aren't like that right now because the education system has trained them to be obedient, to follow the rules to not question the requirements. And if we create those kinds of people, these are the exact people that will be replaced by artificial intelligence 'cause the. is really good at following orders. It is really good at follow a set of instructions. So, maybe inspire more creativity, maybe inspire more curiosity. These are human qualities. And AI cannot have real life experiences based on scarcity to elicit creativity, but humans can. And I think those things would be much more important. [00:35:36] Jason: So later this week, helping to tee up this conversation among faculty about agentic AI. What can we do? One of the big questions, it's one thing about learning, right? And I completely agree. I loved hearing about your learning experiences. I'm very optimistic about students who want to learn that they will learn and they will be curious, they'll find their opportunities when given the opportunity for freedom, they will use that to learn more and et cetera, et cetera. So, I completely agree. One part of being a professor. That's most of us. It's not our favorite part. Okay? If, For any of us that are teaching is actually assessing what has been taught. We have a responsibility to assess and to feed back to the school whether or not the student has actually earned the grade that we said that they have earned. We usually have to have some artifacts around those grades to show ideally, these all tie into learning objectives, right? That we're only assessing the things students should be learning. I hate busy work. I hate assigning it. I hate doing it. I don't suggest it. Right? Just have your students do things that are actually worthwhile that would help them learn, help them be more successful later, et cetera. But the question around assessment comes how do we assess then in the age of agentic AI that can do the assessment? They can pretty easily do a quiz. I can open up my Claude Cowork and just open it up to my school folder, have it write papers for me. Then open up, my Comet browser and have it submit everything for me or whatever. [00:37:11] Advait: Yeah. [00:37:12] Jason: What would you say to, to professors as we move into this agentic age? [00:37:17] Advait: Great question. How do we assess? I would actually counter with what are we assessing for, what do we need to assess? Like, for example, hypothetically if an AI agent can do better work than I can. I think the assessment would be, how well can you orchestrate these groups of AI the task at hand? So, for example, let's say I have a history assignment and my goal, my previous classes, let's say, would ask for a paper this time in the age of ai. I would actually say, why don't we try making a movie instead of a paper? Try using these AI agents, try figuring out a script, generate images, weave them together generate a movie that explains it, that shows your creativity and also explains what you learned. And now the outcomes are just so diverse that it. You can see and feel the human creativity. I would imagine that right now, if that question was posed as an essay assignment, you would quite get very similar outposts because everyone's copy pasting ChatGPT. But when you add these extra steps and layers to assignments, you'll realize that actually not good at everything. It can't just make a movie by itself at least yet. And I think that would be a way of evolving with tech. Maybe the next step would be can you get people to watch the movie? 'cause that's the hard part Then. How would real people interact with it? Like in a course that teaches software, what is the end goal? It's either to create new research or it's to create a product that makes money. So why do we teach the blocks when we can just the student to build a building? Why don't you. it, now we can start assessing the end goal instead of the step by step. What is the outcome we expect this major to have in society, and how can we just beat that up into a class? How can a student create real value in a class? Because that would make people more excited. Now I was in this program. It was called Build Space. It was a school for people with curiosity, it was six weeks, I believe. They had a few seasons with it, like a few semesters. And it was basically people would come in with an idea and every week the assessment would be, how many users did you get? How much revenue did you get? What progress did you make on that idea? The amount of creativity that I saw in that room incredible. Like they were artists, there were musicians, there were people creating a YouTube channel from scratch. There were people building a startup. It just catered to their true interests like creativity, it, and at the end of the six weeks there was this demo day where everyone could just demo what they did, the progress that they had, how it impacted the world. It was very, what's your curiosity and how can you impact the world with it? And that really brought out the best in people. And I would imagine that in a course it, there will be students that, that try to game the system, but I would focus more on the people that don't. Like, and there are actually a lot more students that don't want to gain the system than people who do, because truly I've had these professors that were just so passionate about the subject that they were teaching, that it made me drop everything I was doing and just sit and listen to them. They would include jokes, they would just be human, which an AI cannot do. But when it's a professor that's just reading lecture slides or they gave me a prerecorded video online, I can see the effort that was put into the assignments, and I mirror back the effort in the assignments. I would say students are very reactive in that sense. They can sense effort the teacher has put in into the assessment and they would in, they would return it with their work. [00:40:53] John Nash: You raised some things that are are in discussion with my colleagues when we think about. high school education and even undergraduate college education and also something that one of our previous guests, Dr. Brandeis Marshall, has brought up about "what's un-AIable?" And you touched on it. I mean, the topic say was history, but how do you assess? And my colleagues would say. It's not necessarily being able to regurgitate or state back the historical content, although that's important. But can you demonstrate critical thinking and problem solving? Can you demonstrate that you are an effective communicator with other human beings, not with agents? Can you demonstrate that you can collaborate? Your demo days touch on these things. And so, I think what we're all wondering is that in the space where a lot of courses are now online and where agentic AI is inserting itself, how do we assess in what we call public demonstrations of learning? They're like your demo day. And it's un-AI able. I if, if IH hold you, Advait, to some criteria I have for my course that you have to demonstrate to me your thought processes that brought you to your demo, you have to tell that to me. You can't have AI tell me that. Is that what you're driving at? [00:42:14] Advait: Correct. And I do have this thought that online education's entire goal was to democratize education and learning across different communities. And I feel like, it's not as effective. And please correct me if I'm wrong, but I would argue that online courses just lack that human touch. I, I can't discuss with real human beings in person. So maybe like some, it I, there's a lack of physical space as well. So, creating physical spaces for education I think would be very important. I love how the Greeks did it. They just had these like small communities of. thinkers just for the sake of creating virtuous people. And I feel like in the educational institution sense we can create these small communities and also reduce the cost. But I feel like that's a conversation for another day. But yeah please, like, tell me about online education. What are your stats on that? Do, are people learning are they getting actual outcomes in the real world? Is it [00:43:13] John Nash: Say, Jason's going to answer this, but I'll just say to you, Advait, that this is all Jason and I talk about, and actually our sort of raison detre is to humanize online learning and make it as interesting, and engaging, and effective as any face-to-face experience. Jason. [00:43:33] Jason: Yeah. And from a research standpoint, we now, we have thousands of articles that point to the fact that an equivalent online program or online course can either give you an equivalent or outstrip actually the outcomes of face-to-face there's lots of research that actually supports that. However, I do agree in terms of the so just like meeting outcomes, like learning your, getting your objectives met basically in terms of that learning. I completely agree. In terms of the, the personal connection, and this is one of the things that John and I talk about, which we really think is that growing edge of online learning for the, in the second half of life. This podcast is called "Online Learning In the Second Half." Talk about, you know, we've been able to deliver content to people even have good outcomes for 20 or 30 years now, you know, and it's getting better and better. The media's pretty good. I like it from a, as you said, a democratizing standpoint that it, it provides access to people that normally couldn't provide or be able to access good education. Asynchronously is ideal. You know, think about people who are working and they have kids, all these kinds of things, right? I love it for all those reasons. However, that growing edge is feeling like that you're really connected in a learning community and that you're connecting in such a way with the fellow students as well as your teacher, that it really does inspire you to learn and persist. And I think that's the part, persistence isn't fantastic with online. And I think that's part of the part of the concern right there is that personal connection that helps students persist through the end of a course and through the end of a program. [00:45:17] John Nash: So I would say Advait just to add to that, the picture you were painting, that about what your ideal teachers did and what would keep us away from the allure of an agentic tutor or an agentic tool coming in to do my work for me , and creating a community, it all comes down to learning design. The learning design that is created by, either the professor or the organization who is intentional about having those be the outcomes, those things we've been talking about. And so, it's really, it's just an active it's an agent, a different kind of agentic decision. It's a, it's an active decision on the part of the humans in the room to decide that's how we're going to do it. And unfortunately, we don't always do it that way. [00:46:02] Advait: Yeah. I would add on, and I think the only thing missing from an online education or learning perspective is the human, right? Like, it, it is basically a in-person teacher. It is basically an a course, like an in-person course minus the teacher. Because that's what the online section is, right? Imagine if I just had access to a course with a bunch of videos, except an in-person one just has a teacher in front. think an online can do that with some sort of AI system that can understand all the pieces of data that's involved in the student's educational journey. I think in person it's quite hard to see, oh, is a student paying attention? Are they watching? online, at least you have that data because everything's on the platform. And now on top of that, you can abstract that with this AI agent that as soon as you log in, it's like, Hey, how are you? Let's learn together. That's what are we having difficulties with? And it creates this empathetic, patient and caring environment for the person to feel not judged. I feel like a classroom can have kind of a judgy environment based on, oh, am I raising my hand too much or am I asking that questions? You don't have that online. And I think the best online teacher I had was Sal Khan. He's done an incredible job at Khan Academy, and it genuinely feels like it's my uncle teaching me 'cause. The way he explains concepts, the way he like brings in emoji, the way his like voice is so soothing. I can see the effort he put into drawing the certain diagrams he does, bringing pictures, creating the quizzes. It just feels like I, I would just not want to cheat even if I can. And I think what people are more focusing on now is how do we lock down browsers? How do we create this secure testing environment compared with how do we make the student not want to cheat in the first place? [00:47:53] John Nash: Interestingly enough, the research also bears out, Advait, that the students who don't feel compelled to cheat have a sense of belonging to their learning experience and to their instructor. [00:48:05] Jason: Yeah. And as John said too, I think so much of this goes back to good design within the class, both designing the class itself, but you can design a class to be either more or less personable. Have more or less teacher presence. And we really encourage, at least in our, in my little world here, encourage to design a class that has a strong teacher presence and has a strong student to student presence. And so that they're connected with one another. They're connected with the teachers in ways that they feel both supported. They're responsive, they're communicative, they show up. And I think for me, this is where the next level like. I think whether or not I'm providing the content or the knowledge for my students is neither here nor there. But I think for me, and this is the feedback I've gotten from my students as well, when I really show up, they appreciate it. Like, this is what they appreciate online. And we've had really close connected, supportive groups where we've never, we don't even know what he, they don't even, they know what I look like 'cause I have videos, but I don't even know what they look like, right. But in those supportive environments, I think when they feel that, when they know the teacher is present in there, I think that's our next goal, I would agree. [00:49:22] Advait: What are your thoughts on an AI clone? If you just on the screen, a chatbot and people can talk to it. It has your face and it just, it has some instructions. I think that would be pretty interesting. I'm greeted by. You and you're in all the videos, so there's a sense of continuity. maybe you can animate the students' profiles as well. That would be pretty interesting. [00:49:45] Jason: Yeah. What I want is the AI clone that can put away the dishes, that can mow my lawn, that can do my taxes for me. Those are the things I actually want to connect with my students. Right. So, and that's where I would hate to take away too many opportunities to do that. I feel the same way about grading, actually auto grading. I think that there's a first pass grading that I'm not against. Like do I really need to be teaching another student about where to use a comma? No, I'm okay with, some sort of autobot teaching my students where to place their commas and how not to have a split infinitive, but I want to be connected to them because I, believe that's where the transformation happens. I'm still kind of a romantic when it comes to teaching. I believe that teachers can actually make a difference with students and not just giving them a head full of knowledge but actually seeing them develop in a way that their lives are better moving forward because of it. So, [00:50:44] John Nash: so Advait, I think the direction of this conversation we've just had over the last hour going to surprise everyone that listens to it when they take the click bait and say, "Oh. Jason and John talked to the Einstein guy," and this has been a really rich conversation about the future of ed. And is it because you really decided to launch a site that would punk institutions? [00:51:13] Advait: Like it. And actually, my goal is to start a school. I want to start a school someday inspired by the Greeks. Very small community, maybe 10 people, maybe 20. And maybe other people can take inspiration in the rest of the world to create their own communities fully run by AI and autonomous. And people can just explore whatever subject they want out of pure curiosity’s sake. and I'd be very interested in seeing the learning outcomes from that compared with like an actual educational institution. [00:51:46] Jason: I think that sounds wonderful. And I think we need more of them. That's my opinion. Even though there's a way in which everything that we do, like when we're at these individual institutions, like John and I are, there's one way in which our institutions are in competition, right? We are in the same area; the country we compete for online students. In many ways. But at the end of the day, for us that are educators we just believe that we just need more education. We need more people doing the good work and trying new things and getting out there to, to bridge those gaps and to help, especially people that, that don't have access to it right now. And that access might be in different ways. Like it might not just be the hoops to get into a traditional university, but as we've talked about here, the hoops of learning in what seems to be a traditional way versus really capitalizing in the best way on just somebody's natural curiosity and just what can we do? Like I think it was Plutarch that said, what could we do to do the real job of an educator, which is to light a fire under somebody, right? And just to make that happen. You could use that too. Plutarch. I don't think Plutarch is trademarked, so maybe that could be your next. Uh, I would check though first just to be sure, but he is back far enough that I don't think that they it was pre corporations and stuff, so I think you'd probably be okay. [00:53:12] Advait: No, I feel like I'm going to have to step back from that specific field. 'cause I've done my job. My goal wasn't to create a cheating tool. I like to say Einstein was a thought experiment except [00:53:24] Jason: Well, you made it. [00:53:25] John Nash: Eight days in, so let's pretend it's early days, but Einstein was a thought experiment. were the results of the experiment? [00:53:35] Advait: I actually read a Substack and the person, I'm not sure of the name, but I'd have to find it, but the person was highlighting how there was a deficiency in AI knowledge among the general population. And how it created the knowledge, which in turn will create meetings, will, which in turn will create discussions and hopefully change. So, I, I do think people are taking this more seriously and they will realize that I'm not here to decide what happens, but my goal was to just show that this is possible and the people in power can decide what to do with it. And my, my email was full of deans and professors from different universities. It, it was either shame on you, take it down, or we're going to change your policies. So, hope that more people figure out how to deal with technology right now and yeah, and make the best of it because we have these AI CEOs saying, we're going to automate end-to-end software engineering. We're going to automate end-to-end jobs. We're going to automate end-to-end X, y, z. And that includes students as well, because students are going to get those jobs and they will be automated and the teachers need to rage at the automations instead of the student's job being automated or figure out a way to empower people with the automations. I think that is the best outcome. [00:54:59] Jason: Yeah. [00:54:59] John Nash: you. [00:55:00] Jason: good. Yeah. cause I saw that Substack. Maybe Michael Wagner. Does that ring a bell where he was talking about it? Yeah. [00:55:08] Advait: There, there were a few of them and I made sure to read every single one and I just wanted to understand it. Change is happening, so that's great. [00:55:16] Jason: yeah. Yeah. That's wonderful. Well, that's, I, that's really impressive. And you're an impressive person. I think that you'll go on to do great things would love to hear. We'll have you back when you start your school. The Plutarch School of AI Understanding and Knowledge into the New Millennial. And [00:55:38] Advait: my new school, so [00:55:39] Jason: yeah, there you go. Yep. Yeah, that's what I'm here to do, you know. [00:55:42] John Nash: Happy to license that to you until we send you a cease and desist. [00:55:45] Jason: That's right. That's right. You'll be hearing from my lawyers after this podcast. Sorry. Not even funny. But thank you so much for coming to talk to us. I think this has been great both for us as a community to be talking about this and to kind of give us a little push to be talking about this. But the fact that you're able to come here and kind of give a little more background, I think will be really helpful for our listeners at least. And it's certainly been helpful for us today. So, thank you. [00:56:11] John Nash: Yeah. Thank you. [00:56:12] Advait: I'd also like to thank you for being so curious. 'Cause I think that's such an important skill that most people don't have it, it's like an immediate rejection, whereas you guys are embracing it, you're bringing on speakers. So, thank you for spreading that. [00:56:27] Jason: Well, sorry for my harsh words on the front end, and thank you for still being willing to talk to me. [00:56:34] Advait: I've had enough of that. So [00:56:36] Jason: Yeah. [00:56:36] Advait: I feel like I, I can take anything at this point. [00:56:39] Jason: Yeah. Well, I'll be more thoughtful next time. I'll seek to understand first, next time. So. [00:56:45] John Nash: Advait, if people want to reach out to you, and chat further what's a good way to get in touch with you? [00:56:52] Advait: Either my LinkedIn, Advait Paliwal, or my email Advait at companion.ai [http://companion.ai]. Both would be great. The spelling is A-D-V-A-I-T@companion.ai [A-D-V-A-I-T@companion.ai]. That's my business email. You can reach out to me, happy to have discussions, happy to help understand how the technology works, and maybe even help implement some. And it doesn't have to be through the company. I just want to spread the knowledge and help people so it, it can just be out of pure. It I just want to help. So just pure goodness. [00:57:22] John Nash: That's wonderful. And you've already got my brain running. I've been bugging Jason to give me a quick tutorial on how to get my GitHub going, but I think I'm just going to skip that and have you spin me up a computer in somewhere in the cloud. And so, [00:57:37] Advait: I can actually do that so we can chat off podcasts and let's do it. [00:57:42] Jason: That sounds good. [00:57:43] Advait: By the end of this, you'll have a beautiful personal website, an app of yours and even some courses that you just made possible. [00:57:50] John Nash: Heaven knows what I'm going to make now. Yeah. Okay. [00:57:55] Jason: That was wonderful. Well, thanks so much and for those people listening, our website is online learning podcast.com [http://podcast.com]. We'll put the show notes in there, transcript as well as links to Advait and any of the resources that we talked about this time around. So that you can check those out. Obviously, we're always interested in your feedback and so reach out to us on LinkedIn. You can find John and I there and our links to our LinkedIn are right at the top as well of our notes. Great to meet you. Thank you so much for jumping on. [00:58:22] John Nash: To meet you. Thank you. [00:58:24] Jason: we'll probably roll this out fairly quickly just because people are curious and I think it would be helpful while people are thinking about it, so, [00:58:31] John Nash: Cool. All right. Take [00:58:33] Jason: Okay. [00:58:34] Advait: time. [00:58:34] Jason: Stay in touch. [00:58:35] John Nash: Yep.

EP 41 - Moving Beyond Copy-Paste AI Syllabus Policies with John Nash & Jason Johnston

In EP 41, John and Jason discuss the evolving challenge of moving beyond 'copy-paste' AI policies to create syllabus guidelines that encourage students to engage in the 'productive struggle' of learning. See complete notes and transcripts at www.onlinelearningpodcast.com [http://www.onlinelearningpodcast.com/] Join Our LinkedIn Group - *Online Learning Podcast [https://www.linkedin.com/groups/14199494/] (Also feel free to connect with John [https://www.linkedin.com/in/jnash/] and Jason [https://www.linkedin.com/in/jasonpauljohnston/] at LinkedIn too) Host Bios: Walk into schools today and generative AI is on the agenda—and many leaders aren’t sure what to do with it. John Nash helps them figure it out. An associate professor at the University of Kentucky and founding director of the Laboratory on Design Thinking, he makes AI practical and useful, not just theoretical. He’s on two generative AI advisory boards at the University of Kentucky and one at MidPacific Institute in Honolulu, advising educators from local superintendents to teachers in international schools. He teaches courses in design thinking, leading deeper learning, and mixed methods research, and his research interests study the application of human-centered design in organizational leadership. Jason Johnston is the Executive Director of Online Learning & Course Production in Digital Learning [https://digitallearning.utk.edu/] at the University of Tennessee, Knoxville. His background includes developing and launching online degree programs, directing educational technology, teaching, and working as an audio engineer. Holding a PhD in Educational Leadership, an M.Ed. in Educational Technology, and an M.Div., Jason advocates for humanity and equity in online education while helping educators leverage technology for the future. He co-hosts the podcast Online Learning in the Second Half (www.onlinelearningpodcast.com [https://www.onlinelearningpodcast.com/]) and enjoys playing guitar, building Lego, and traveling with his family. Resources: * University of Kentucky Syllabus Policy: https://celt.uky.edu/ai-course-policy-examples [https://celt.uky.edu/ai-course-policy-examples] * University of Tennessee, Knoxville Syllabus Policy: https://writingcenter.utk.edu/sample-syllabus-statements-for-ai-guidelines/ [https://writingcenter.utk.edu/sample-syllabus-statements-for-ai-guidelines/] * Jason’s Policy Icons: https://docs.google.com/document/d/1MG9h68__uqPSz6HXNeVymJhal1VNapjyK-2PFa5QFxI/edit?usp=sharing [https://docs.google.com/document/d/1MG9h68__uqPSz6HXNeVymJhal1VNapjyK-2PFa5QFxI/edit?usp=sharing] * John’s Policy Example: https://johnnash.notion.site/John-Nash-s-Stance-on-Generative-AI-Use-by-Students-in-Courses-2eff24fd17cc8043ae2be34712680c28 [https://johnnash.notion.site/John-Nash-s-Stance-on-Generative-AI-Use-by-Students-in-Courses-2eff24fd17cc8043ae2be34712680c28] * Chronicle article by Geoff Watkinson “I’m an AI Power User. It Has No Place in the Classroom. Learning to think for yourself has to come first.“: https://www.chronicle.com/article/im-an-ai-power-user-it-has-no-place-in-the-classroom [https://www.chronicle.com/article/im-an-ai-power-user-it-has-no-place-in-the-classroom] (paywalled - should be able to read for free with login) Theme Music: Pumped by RoccoW [https://freemusicarchive.org/music/RoccoW/contact] is licensed under an Attribution-NonCommercial License [https://creativecommons.org/licenses/by-nc/4.0/]. Battle Hymn of the Republic is public domain from the Library of Congress https://www.loc.gov/item/jukebox-767050/ [https://www.loc.gov/item/jukebox-767050/] Transcript We use a combination of computer-generated transcriptions and human editing. Please check with the recorded file before quoting anything. Please check with us if you have any questions or can help with any corrections! [00:00:00] Jason: Can we do the quick intro? [00:00:02] John Nash: Yeah, hold on. [00:00:03] Jason: That was the intro to your other podcast. [00:00:06] John Nash: Yeah, [00:00:06] Jason: John, have you [00:00:07] John Nash: exactly. [00:00:08] Jason: Beyond My Back? [00:00:10] John Nash: No, I am not podcasting. Behind your back. I'm John Nash here with Jason Johnston. [00:00:15] Jason: John. Hey everyone. And this is Online Learning in the second half the [00:00:19] John Nash: I. [00:00:19] Jason: Learning podcast. Mm-hmm. [00:00:20] John Nash: Yeah, we're doing this podcast to let you in on a conversation we've been having for the last three years about online education. Look, online learning has had its chance to be great, and some of it is, but a lot still has a way to go. How are we going to get to the next stage, Jason? [00:00:35] Jason: is a great question. How about we do a podcast and talk about it? [00:00:39] John Nash: Perfect. What do you want to talk about today? [00:00:41] Jason: You know, you always ask me that question and I really appreciate it. But what do you want to talk about today, John? [00:00:47] John Nash: Oh, you know what I want to talk about today? I want to talk about the struggle that instructors are having to set guidelines for the use of generative AI in their classes. [00:00:57] Jason: I think that sounds like a great conversation, especially the front end of a semester here. [00:01:02] John Nash: Yeah, is it the lawyers that say, or the justices that say, "this is not settled law?" [00:01:07] Jason: Hmm. [00:01:07] John Nash: This is definitely not settled law. We, we are not lawyers. We do not play them on podcasts. We are just a couple of, a couple of folks that are trying to think this through. So, Jason, we just came off of a really cool episode with Megan Haselschwerdt at University of Tennessee, one of your colleagues, who engaged your office to think about ways to deal with how her students were using generative AI in her class. And that's really made me think a lot about how a lot of us are wondering how we can have guidelines that work in our classes where it really doesn't matter what it is we're teaching. I think it'd be good if we could talk about that. I've had some really big evolving thoughts around my own stance on this. Even after three years in, I think I've finally written down something I can live with. But there's a lot of options out there for faculty and teachers and instructors. There are stoplight protocols. There are guidelines out there that universities have put out that faculty can adopt just right out of the box. But do they really fit? I talk with my colleagues at my work and they're sort of saying, sometimes "just tell me what I should put in my syllabus" and I'm replying with things like, "well, it's hard to say that this will fit your syllabus because really it's an ethical conversation you need to have with yourself and your students." And so, I think it'd be really good for us to sort of lay out what we're doing, what we're hearing, and get people to give us feedback on what they're doing and even call into question our own stances. [00:02:43] Jason: Yeah, at the end of the podcast with Megan, we're kind of asking for advice for faculty. One of the things she talked about was that she wished she had a little more coaching before she started her class when she was designing her class. Wish she had a little more coaching, a little more time to talk about it. And I think that makes complete sense. I think what we're seeing is that, without a lot of time spent on it, I think faculty are, are kind of defaulting to the two extremes. Either they're not saying anything about it, maybe just letting anything happen with regards to ai, or they're putting in a really quick statement, or maybe they say something in their first class how they don't allow any AI use whatsoever. I think Megan's story, and this is a spoiler about how good it is, but I think it was just a great story to talk about how faculty might be able to kind of wrestle with that and figure out where there might be a middle ground that would actually increase learning for the students and engagement, but also potentially decrease the actual use of AI. [00:03:51] John Nash: Yeah, maybe talk a little bit about what the overarching approaches are at University of Tennessee Knoxville, and I can say what has been promulgated here at the University of Kentucky. I think they're a little bit similar and just sort of, there are some I call them stoplight protocols. You get a red, yellow, or green kind of approach that a faculty can copy and paste and drop into their syllabus. And can you also talk a little bit about what the pros and cons are of that copy paste approach without maybe thinking about what the actual work is that students might do in your class. [00:04:27] Jason: Yeah, we a similar thing it sounds like to UK, which the provost office has the kind of the, the, the strict no AI, the, know. AI freedom kind of thing, and then a moderate approach. And we can put the links into both of those. Both at UK and University of Tennessee, Knoxville. And, and so that's, so that's, essentially the guidance we've been giving for syllabus. And I think people do copy and paste those in. the kind of what our approach as we're helping. Faculty design online courses have been a little bit more customized and to spend some more time in that messy middle of "moderate," because when it comes to "moderate," it does take, I think, more intentionality and more communication and more thought you approach it. And probably more specific. Policies that would apply to certain assignments. And so there may be one assignment that has a different kind of moderation another assignment. So, for instance, in my class the, I teach a couple of online classes a year for UT and in my human computer interaction class in the fall. In their reflection pages, I ask for no AI use whatsoever. So, I have a strict policy when it comes to my reflection because, up to a level anyways, I want to see the mistakes, I want to hear their thinking. I want to kind of walk with them on, on what they're, even if it's some rambling. But., on a lot of the other projects I have, have more of kind of a human first, human last kind of approach, or we've called an AI human sandwich with AI in the middle to help called human in the loop kind of approach. Most of my assignments are that way and I just. Really want more transparency and I'm able to articulate that and I'll put in the link as well. You're free to take a look at and use something we use. I try to communicate with icons as well to try to communicate what that process would look [00:06:37] John Nash: Right. [00:06:37] Jason: So, it might look like it might look like a, a human then an AI or just a human and AI symbol, and then a human again, or it might just be human AI. And so, we expect you to do most of the work with a human first, and then you can use AI to kind of clean up at the end. Or even have an assignment that I ask them to use AI to come up with the initial idea, Right? So, AI first [00:07:02] John Nash: right. [00:07:03] Jason: but then human last [00:07:04] John Nash: Yeah. [00:07:05] Jason: to evaluate and critique it. [00:07:07] John Nash: And those icons, you can put those in the syllabus by each assignment, so at a glance, they know in advance like, oh, okay. Yeah. [00:07:13] Jason: Yeah, without even reading. If they get it from the beginning, and I put the examples in [00:07:17] John Nash: Mm-hmm. [00:07:18] Jason: and then in the actual Canvas assignments, they can see at a glance what my expectations are. You know, I think a lot of policy is about communication, right? And trying to communicate what your expectations are. And I think if students are left without that communication, then they'll, they will do whatever they want to do, which is, is, I think, completely fair. [00:07:41] John Nash: Yeah, yeah, yeah. Our stance at the University of Kentucky is similar, and we have a few sites where faculty could consult to get guidance on what to put in their syllabi. One is a group that was convened by the provost called the Advance Group, and I happened to sit on that, we recommended that course policies exhibit four characteristics, that the AI policy is people-centered, is adaptable to the circumstances of the course, is thinking about the effectiveness of what the policy is trying to do, and then trying to keep awareness upfront for the learners too. And so, our Center for Enhancement of Learning and Teaching called CELT, they have a, a, a site where they've got real boilerplate. So, no use, conditional use and unrestricted use. And with some examples. I think, you know, it's interesting as we listen to Megan's episode, if even with no use students will use AI if you have a "no use" policy. And so, I'm wondering if you've got a rationale that is very simple that you say, "well, idea generation and analytical thinking and critical analysis are key outcomes in this course," and I'm reading verbatim from an example, from our university site. And so "as a result, all assignments should be submitted by the student, a hundred percent original work." Great. I think you can say that all day long and students will still jump into ChatGPT. And so, this has been the challenge for me is that you can put these in, but there's another level that has to kind of happen in the actual human interaction in the course to get students to hopefully adhere to , the rationale and interest you have for why you don't want them to use ChatGPT. Isn't that the case? [00:09:29] Jason: I think it is, and I think it's fair that students know why. Right. [00:09:33] John Nash: Mm-hmm. [00:09:34] Jason: A lot of students are coming at these classes with more understanding than teachers have about AI and how it's used and how they use it in a daily basis. Right? And so, I think it's, I think it's really fair to do that. [00:09:47] John Nash: I think it's important that we have these templates for faculty to use. I think that they're just the tip of the iceberg and the kinds of capacity building we need to still do to help faculty understand what happens when their assignments collide with students' desires to use AI in spite of whatever's written in the syllabus. [00:10:10] Jason: Right. Well, and I had an example recently too on a dissertation committee. The, the topic is about AI and instructional design, and we didn't have a policy about dissertations and so, [00:10:25] John Nash: Yeah. [00:10:25] Jason: these cookie cutter syllabus ideas about AI aren't going to fully explain how we make this a rigorous experience for the student, and how we can, with assurance as a committee, sign off on it and say, this is the student's work, right? How do we foster that transparency? But also recognizes 2026, you know, do we, we, we don't expect them to go back to the card catalog anymore, do we? And start writing down all their references and checking out books and putting them on the photocopier. [00:11:05] John Nash: Yes, [00:11:06] Jason: You know, there's lots of technologies, that we have adopted in the last 30, 40 years, and we need to be, adaptive thinking about this. At the same time, I think it serves the student well to communicate well. They know the expectations and we don't get any of us into a spot at the end, and so we were able to develop something that that seems to work in that regard. [00:11:32] John Nash: I think a lot of our listeners are lecturers, instructors, professors, instructional designers and you mentioned sort of the dissertation work, so let's just take a second and talk about that weird special stripe of instruction, which is the dissertation advising. We're talking about that now in our department. We're a chiefly graduate department. We have a lot of dissertating students, and we're wondering now, what is the stance that dissertation advisors should take. It's different from having a, there's no syllabus for the dissertation advising process. It's a six month to two-year mentoring process where the student is expected to engage in independent research to demonstrate that they can be a scholar and execute a study. And, boy, can AI creep in just about anywhere and how shall advisors talk to their mentees about the use of AI? Should there be a department policy? I'm not sure there should be. Or there could be, but I'm not sure there can be because every dissertation is different and every dissertation advisor's ethical stance is going to be different. We're a small department, seven or eight faculty. Very different attitudes towards how each believes students may use generative AI in their dissertation writing process. Some are quite liberal with it, some are very scared by it, and I think it's been difficult to really get down to the fact that it's a personal mentor-mentee discussion and decision about where each stand. It's a little different from just having a syllabus for an undergraduate design course or a writing, course. It's, yeah. What do you think are, am I going too far here or is this a special strip? [00:13:16] Jason: I don't think you are. [00:13:17] John Nash: Yeah. [00:13:17] Jason: I think about how really every classroom, every teacher, student connection and relationship is built on trust, at least on some level. I think that dissertation is probably 10X what you would find in, in need in a master's class, even in independent learning, right? In terms of [00:13:38] John Nash: Yeah. [00:13:38] Jason: building trust, and I think that's a lot of it is trust, transparency. It needs to be there because. If the trust starts breaking down one side or the other, in terms of how this is being produced, and this is, [00:13:51] John Nash: Yes. [00:13:52] Jason: Pre AI as well as we know, right? That the hardest dissertation experiences have been when that trust has been eroded, especially between the chair and the, and the student. And so, this is nothing new, but I think AI just brings in another, little possible foil to that relationship and something that on the front end we need to communicate, we need to put on the table and talk about, and also not just come down with some sort of edict from on high. This is the way it'll be. But let's figure out what this looks like between you and I, the, our, our own relationships with technology. The content that we're trying to make, the kind of analysis that we're doing. You want to create new knowledge? Well, in some, cases you're going to have to use AI in order to take this a further step than, what the last person that studied this was doing. [00:14:48] John Nash (2): Yeah. And the other wrinkle that comes into this is that the, the generative ai, while it can generate text, that can be technically accurate in particularly in study design and in thinking about framing literature, the dissertation process is not just the end written product, which everybody's very familiar with. You go to the library, you see the printed dissertation, but to get there, that student must orally defend their ideas in front of a committee. So, dissertations, if you're not in this world day to day, are challenging and difficult, tricky political and scholarly activities because the advisor's reputation is at stake with the rest of the committee members. The student's reputation is at stake with their committee members and how they talk about their ideas. If the ideas are not flowing well from the student in a defense, then the other committee members can wonder if the chair of the dissertation was doing their job. It's fraught with all kinds of pitfalls, and I think with generative AI in play now, I'll speak only for myself as a dissertation advisor going forward, I'm going to be thinking about more mini defenses. With me and my student, mini oral defenses to make sure that they can actually talk through what they're putting out there, do a public demonstration of their learning, because if I already know they're a good writer and I've read their writing and then they start using AI and I can't tell the difference and then I think they understand what they're talking about, but they don't. I'm in trouble. They're in trouble. [00:16:21] Jason: Yeah, I love that. You know, and it kind of points to things we've been talking about the last two years, how rethinking what pedagogy looks like in the classroom because of AI has actually [00:16:34] John Nash: Yes. [00:16:34] Jason: uncovered some of the cracks. I think there's lots of times where someone gets to a dissertation defense and they're not prepared. And it is partly their fault. And maybe mostly the student's fault, we might say this is their defense, right? They're the, they're supposed to be the experts, but it also leans on the, on the committee and the chair. [00:16:51] John Nash: Yeah. [00:16:52] Jason: And how much better it be to have those check-ins, to scaffold, not just the writing of the chapters, but to scaffold the defending of the chapters and the ideas and putting it to [00:17:07] John Nash: Yes. [00:17:07] Jason: it to the test in terms of speaking it out loud, with somebody who knows a little something too and can, can push back. [00:17:15] John Nash: So, I read something recently, Jason in the Chronicle of Higher Ed. It was just actually within a few days of us recording this piece here. It was on January 9th an essay by Geoff Watkinson, and the, the title really caught me. He said, " I am an AI power user. It has no place in the classroom." And this immediately struck me. I thought he was. Talking to me, I'm an AI power user. I'm using this tool nearly daily. I think you are too. And I've been concerned about what place it has in the classroom. I've talked before in our episodes and with others that I teach a design thinking course that I think is almost un-AIable. But I also teach dissertation writing courses that are AIable. And so, I'm, I'm thinking, wow, what did he have to say here? And he said, what I think I've been thinking is that he's been using generative AI since the very beginning, and he saw a change immediately in his own work. Tasks that took him a full day now took 30 minutes. He paid for a premium version of Claude, and he taught himself through an AI certification, how to use these tools. And his output increased and his quality increased. And for the tedious administrative work he was doing for writing proposals for a tech company it was great. And then he said, I'm able to do this because I have been an expert in this field for years, and the way I'm using AI is to advance work that I already know how to do. But this is a world apart from teaching 18-year-olds how to put their thoughts on paper and talks about how really, it's the productive struggle in teaching and learning. That's important that AI can eviscerate and therefore he's being very careful as a power user to be thoughtful about the way it comes into his classes and almost doesn't come in at all. And I thought this might be the way, this might be the way to think about this, but how do you frame it in a way that it works for you in almost every way you teach? And that was, that's what I've been struggling with. [00:19:24] Jason: That's really interesting and it points to this idea that is best used in the hands of experts. But the very [00:19:34] John Nash: Yeah. [00:19:35] Jason: reason you're in school is because you're not an expert. [00:19:37] John Nash: Right. And he's not worried about cheating. You're like, wait, what? No. He's worried about losing the moments of revelation and growth that a student gets when they do the cool productive struggle that they do [00:19:52] Jason: The aha moments, the light bulb moments [00:19:53] John Nash: mm-hmm. [00:19:54] Jason: which are [00:19:54] John Nash: Yeah. [00:19:55] Jason: those are teacher payback times too, right? To be able to see those things happen. I think for any of us that teach, those are the very reasons why we, we get into it. So, you would, yes. You would hate to lose those. [00:20:05] John Nash: Yeah, yeah, yeah. And so, I wrote down for the very first time late last week in three years of using generative AI a policy kind of guidelines that I think I can live with in almost all of my classes. It's built on what Watkinson said, and also from some folks that you've introduced me to through our podcast related to the ideas of feminist pedagogy and feminist pedagogical frameworks. And for those who are not familiar with that, it's really about strategies that support learners' goals by promoting learner-centered approaches presenting community-driven content, keeping the learning experiences as transformative as possible and close to the ground of the work that they're doing so that you can achieve the goals that you want to do. And by using some of those thoughts and from a book called "Feminist Pedagogy for Teaching Online", edited by one of our past guests, Enilda Romero-Hall, with also Jacquelyne Howard, Clare Daniel, Niya Bond, and Liv Newman. And by the way, we will be interviewing Jacquelyne Howard and Enilda Romero-Hall about this book. But these notions along with Watkinson ideas, bring me to this place where I have an ethical stance about why I think the productive struggle is important, and we'll talk together every time about whether AI is appropriate. And if I catch it getting used in a way that doesn't seem right, we're going to talk about that and help you get back to the productive struggle. I think that's kind of where my head is at now. [00:21:37] Jason: I love that. Can you give us a little sample from your syllabus? [00:21:41] John Nash: Yeah. [00:21:42] Jason: you do a dramatic reading of it or, [00:21:44] John Nash: no. Yes. Like yeah, dramatic reading. You pick the, pick the voice of the ones that I like to do, like to a Sean Connery voice, or no, I'm not going to do that. [00:21:52] Jason: Music. [00:21:53] John Nash: can do it to Barney, the Dinosaur. [00:21:55] Jason: Mm, Yeah, you could do that. [00:21:57] John Nash: Yeah. [00:21:58] Jason: I was of thinking, with an orchestra playing [00:22:01] John Nash: oh, right. Yeah. [00:22:02] Jason: behind you, kind of like escalating to the final moments of the syllabus. Sorry, go ahead. [00:22:08] John Nash: no, no. Thank you for the offer. I'll give [00:22:10] Jason: With all of that aside, what would you, just give us a little sample from it. [00:22:14] John: Yeah, I think it just, it does three things. It, it starts off by talking about how AI is powerful and it's powerful for work you've already mastered, and it's dangerous if it's a shortcut for work that you're learning. And, in it I confess that I've been using AI extensively in my professional work for tasks I already know how to do well and it makes me faster and better at that work. But in education and the work that we would do in a class, that requires struggle, cognitive work, messiness of not knowing, ambiguity, all the things. And AI can eliminate that struggle. And for me, and I say to the student, to you, that's what I want us to have. So, the Second section tells them what I'm asking of them, and I'm asking them some of the typical things you would expect to hear but maybe expand it upon a bit. But I want them to own their thinking. I want them to be transparent about the AI use. I want them to do the foundational work that needs to happen to have the productive struggle. And I want them to name it when it's not working. So, if they're struggling, I'm there. I'm not throwing them into the deep end of the pool without anything, I'm there. And so, my last section is a commitment to them. And by telling them I'm going to make my criteria explicit. I will show my own thinking and my uncertainty, and I'm going to acknowledge when things might not be working and how we can work together because it's really for us to do this together. And I think it kind of, maybe we take a page as I'm always talking about Michelle Miller's "same side pedagogy." I don't want to be adversarial here, but I want to also admit that you're here to learn something and I'm here to say, I think I can help you do that. [00:23:55] Jason: I love that. It really is an invitation into that messy middle of. Of the syllabus approach to AI in a sense, right? Because [00:24:07] John Nash: Mm. [00:24:08] Jason: You are welcoming the student then into that productive struggle, even through your approach to how AI may or may not be used. [00:24:16] John Nash: Yes, and so we'll see how it goes. I think that we as instructors need to be helpful to ourselves and helpful to our students by saying "we're in this together, and I think I have some guidance that can help you get to a goal you're hoping to reach. And if you take a shortcut and I notice it, I mean I can, I can look squinty-eyed at what you turned in, and I can tell that you probably skipped some steps and used AI. I'm going to politely call you out on that and tell you, maybe you should do this again, because I want you to really learn this stuff." And I think that's where we've got to get. [00:24:53] Jason: Yeah. Yeah, I love that. Would you share that perhaps with our listeners? Can [00:24:58] John Nash: Yeah, absolutely. [00:25:00] Jason: you share a copy of that. We'll put it into the resources as well. [00:25:02] John (2): And I invite anyone listening to this and reading it to push back on it. Poke holes in it. Tell me how we can make this better. I mean, maybe I'm being pollyannish here and hopeful, but I think it's, I don't know. We'll see. [00:25:12] Jason: Yeah. No, I love it. [00:25:13] John Nash: I. [00:25:13] Jason: John for that. And yeah, this has been a great little conversation about AI and syllabus policies. I think it's really helpful at the front of a semester to talk about these things. I think we need to have more conversations and so we welcome conversations from you all. We're on LinkedIn. We'll put at our website online learning. podcast.com [http://podcast.com]. You'll see all the notes for this show, as well as places that you can reach us at LinkedIn. to have more conversation about this, and if there's any way we [00:25:42] John Nash: Yes. [00:25:42] Jason: you either through ourselves or through these resources, then yeah, please reach out. [00:25:49] John Nash: Yeah. If you're a first-time listener, please hit follow and subscribe to our podcast. You'll get this in your feed. If you like what you're hearing, you'll, you'll get more. We've got some good guests coming up. [00:25:59] Jason: Oh, man we've got some great, I'm really excited to release our upcoming episodes. They just keep getting better and better and a lot of themes that are happening this year as expected, I guess, around wrestling with policies and use and technology use. Of course, these are our ongoing themes, but yeah, we've just got some great podcasts coming. [00:26:18] John Nash: Good talking to you, Jason. [00:26:19] Jason: Good talking to you. Good talking to you, John. And with that, we will leave you listeners with a dramatic reading of a selection of John's AI syllabus. [00:26:32] John: AI is powerful and it's powerful for work you've already mastered, and it's dangerous if it's a shortcut for work that you're learning. I confess that I've been using AI extensively in my professional work for tasks I already know how to do well and it makes me faster and better at that work. But in education and the work that we would do in a class, that requires struggle, cognitive work, messiness of not knowing, ambiguity, all the things. And AI can eliminate that struggle. And I say to the student, to you, that's what I want us to have. and I'm going to acknowledge when things might not be working and how we can work together because it's really for us to do this together. I don't want to be adversarial here, but I want to also admit that you're here to learn something and I'm here to say, I think I can help you do that.

EP 40 - Does Allowing AI Reduce AI? A Surprising Finding with Dr. Megan Haselschwerdt

EP 40 - Does Allowing AI Reduce AI? A Surprising Finding with Dr. Megan Haselschwerdt In EP 40, John and Jason talk with Megan Haselschwerdt about her transformative semester moving from a futile "cat-and-mouse" game of AI detection to a trust-based partnership with students, demonstrating how transparent dialogue, a more open policy, and addressing "insurmountable" assignment loads are far more effective than policing. See complete notes and transcripts at www.onlinelearningpodcast.com [http://www.onlinelearningpodcast.com/] Join Our LinkedIn Group - *Online Learning Podcast [https://www.linkedin.com/groups/14199494/] (Also feel free to connect with John [https://www.linkedin.com/in/jnash/] and Jason [https://www.linkedin.com/in/jasonpauljohnston/] at LinkedIn too)* Guest Bio: Megan Haselschwerdt, Ph.D., serves as an Associate Professor and HDFS Graduate Program Director in Human Development and Family Science at the University of Tennessee, Knoxville. She earned her Ph.D. and M.S. in Human and Community Development from the University of Illinois-Urbana Champaign and a B.S. in Psychology from Indiana University-Bloomington. As an interpersonal violence and family science scholar, Dr. Haselschwerdt’s research focuses on intimate partner violence (IPV) from the perspectives of victimized adults, young adults with childhood exposure, and support professionals. Specializing in qualitative methodologies like grounded theory and reflexive thematic analysis, she also collaborates on mixed-methods studies to examine help-seeking behaviors and develop trauma-informed interventions. She currently directs the Family Violence Across the Lifespan research team, leading initiatives such as the REVEAL Project and the Young Adults Live and Learn Project to promote safety, healing, and justice. Resources: * Dr. Haselschwerdt’s Scholarship: https://scholar.google.com/citations?hl=en&user=hTSsBcQAAAAJ&inst=9897619243961157265 [https://scholar.google.com/citations?hl=en&user=hTSsBcQAAAAJ&inst=9897619243961157265] * Megan’s AI Use Policies: https://docs.google.com/document/d/1I2THuGIaKYstGyZylQS4FgvbNV0B1j3Q3hEpGI4K_l4/edit?usp=sharing [https://docs.google.com/document/d/1I2THuGIaKYstGyZylQS4FgvbNV0B1j3Q3hEpGI4K_l4/edit?usp=sharing] * Jason’s AI Policy (and free / open source icons for communication) https://docs.google.com/document/d/1MG9h68__uqPSz6HXNeVymJhal1VNapjyK-2PFa5QFxI/edit?usp=sharing [https://docs.google.com/document/d/1MG9h68__uqPSz6HXNeVymJhal1VNapjyK-2PFa5QFxI/edit?usp=sharing] Theme Music: Pumped by RoccoW [https://freemusicarchive.org/music/RoccoW/contact] is licensed under an Attribution-NonCommercial License [https://creativecommons.org/licenses/by-nc/4.0/]. Transcript We use a combination of computer-generated transcriptions and human editing. Please check with the recorded file before quoting anything. Please check with us if you have any questions or can help with any corrections!   [00:00:00] Jason: I'm going to butcher your last name probably [00:00:02] Megan: Oh yeah, that's okay. We do too. [00:00:03] Jason: us with that [00:00:05] Megan: Yeah. So, we say it all very differently in our family, so it's totally fine. I, I say Haselschwerdt as though there's a z Yeah. Other people, other family members say hassle, but yeah, I say it as though there's a z Hazel Schwart. [00:00:16] John Nash: Okay, cool. I was, [00:00:18] Jason: Hazel Schwart [00:00:20] Megan: Mm-hmm. [00:00:21] John Nash: I took German for years and so I love the sound of German, so I Yeah. I said, [00:00:27] Megan: Yes. Yeah, we, we. [00:00:29] John Nash: be totally German of [00:00:30] Megan: We offend Germans when I say, like, what, how we say our last name? Yeah. It's unrecognizable. Yep. [00:00:36] John Nash: Nice shirt. Yeah. Okay, cool. [00:00:40] Jason: Haz-el-Schwart [00:00:41] John Nash: Haselschwerdt, [00:00:42] Jason: You [00:00:42] Megan: Yep. That's totally fine. [00:00:44] John Nash: Okay. Haselschwerdt. [00:00:45] Megan: I think Jo, uh, John has it the authentic way and Jason's American butchered from Ellis Island Way works just well too. [00:00:53] John Nash: I can't, yeah, I can't help myself though. So [00:00:55] Jason: I can't help myself either [00:00:57] Megan: ha ha... Intro [00:00:58] John Nash: I'm John Nash here with Jason Johnston. [00:01:01] Jason: Hey John Hey everyone. And this is Online Learning the second half the Online Learning Podcast [00:01:06] John Nash: Yeah, we're doing this podcast to let you in on a conversation we've been having for the last three years about online education. Look, online learning has had its chance to be great, and some of it is, but a lot of it still has a ways to go. How are we going to get to the next stage? Jason? [00:01:22] Jason: That's a great question How about we do a podcast and talk about it [00:01:26] John Nash: love that idea. What do you want to talk about today? I. [00:01:29] Jason: today I don't know how many times I can do this joke but how about we talk a little bit about AI and education [00:01:34] John Nash: Wait, is that a thing now? [00:01:36] Jason: Yeah [00:01:37] John Nash: I'm willing to try. [00:01:38] Jason: Yeah, I don't know much about it seems like everything I learn about it then I unlearn about it as well. But I'm really interested today We've got a colleague here one of my colleagues from University of Tennessee Knoxville with us Megan and we are just talking about me butchering her last name And so I'm just going to let her introduce herself Megan [00:01:58] Megan: Hello, I'm Megan Haselschwerdt [00:02:00] Jason: Welcome to the podcast It's so great to have you here and tell us a little bit about what do you teach what do you do at University of Tennessee Knoxville [00:02:08] Megan: So, I'm an associate professor in the Department of Counseling, Human Development and Family Science. I'm in the Human Development and Family Science side of things, and I'm the Director of Graduate Studies. Outside of teaching, I'm an intimate partner violence researcher, family violence researcher. But in the context of teaching at the undergraduate level, I teach HDFS 385, which is Child and Family Diversity. I teach this course at the graduate level, and I also teach some qualitative research methods, family theory, that sort of thing. But for undergrads I've been teaching Child and Family Diversity, fully in person, hybrid, kind of split between in person and asynchronous online. this semester is our first time with the fully launched distance ed online course. [00:02:50] Jason: That's great we really got talking because of really my role at University of Tennessee which is in more of a centralized office And I was included in a conversation where we started talking a little bit about AI the classroom And first could you give a little bit of a context about how long the course that we're talking about and then how long you had been teaching that course. [00:03:16] Megan: Absolutely. So, I've taught this class since fall of 2020. And so, it's had the, I can't even keep track of the number of modalities I've taught it in since 2020, which I think some people don't realize if they haven't taught between multiple modalities, that really, you're ending up creating a new class multiple time [00:03:35] Jason: Yeah [00:03:36] Megan: Things don't translate. And I, that's part of the learning this semester is what translates well from even hybrid into fully online asynchronous. So, this course is a semester long class. It's a three-credit hour course. And this is my first time teaching it this fall 2025 in this current form. it's mostly traditional brick and mortar students. It's our new distance ed program. So only five of about our 85 are fully online. The other 75 or so are, traditional students who either opted into the online version or were registered for an in-person version that we ended up canceling and merging into my course. [00:04:16] Jason: Interesting. I wonder about how many of our classes and programs will shift into more that digital doesn't necessarily mean distance [00:04:25] Megan: yeah. [00:04:26] Jason: know that as students are taking these online that as we flex modalities you talked about the modality shift of your classes. Thinking about the number of modalities that they have taken to go through an undergrad Then potentially a graduate program [00:04:41] Megan: And it's been interesting, our students, we collected data, coming out of, the early parts of the pandemic at least. And what we found is that students really wanted more in-person classes. they said they felt frustrated that they'd had mostly online courses. And so, we tried to make sure we were always offering a good balance. But when we offer an online and an in-person in the same semester, the in-person course is under-enrolled. We think that maybe this is specific more to juniors and seniors who have established their social networks, they have work, they have volunteering, they have just a lot of complicated demands, and so maybe for them online is more appealing. But we've also heard from students how difficult it is to, if you have an online class and an in-person, like racing out of the in-person to find a place where they can do the online, especially if it's a synchronous online class. So, the data that we get does not align with students' registration decisions, and that's made things a little complicated. [00:05:36] Jason: Yeah, so getting into this class, I think that's the wonderful context and I love the fact that your program is thinking about this and being thoughtful about the student experience. Thinking about this class you've been teaching it for five years or so tell us I really want to get into the story of this and [00:05:52] Megan: Yeah. [00:05:53] Jason: what has happened this semester Tell us a little bit about why you reached out to our department initially. [00:06:00] Megan: So, when I was building this class, so I've historically always taken you know, University of Tennessee provides us with an AI kind of general policy syllabus statement of, you know, open access, moderate use AI or like no AI. And I've always historically done the moderate use. However, I have not taught undergrads in the AI era until this semester. I'd only taught graduate students. And so, this semester went into it with my syllabus, decided because it's fully an online, the smartest thing I could do just my own thinking, not from literature was I should say no AI use because I really want this class to be reflective. I want it to be drawing upon their personal lived experiences. I want it to be. And so I went back on a policy, like kind of how I'd approach things in the past. And so there was a blanket no AI use. We made clear as best as we could through our syllabus, through our videos, through all the onboarding that we're not grading for grammar. We're not grading for your opinion. We're not grading for you to agree with the content. It doesn't need to be polished. And I think we naively thought that through these engagements and how we wrote about it and how we talked about it, that students would feel really comfortable trusting that we were really comfortable with the unpolished version of their written selves. And so that's where we started. I have a 20 hour a week graduate teaching assistant who was invaluable to this class. And in the beginning, we both were grading and taking note of things. because I had taught this class hybrid in fall of 2023, before AI really became, like in our class, we, I think it, we hadn't seen as much in our undergrad class of around how AI was going to be utilized. I have all these examples of how students responded to the same kind of prompt. And so, in this class, we do a premodule reflection just to kind of get them, it's supposed to be like a 10-minute brain dump. We make; we try to make it as low stakes. We suggest you set a timer for 10 minutes and you just kind of brain dump on the topic. So, if our topic is going to be around foundations like, kind of like white supremacy, the foundations of like, on which the country was founded. We have a prompt that's like ", tell us the story that you remember learning of the founding of the United States focusing on pilgrims, Indians. Like, kind of like the, the prompt kind of reckons back to like, what were you taught about this in elementary school? And so, we're used to getting very specific, like I remember this time. Mr. Johnson's second grade class, you could tell just like real examples from their childhood. But what we noticed early on, this was like week two or week three of the class that we were getting like almost the exact same thing. And told in the same style and told with the same phrasing and using words that we hadn't put in the prompt, but like we're very generic and just, and you know, if you saw one or two, that's one thing, but all of a sudden we're like at 60, 65 of the 80 or you know, and so then we went down this rabbit hole using AI ourselves to see like to what extent does this seem like we, is it, how is, how likely is it that everybody just has the same sort of thing to say? Then I went back to my, you know, the 2023 responses and realized no one of that year said anything like these. So, we watched this for a little bit, and then the final straw I think was at the end of that week. It was like, if there's a quote from one of the documentaries, you know, like "America is sold as this melting pot, but like, is it really?" And so, I'm like, so if America's not a melting pot, like. What is it? And it was like one third --the TA started calculating, right? So now we are spending ample amount of time trying to figure out how much of a problem this is. It was like one third of them described it as "a salad plate" or "a salad bowl" in quotes. And I never had that response. And it was all of, and it was like in quotes, and it was just it, you know? Then I went back to 2021. Nobody else talked about a salad plate or a salad bowl, so that's when we started to feel like, honestly, like in our, like we felt like hostile towards the students. Like we felt just because also with building an online class. I don't think I understood, like truly, and you know, we've done survival online teaching during the pandemic, but having gone through Jumpstart, I don't think I, you know, it is so important and so kind time consuming, especially that first time. So, I'm like, I am spending all this time trying to create this exciting and engaging and thoughtful course and my TA's working so hard, and so we were just like, but it looks like they're just slopping everything into ChatGPT, and they're not even engaging it. And so, we felt like, I think hostile would be a fair way to describe it. And again, since we're not interacting with them face to face, like we don't see them as humans and it feels like they don't see us, they don't, they're not respecting our time. and so that's when I was like, okay, we need to do something different. And that's when I reached out to digital learning and TLI. What do we do here? Like, this isn't working. during, fall 2021 we had an experience where I made a small tweak to the assignment. I took out a final exam and made it a reflection paper. And that unraveled some students, that change. Just like any change, just some students were just like, the syllabus says this, how could you, you know? And so, I've been afraid of making substantial changes in an undergraduate course during the semester thinking, you know, people need stability, they need the same, making a big change could cause, you know, even more disruption. But we had the idea of doing a survey. So, we did a survey. It was instead of one of their normal assignments, the assignment was to complete this survey, and we asked them about their AI use. We asked them just general questions about how they felt the class was going, their trust in us. Usually what I would do at a mid-semester point, this was a quarter semester at this point, check in, how they feel about the content. is the content the impediment? and so what came out through the written and then the more open-ended responses, was that it was anonymous. That's right. 'Cause it was, are you using it? How, and if so, how are you using it? You know, the blanket statement says no use, but how, okay. Clearly, it's being used. So, if so, how are you using it and why? And that's when it got very interesting. So, most of them said they were using it to refine what they wrote. Some of them said they were using it for brainstorming, very few anonymously, acknowledged that they were fully, just plugging in the prompt and, getting something out. But most said that they were using it, I think almost nearly all said that they were using it in some way and the reasons why is what really surprised us. So, most of them said I acknowledge that you said we didn't have to pay attention to grammar, that kind of thing. They were deeply afraid of being judged for their writing style, and they were very uncomfortable and not fully trusting in how we would receive their more unpolished writing because it really was in these pre and post module reflections that are supposed to be kind of ugly and not pretty, not written perfectly, where we saw the most pristine writing. So, though we thought we did enough to build up trust and mutual respect and like a, a recognition that we would accept their work as it was, they didn't feel that yet, and they were just also afraid. And then there was a group work component to things each week, and they were really concerned that they would come across as uninformed or they would be unintentionally offensive or that they wouldn't know the right words. And so, we talked about that a lot in the beginning through our videos and things, but that didn't assuage their anxieties as we thought it would. And so, and they were also anxious that they could be authentic and that we wouldn't penalize them for their questions or concerns. [00:13:48] John Nash; It's fascinating. I'm hanging on every word. [00:13:50] Megan: Good. [00:13:51] John Nash: it's like a drama story unfolding. [00:13:53] Megan: Yeah. [00:13:54] Jason: I'm really impressed by the fact that you reached out that you recognized maybe what was going on And I think that was one of my first questions to you was about how do you know and I think that made a lot of sense and partly because of your experience in this class that you're able to recognize it So I'm really impressed that you reached out that you recognized it. But then I think really impressed as well that you then followed up with that survey to try to explore the why of this and not just knee jerk reaction. Even though I appreciate you talking about this feeling of hostility almost this lack of human connection I appreciate you being transparent about that because I think this is very common I appreciated the way you followed up with the survey to get more information to get into the whys of why this is happening and it makes a lot of sense to me [00:14:42] Megan: Can I share one other example in I’m trying to remember back our levels of frustration. So, I'm teaching a graduate course at the same time. And so, I've been bringing this to them as a discussion, like, okay, I'm teaching professional development class for our first-year PhD students. a suggestion from a graduate student who was on the conduct board at a different university, said that when we were on conduct board, what we discovered, the only thing faculty could do at that time a couple years ago to prove that there really was like cheating, was by adding a Trojan horse into the prompt. And to see if you were just copied and pasting how this would play out. So, the Trojan horse was like, and so I this, we did do this, and it was kind of funny, but that's when we realized we'd kind of crossed over into like, we need help because we're being, we feel too hostile. So, I started adding, I actually used AI to help me create a Trojan horse for a couple different assignments where they came up with a phrase. So what we did is you had the prompt in canvas, and then there was this line afterwards that was in white font, so they couldn't see it, and in tiny font that said, in your answer, make sure you address x, something completely unrelated to the topic, something you would never have had in the class. And so, when if they copy and pasted it right into chat, GPT, the chat GPT response would engage around some doctrine or something that was completely made up and you couldn't Google. It doesn't exist. And we found that a handful or plus of students who were not paying attention to even their prompt and to chat GPT, were just using this made-up thing. I'm so confused. I did use this prompt, but when I copied it into my Word document to work on it, this part came up and I thought, oh, I just missed that doctrine in the documentary series. And so, the Trojan horse did not work 'cause this student was right, like we could see she walked through the logic of it. And so sure, maybe it did catch people who were inauthentically. But we tried a couple of different things that did not work in AI detection, and that's what led to the reach out, the survey and ultimately realizing that any detection effort is going to be a dangerous rabbit hole that's going to lose your sanity And it actually is not truly an accurate representation necessarily of AI use. [00:16:49] Jason: Did you try to use any other AI detectors [00:16:53] Megan: Yes. We used some of the different, I can't remember their different names, a couple AI detectors and they were okay. But actually, we spent, I ended up purchasing a subscription. I overused my limit with chat, GPT. And so, I ended up purchasing, I would say in total, we probably spent me and the TA maybe collectively 10 to 15 hours trying to get to the bottom of the extent to which this was a widespread problem because we felt like it was just had hijacked learning in the class. So, we used ChatGPT lot and actually it was very helpful. But at the end of the day, it couldn't do anything to help us. Individually, I had no interest in taking 60 students to student conduct board, especially when realizing that like this was going to yield nothing. And so, it was a fruitless effort of using the AI detection tools. But we did, we spend way more time than we should have tried to deal with that. [00:17:49] John Nash: Megan, it sounds as though there was a point then where you and your TA sort of collectively threw up your arms and so you, you consulted Jason shop and you went in a direction. And before you tell us that story, I want to talk a little bit, if it's okay with you, about what you think you learned right to that point. Was it because we've talked to different people and I've sort of consulted my own thinking around how assignments should be made in the effort to try to sort of catch students and have them, and I'm going to use words carefully here, but I understand how you might feel resentment towards the students for not taking the work seriously. And, and in some ways, I've even felt it, you say, I need them to bend to my will because this course is important, and I've spent a lot of time structuring this. And so, I need them to, to you know, [00:18:42] Megan: adhere [00:18:43] John Nash: to the standards that I'm sitting here. And then at some point. you realized maybe I'm not structuring it the right way, and maybe I [00:18:51] Megan: Yep. [00:18:51] John Nash: rethink this, these strict notions I have of how they must bend to my will. That the learning outcomes could still be achieved. But what I'm doing now is not going to, because now I'm going down this rabbit hole. [00:19:03] Megan: Yep. [00:19:04] John Nash: is that fair? [00:19:05] Megan: I think that through this and the data that we obtained, I realized also Jason, I can't remember if it was you, somebody sent me that tool for calculating how much time was being spent in the class based on the number of pages, assigned videos, that kind of thing. [00:19:17] Jason: one of our instructional designers [00:19:19] Megan: Oh, that's right. Yes. Yep, that's right. Who was my digital learning coach this summer? What I realized with that, 'cause one of the things that came up in the survey was that the one other reason they were using AI was because of the volume of content that they were receiving. I, when I re, when I did the calculate, my assessment of how much time was spent on the class was probably half of the reality. You know? So, if I was thinking this would take them four to six hours when I plugged in some early weeks, we were getting in the six to eight plus hours, which technically according to university standards, that falls within whatever. But it was, it was a lot. It was, it was more. And I think that part of that is the issue of teaching, like. The diversity class. Not that we don't have people interweaving diverse ways of thinking and doing in families and children across other courses, but in this class, I do feel like a very strong obligation to like to ensure that, if nothing else, these students leave our program without doing harm to others. And I feel like this is like the one shot I might have, you know, to really help that. And I, that's not really a fair depiction of like our courses or curriculum, but I think because of that I do tend to have too much in there. And so, I think that that did help us reflect on how much we were assigning, how many assignments we were assigning. Also, that we did module zero, like the introduction and module one in the same week. And next time we will devote a whole week to the module zero and onboarding to the class. And I think that that actually will make a big difference in terms of setting the tone to help with some of the trust pieces. There was a reflection on it wasn't just them, it was us, and that we had to make some pivots in this class, and then some that we couldn't control, like how we onboarded the class in the future. [00:21:08] John Nash: I feel like it was new to you that they, that you naively thought that they would trust you out of the box. I wonder in 2023 and 21, did you feel like they trusted you or did something shift? [00:21:21] Megan: I think that, so interestingly prior to 2021 when we were in person masks off. I am able to. Engage with students in a way where they see my humanity and we can get to know each other even if we don't know each other quite well. 2021 was when we were in person, but in masks. and also online, I can build rapport, I can build that trust. But 2021, I was really surprised when we were wearing a mask how. It was very hard to build rapport with them and vice versa. They couldn't see when we're talking about hard things, they don't see me smiling. They don't see. And so, I actually felt 2021 of all the semesters I've taught was the most challenging for them seeing me as a human, as an instructor. The reason I could tell this is because I was pregnant for the second time in that semester. The first time I was pregnant in 2019. I had to stop the students like kindly from like their questions and curiosities and excitement surrounding my pregnancy. We teach mostly women, young women who work with children. So, there was a lot of, in curiosity in 2021. When I had to explain that I was pregnant 'cause there was a day we missed 'cause I was unwell and it was, I was becoming clearly more pregnant. There was no human connection with me at all around the pregnancy. And that was fine. It just was a noticeable difference. Like they did not see the masks did hinder them from seeing me as human. So, I was mindful of that. So, 2023, it was fine. Again, normal back in person. Great. So, I am still kind of always mindful of that 2021, where how the class is perceived when they don't see the instructor who's teaching con content that challenges them personally as a human. And so, I think sometimes when you've gathered a lot of respect from students and trust, I forget that I forgot this semester was a whole new group. They don't know me. They have no connection to me. There's no reason why they should trust me. This is a supercharged political, historical moment. I think that I overestimated students' ability to trust me based on what I said in a video or on the syllabus. [00:23:22] John Nash: Yeah. Nice. Thank you. [00:23:24] Jason: Yeah, pick up the story So you've [00:23:26] Megan: Yeah. [00:23:27] Jason: For those listening we're in the fall of 2025 for those listening in the future. Actually everybody will be listening in the future but we're in the fall of 2025 and so you've come to some of these realizations some of these design changes that perhaps you want to make and you've decided that you're going to step ahead with your TA to try to make some of these changes At what point in the semester do you start to make this pivot [00:23:53] Megan: I think that this was during week four is when we did the quarter semester check-in. And I substituted out an assignment. So, this was a full a full assignment shift. and we started making the changes immediately. So, you know, I think module five, like week five had already been loaded. And so, they were already underway, we explained the changes, and I think it started in module six. So, you know, it was pretty quick, but it wasn't immediate, but it was as quick as we could in terms of uploading the next content. The first thing we did was explain the new AI policy shift and, you know, Jason kindly sent me what he uses in his syllabus And so I modified that for my class of, you know, human first and human last. You know, we want your brain doing the work, but we're here to figure out how we can use AI to kind of help you. Further your thoughts, further your work, and so on and so forth. And so that's when we implemented that. We implemented it like the icons above each different assignment. A icon of like, no, like there are certain things like quizzes where you, like, they, like, we're not supposed to, it would be pretty hard to just the timeline and things, but like other assignments where you can, you could, and that they were supposed to utilize a statement of how they utilized it. I would say some utilized that statement throughout, others didn't, and we didn't really have a policy in place where you were penalized for not doing so. But that's something to kind of think about in the future. But pretty early on in this semester we made this big shift in our policy and its implications for the rest of the semester. That seems really quick to me to be able to pivot so good on you and your TA to move that quickly You know, at the end of the day, the TA is the one ultimately at this point, who's grading everything. And I know that she agreed that we needed to make the pivot. I think that it was a continued tension for her throughout this semester. And we had talked about it some, because I think. Even though we, saw some substantial improvements. It did still feel hard to feel hindered in like what really could be done when it was clear that a student or two or a couple students were really misusing now this flexibility. So even though it's kind of like the one bad evaluation, everything's great, you're kind of focused on the one. I think that that still was hard. So, I will say it, for me, it's easier for me to feel like, yes, great progress. 'Cause I didn't see, the extent to which the TA was subjected to kind of this shift still in what we're expecting student writing to look like. [00:26:23] John Nash: but you saw some kind of shift that went in a better direction than you expected or a positive [00:26:28] Megan: Yeah, [00:26:29] John Nash: that? [00:26:29] Megan: We actually saw, it almost seems like students started using AI less. On these reflections, we started to see specific examples. We had a couple students meet with the TA to talk about, like, we realized that many of them did not understand what we meant by personal examples. And so, a student who would give feedback like this doesn't show authentic student voice came and met with a TA and as the student's talking our TA's, like, that's exactly what we're looking for. Like that story, that little snippet of your third grade or whatever and they were like, oh. So, we realized in the future we need to be much more clear by what we mean by specific examples. It also seemed that this quarter semester check-in really helped build trust. I summarized, you know, and I shared, I said, here's how I used AI to summarize your feedback. You know, and so there was a lot of transparency involved, and I think that that also started to begin the process of them actually trusting, at least what they were submitting to us. I think also as a semester goes on and students’ kind of test the water with what they push back on or whatever, and they earn full points 'cause they've engaged at the assignment, they started to trust further that they could authentically engage. But we did see a shift in a reduction of what appeared to be AI generated content as we move throughout the rest of the semester. [00:27:46] John Nash: That shift, you noticed there was sort of a, what there's an old methods textbook by Krathwohl. He talks about knowing judgments sort of is one way that knowledge gets produced and that experts may, You and your TA made knowing judgements that these, this looks qualitatively different than [00:28:01] Megan: Yep. [00:28:02] John Nash: Right. [00:28:02] Megan: Yep. [00:28:03] John Nash: totally legitimate. And not to paint too broadly with a brush stroke here, but you saw two things. One is we opened up the allowances of use of AI and AI use went down and by asking students what they thought and felt and incorporating that feedback rapidly, improvement was seen. Are those two big points that I can say are fair to say. [00:28:30] Megan: Another point, and I will say, you know. On the record if you want to use it, or honestly this could be a whole different discussion, which I actually would love to have, is we started using AI very and I, and I typically do in my grad classes, like we play with it, we utilize it, we mess with it. But I had some of our more big or controversial topics. I created assignments with ChatGPT, and I had them plug it into AI and then they analyzed AI's response. And I think our use of it maybe also helped, but I think that that's probably not the substantial part of it. But I do think as like another topic like how to use AI to like, navigate the sociopolitical climate in a way where I wasn't teaching certain things. Like, comparing administration's policy approaches. AI gave the breakdown and then they reflected and unpacked it in assignments. And so, I think our AI use may have helped, but I do think the two big takeaways that you mentioned are the key point. [00:29:35] John Nash: And now I hear it's almost as though prior to this shift, you were on two sides of this situation where you had content and the students were supposed to learn the content, whereas now, after the shift, you came to the same side and particularly around AI saying, here's how I can use AI [00:29:53] Megan: Yep. [00:29:54] John Nash: And here's how you might use it with me. Is that kind of like how it is now? [00:29:59] Megan: I think that's a good way to think about it. And that's also helpful for me to think about because I won't be teaching this again until the fall. But thinking about in that week, that first week, having assignment or something that's kind of around like a video of me talking about the ways that I use ai, the ethical lines that I've grappled with, with ai what we know about the science and employment for people who rely too heavily on ai, like, to get them thinking about that too, the way I do in my grad classes. So, I think giving a little bit more trust and respect to them upfront and owning my use [00:30:27] John Nash: Mm-hmm. [00:30:28] Jason: I was struck too about how you said the survey itself was a trust building exercise essentially, right? [00:30:36] Megan: Yes. [00:30:37] Jason: Which makes complete I really thought about it more of a this is an opportunity for us to gather information so we could move ahead .But really there was something that changed perhaps in the students In the fact that you were asking them and that you were reaching out to them in that way and they had an opportunity to be a little bit more transparent about it And not just trust building but probably communicative in that way too right There's a [00:31:01] Megan: Yes. [00:31:02] Jason: survey communicated something a shift the issue perhaps some changes that were coming down the way so. [00:31:11] Megan: Yeah. [00:31:12] John Nash: Yeah So, Megan, what happened at the end of the semester? [00:31:16] Megan: So, we had been feeling really good. I mean every once in a while, the, like my TA and I would send me like a funny, like one that like clearly, you know, one of 80, you know, that just kind of made us laugh 'cause it's just not how. A 21, 20-year-old would engage in a con, you know, but for the most part we were feeling, you know, pretty good with it. Or we were noticing where we could do things a little different and that would've probably changed the kind of answer. But now fast forward to the end of the semester and we actually see a huge uptick in clear AI use again. In the survey they talked about that there was a large amount of work in the class and that was one reason why they or others. So, I think in the survey we said why might someone use AI? And so that helped it to not be, you know, we know how students usually are responding about themselves, but it helps take that off of them a little bit. I think the end of the semester, even though I had actually been really reducing some of the work as the semester wrapped up for their sanity and ours I think the overwhelm of the end of the semester, even if it's unrelated to our class, did probably play a role in what we saw was a bit more AI use. So, there was two kind of final reflective assignments and in one was individual and one was in their group and in the reflective individual assignments. We had a question around looking back at the course description or the goals, you know, like how did this align? And then there was one that was like, what would you have liked more of? Like what would you have liked to have understood More? Like what? Or like, you know, something and we had like two. There was like something more about resilience and then something more about disability, but in a very specific way that was clear that most of these had had a very substantial AI role versus just editing. So that was one thing we noticed. And then in the, the final group reflection in their team. So, they'd been with the whole semester, they were just supposed to think back. I think a couple questions of like, how did doing this work in a team like enhance your learning or something like that. And my TA sent this is making her cuckoo bananas like all over again because it was so many of them what they plugged into ChatGPT, ChatGPT interpreted this as a group project, You know, we're all contributing to a presentation where this is just a canvas discussion board that they were having. There was no, and so these responses are completely disconnected from actually what they were doing. And we saw this among students, even those that had not, had been really strong and not using AI throughout. And so there was, at the end of this semester where we were most excited by these more authentic reflections. We hope to receive a bit of a return to the beginning of the semester, so we haven't fully unpacked that at this point. But there is something there were maybe the stress of the end of the semester; they just need to get this done. Maybe they have another big exam, and this is small scale stuff where we saw a return back to overuse. From our perspective, [00:34:09] John Nash: Fascinating. I can concur from personal anecdote, similar situations, not necessarily to use of AI, but it, when students, undergraduates have taken my class, they've seen it as a secondary effort to their courses in their major. Whether it's in chemistry or in business, and they'll complain that they have so much to do over there that their effort towards the end of the term when our big project was happening was lackluster. And I think, I wonder if it makes us think about the ways in which we should load up assignments. Towards the end. We wanted to be a culminating, like, look at the bow we've put on this thinking and look where you've come. And now I wonder from your story and my experience, how we maybe rethink that a little bit. [00:34:51] Megan: Yeah. [00:34:52] John Nash: Hmm. [00:34:53] Megan: That's where we're kind of left. So very positive shifts. We learned tremendous amount as an instructor, as a ta, and as the TA will independently teach in the future too. But I think that, I will say, if I were to ask my TA kind of how they would've described this grading experience, I think that they would probably say it overall was kind of a soul sucking experience. And I think part of it was. Having to shift how we think about doing things, and I think also she would say that there was tremendous growth in the students. There was a lot of positives. But I think as learners whenever we have signs where it feels like people aren't using their brain, I think from my TA's perspective, that felt a little scary about the future of learning. [00:35:34] John Nash: That's almost like back to, that's the knowing judgment, isn't it? I, I'm getting feeling in my gut that people aren't using their brain and that's, yeah, that's real. Jason, you want to ask this cool, final question. [00:35:46] Jason: Oh, sure I was going to throw in just as an idea of, we've been reading through this book Opposite of Cheating which I highly recommend [00:35:54] Megan: Okay. [00:35:55] Jason: and in chapter three talk about one of the things they talked about just made me think of why students are more likely to cheat when they feel the following One of them is that an assessment is insurmountable And I thought that was interesting like as you described this great midpoint where it seems like that students are in the flow and they're being more authentic and they're using AI less and so on And then this description of coming up to the end of the semester And I think about not just a single assessment but you think about the end of a semester sometimes feeling insurmountable and how those feelings may compound across your courses When this is our finals week a library and told me yesterday that the library was packed right They were pulling out foldable tables for people to sit down and to do their work and You think about that And I wonder about too in terms of overall design whether or not there are ways in which can rethink perhaps these as wonderful as they are as you said John putting a bow on the semester Let's see really dig in Let's see really what you have learned but maybe there's some ways to rethink the end of our semesters in terms of these big assignments What do you think? [00:37:13] Megan: No, absolutely. I agree. And I think actually when you said this about insurmountable, it made me think, so the questions, I think, so there was like five short, but open-ended questions like kind of reflection paper kind of style. And the first two questions we asked might have honestly been too intellectually challenging for this level. They are the same questions I've used in the graduate version of this class. So, I expect a different kind of response from the undergraduates, so the goal of this class is to understand how did we get here and why does it matter? And I ask them to reflect on that. That might be too big. For, the group. And then the other one was how well did the class, tell the tale of the ongoing struggle of oppression, resilience? And those might have been a bit too intellectual like that might've. Triggered some anxiety or imposter syndrome. Like I know what those responses typically look like, but that those questions might have been too big. And then I get into these smaller ones of like, what did you like? What did you not like? What would you tell future students? And those questions are lower stakes, And so, I think thinking about what kinds of questions we're asking at the end of the semester is probably important. [00:38:27] Jason: That's good thanks for reflecting out loud with us on this You're just barely ending the semester here so we really appreciate you spending the time with us I kind of had a final question which was as we have a lot of instructors and instructional designers who listen to this podcast what advice do you think that you would give instructors then given your experience this semester when they're thinking about setting up their class for the next semester [00:38:57] Megan: Yeah, I think that I was so focused on the diversity, like addressing those pieces and then jumpstart and that was me, you know, and also just the demands. I think that there should be a substantial amount, more time spent discussing. AI when working with faculty members who are going into distance ed courses or online teaching. I think that that wasn't something I did enough of probably. So, I take responsibility, like, you know, I think that there should be a lot more thought and reflection and thinking. Before teaching your first, or as you're teaching online distance Ed. So, people working with faculty I think we need to have a lot more engagement around it because I have some faculty members who are much less engaged with it and have no idea how much AI use was even going on in their course. And so, I think that there's a large range within faculty of those who are like, no, stay away from it. Avoid it. Don't engage. Hope it's not happening. And so, they don't see kind of patterns to those of us who are like, who went too far down this rabbit hole of trying to like to solve the problem. And so, I think that that range exists probably in all departments. And so, I think more discussion and support as built into like those structured programs like Jumpstart, like I know the university offers all kinds of workshops and trainings and so forth, but I think when working with faculty, I think, that would be ideal to be part of the conversation more so. [00:40:11] Jason: Yeah, that's good [00:40:13] John Nash: Wonderful. Thank you so much, Megan, for joining us. This was a tale for everyone to hear. I really think so. I think, yeah. Yeah. [00:40:21] Jason: you be willing to share the final policy that you used in your class [00:40:26] Megan: Mm-hmm. [00:40:26] Jason: With us Okay That'd be great we'll put that as well as I can share the policy that you started with The one that I share out to faculty I actually have some icons as well that I use for communication that I can share out And we can put those in the Podcast notes and for those listening online learning podcast.com where you can find this episode And a lot more episodes too we've building a little bit of a library John but where you can access these notes and look at these policies for yourself Think about how you're applying your thoughts about AI into your next Course or semester So thank you so much for spending this time with us It was amazing to connect with you a little bit more and hear your story [00:41:12] Megan: Thank you so much. This was really great. Thank you for having me. [00:41:14] John Nash: We did it.                    [00:41:15] Megan: Woo! [00:41:17] Jason: We did it

EP 39 - The Higher Ed AI Solution: Good Pedagogy with Lance Eaton

In EP 39, John and Jason discuss with Lance Eaton the threat that AI-driven "agentic browsers" pose to continue industrialized online learning models, the necessity of clear institutional policies to support instructors, and why good pedagogy remains the best solution to the “AI problem” and faculty exhaustion. See complete notes and transcripts at www.onlinelearningpodcast.com [http://www.onlinelearningpodcast.com/] Join Our LinkedIn Group - *Online Learning Podcast [https://www.linkedin.com/groups/14199494/] (Also feel free to connect with John [https://www.linkedin.com/in/jnash/] and Jason [https://www.linkedin.com/in/jasonpauljohnston/] at LinkedIn too)* Guest Bio: Lance Eaton, Ph.D. is a writer, educator, faculty developer, instructional designer, and educational consultant based in Providence, Rhode Island. He holds degrees in History, Criminal Justice, American Studies, Public Administration, Instructional Design, and Higher Education. His writing has appeared in newspapers, trade publications, academic journals, books, and encyclopedias. With more than 15 years of experience creating online content—including blogs, a YouTube channel, and other digital projects—his work spans education, technology, and learning design. He has extensive experience working with youth, nonprofit organizations, higher education, and online communities. Connect with Lance at his website here: https://www.lanceeaton.com/ [https://www.lanceeaton.com/] , his substack here https://aiedusimplified.substack.com/ [https://aiedusimplified.substack.com/], and LinkedIn here: https://www.linkedin.com/in/leaton01/ [https://www.linkedin.com/in/leaton01/] Resources: * Post: Looking for ChatGPT Teaching Advice? Good Pedagogy is Nothing New [https://autumm.edtech.fm/2023/07/19/looking-for-chatgpt-teaching-advice-good-pedagogy-is-nothing-new/], July 19, 2023 by Autumm Caines * Lance’s appearance on AI Diatribe Podcast. [https://www.youtube.com/watch?v=frQm42ie5I4] * NCFDD Workshop “The AIs Go Marching On: Finding Our Way with AI in Education” - https://members.ncfdd.org/finding-way-ai-education-webinar?submissionGuid=b1228e61-a304-42ca-a174-83c92a56a7e5 [https://members.ncfdd.org/finding-way-ai-education-webinar?submissionGuid=b1228e61-a304-42ca-a174-83c92a56a7e5] Theme Music: Pumped by RoccoW [https://freemusicarchive.org/music/RoccoW/contact] is licensed under an Attribution-NonCommercial License [https://creativecommons.org/licenses/by-nc/4.0/]. Transcript We use a combination of computer-generated transcriptions and human editing. Please check with the recorded file before quoting anything. Please check with us if you have any questions or can help with any corrections! Cold Open [00:00:00] Jason: Lance you had brought up AOL - don't know if either one of you knew, but just recently, September 30th, they actually finally stopped servicing their dial up. Just September 30th, 2025. [00:00:14] John Nash: I heard about that there were some people That were still dialing up. [00:00:19] Jason: Yeah. for a lot of our listeners, they don't even realize that, that we use the dial up sound in the beginning of our podcast very intentionally because we were talking about online learning in the second half, and that was an artifact of. The first half of online learning, we would call it where people had kind of sketchy internet connections weren't able to do a lot, but it was the beginning, you know, as, as we kind of talked about, you talked about Lance starting in online education even in the early two thousands. So, so I wondered if maybe because I, I thought you guys would understand, maybe we take a moment of silence for the, the AOL dial up service. [00:01:01] Lance Eaton: I feel like there should be a digital bugle. [00:01:05] Jason: Yeah, yeah, that's right. Playing some digital taps. [00:01:09] John Nash: Alright. [00:01:09] Jason: Yep. Intro [00:01:10] John Nash: I'm John Nash here with Jason [00:01:11] Jason: Hey John. Hey everyone. And this is Online Learning in the second half the Online Learning Podcast. [00:01:17] John Nash: Yeah, we're doing this podcast to let you in on a conversation we've been having for the last three years about online education. Look, online learning has had its chance to be great, and some of it is, but some still has a way to go. So how can we get to the next stage, Jason? [00:01:33] Jason: that is a great question. How about we do a podcast and talk about it? [00:01:38] John Nash: I think that's a fabulous idea. What do you want to talk about today? [00:01:40] Jason: Well, first I want to say it has been three years for the podcast, I think now. Right? But you and I have been having this conversation a lot longer than that now, John. [00:01:49] John Nash: We have. We have, we [00:01:51] Jason: We probably first met in 2016 now, so we're coming up on like almost a decade, I think at University of Kentucky when I was there. [00:01:59] John Nash: Yeah. [00:02:00] Jason: You were [00:02:01] John Nash: Yep. [00:02:02] Jason: 2016, a potential professor as I was thinking about the PhD program and then became my professor and chair of my dissertation study. [00:02:10] John Nash: Well, I was a, I was a professor. I wasn't a potential professor, but I [00:02:14] Jason: That's right. [00:02:15] John Nash: I was a, I was a potential dissertation advisor. [00:02:19] Jason: You know, don't lock yourself down. I think you have a lot of potential, John. [00:02:23] John Nash: Yeah. And we were teaching online in that department since 2012. So, with the days of Adobe Connect, oh my God. And Moodle and yeah, all the fun things. [00:02:36] Jason: Yeah, yeah. And we have with us today, sorry, I don't want to ignore our guests. We have with us Lance Eaton. You know, the bottom line is this is just an opportunity for John and I to talk. Hope you don't mind just stepping in here and listen to us jabber. [00:02:49] Lance Eaton: To pull some popcorn. Like I, I'm interested why I'm into this hook. Where's the season going? I want to know. [00:02:55] Jason: You could probably even riff off of like you've been at online learning for a long time, Lance, so we'll get into a formal introduction in a second. But what are some of your earliest memories of doing online learning and technologies and [00:03:07] Lance Eaton: I, I often talk about, like, I took my first online course in 2000 [00:03:11] Jason: wow. [00:03:11] Lance Eaton: then I taught my first online course, I want to say around 2009, [00:03:16] Jason: Mm-hmm. [00:03:17] Lance Eaton: And then started doing instructional design with faculty around online learning in like 2011. So, it's definitely been, been around the block and, and have been at different institutions from community colleges up to like Ivy League institutions doing this, this kind of work. [00:03:33] Jason: Yeah, that's good. Well, tell us a little bit more about yourself where you are located, what your current role, what, what you do with yourself. [00:03:40] Lance Eaton: Gosh. That's a loaded question. So yeah, I'm Lance Eaton. Full-time, I’m senior Associate Director of AI and teaching and learning at Northeastern. And then outside of that I've been teaching part-time at different places like North Shore Community College and College Un Bound for years now, and then I've been doing a whole lot of like, talks and workshops and consulting around AI in higher ed since pretty much like March, 2023. I think I have hit the hundred mark in terms of like talks and workshops that I've been doing. Just helping and, and thinking and, and working with folks to. Help figure this out. And it's been quite the trip. I'm located in Providence, Rhode Island, just for geographical sense. [00:04:27] Jason: Yeah. [00:04:28] John Nash: Yeah. That's excellent. So, so Lance, what are you noticing right now seem to be the experiences that faculty are having on what's working and what's not working with generative AI? [00:04:38] Lance Eaton: I mean, what is working is what has been working, and I go back to Autumm Caines of like, if you want to figure out how to navigate ai, good pedagogy. And like that, that there's a post she wrote back in 2023, the post is like, you know, the solution to Chat GPT is, is good pedagogy. So, I think that that's one thing is just recognizing good pedagogy is adaptable. It is, it is often thinking relationally with the students. In a lot of what I have been seeing is where there's the most success is also where it's often. And, and this is hard, but it's, its often where like faculty are engaged with students thoughtfully about how to navigate this, these new set of tools that are familiar to some things in the past and also new in certain ways and, and building that trust and that rapport. So I've seen that work really well. Also, like that can, like trying to find, you know, there's real struggle there because there's some spaces that doesn't work as well or it needs something else to move it along. So large enrolled classes, you've got 200 students. That becomes harder. Asynchronous courses also become more challenging in this. So, I think I've like, I've seen movement in that space. I think a lot of it is like what assignments are and what they should be, what they're assessing, how that, that gets structured is a lot of the discussion and a lot of the good discussion and, and things that I see people coming up with that get me excited. We're also in this cool moment where we get to figure out what it looks like next. I know that's hard to like hold onto because we're on this exhaustion wheel. Not just like, not just with AI, but like AI coming after the pandemic, the attack on higher ed and all of [00:06:26] Jason: Mm-hmm. [00:06:27] Lance Eaton: You know, government being shut down for a month at this point and like the impacts that's starting to have, so like there's, there's real hard things that are settled on our minds, but there is also, there's a way of looking at AI as a, oh, this means we, we are going to have to change and we get to change because we're realizing what I think a lot of what we've done certainly hasn't worked for everybody and at times relied elements of convenience that weren't necessarily great demonstrations of learning for, for different courses in different environments. So those are some things just kind of percolating in my, my head at the moment about like where my, where I want to put my attention. [00:07:09] John Nash: Something hit me because I caught you on the AI Diatribe podcast, which was on recently, and it was your take on how the tools keep changing in these last three years. Tools keep changing the fraught environment around faculty is changing and then AI is changing also, and there's this cognitive demand. This is what struck me that you said was sort of, that's the thing that hit me, that faculty who have already been, and I think you said, I'm, I grabbed a quote here. "You've already been hamstrung by so many other demands. Having to once again, think about their curriculum, think about how they teach, how they engage with students for like the third or fourth time in five years." No wonder this is a challenge. How do you work inside that environment? As we try to help faculty think about good learning design, good teaching and learning. [00:08:00] Lance Eaton: I think that's a great question, and I don't mean to pat myself on the back, but that, like, that is a really important, that is a [00:08:05] John Nash: You know, [00:08:05] Lance Eaton: point of like [00:08:07] John Nash: it was great. [00:08:08] Lance Eaton: on faculty and the absence of, of recognizing that. So that's, I mean, when I step into spaces with faculty around this kind of work, that's one of the first things I, I look towards is just validating: this is a lot. We haven't closed loops or come back to do any kind of, like any opportunity to reflect on like what do we want to take out of the deep lessons we learned in the pandemic? And so much of that got thrown away. As a first step, just naming, like, and this is often a slide I'll have where I'm like, here's all the things going on. It's like AI as it is now and then having to think about what AI is going to be in six months. And then it's thinking about, you know, the higher the, the, the larger landscape of the world, all these existential crises. So that validation, it doesn't cure anything. But it also, lets, lets faculty know they're in a space that gets that. And I think from there it; it's often doing a couple different. Moves that I think are really valuable. One is also helping them connect, you know, its learning, right? So, help them connect prior learning to what they're going to do next. So, you know, I will jokingly or not, like, make mention of Wikipedia in the Wikipedia war that we were all fighting 15 years ago, right? [00:09:26] Jason: Mm-hmm. [00:09:27] Lance Eaton: You know, and then do callbacks to like. And we had so many of the same or similar conversations about the internet and what that would mean for teaching and learning. And so, this rightful tension we have with emerging technology and how it impacts our learning, you know, we can always go back to, like, all the way back to Socrates and, and you know, the whole writing thing was, was meh. So, like that is a, that is a perennial thing that we encounter. if they've been teaching for a while, they know that they've found solutions. And so that's one thing I try to look towards is like, you know, you found solutions, you know, and, you know, you've been able to figure it out. Then I move into like one of the final moves, which is, and we figure this out together. Like you figure this out by trying things and sharing things so much. And again, this is a, this is a byproduct of the time in the world that we're living in is like the isolationist experience of like, you know, more and more, more just on your own, but like sharing with colleagues and finding platforms and spaces to share. And then I'll use that space to also move into. talking with your students, like, I can't, like, we can't figure this out without them. This is one way where I think the technology is different is that it it's potential to be to be helpful or hurtful in the learning space and do so in, like me, you know, many, many different ways does require us to be in better conversation with students because time and again. Like I learned from students new things that help me better use ai, but also how they use it. Like of course we're going to have the folks that are using it inappropriately, if you dig down and build that trust and you help students build a curiosity about how can I use this to support my learning and not bypass it, like. They're going to, they just come up with great stuff, and that's the stuff I want to take and curate and put into my course as like, Hey, here's, like, it's out there, it's ubiquitous. You can't open a browser without finding ai, just like jumping out at you. But here's some useful ways to engage with it. Here's some useful things to do that can help you as opposed to hinder. [00:11:45] Jason: Mm-hmm. I wanted to say we thought the Wikipedia wars were bad. Wait for the Grokipedia wars. [00:11:50] Lance Eaton: Oh, [00:11:51] Jason: That's going to be something. [00:11:53] Lance Eaton: Oh. That is, I don't, I, that, that, that worries me. Yes. AI and, and, and can feel like very extreme ideology. [00:12:05] Jason: Yeah. Well, and it just points to the fact that there is always going to be something, right? I think that teachers have not only had this job, within the digital age here of, of re-identifying, recreating their content teaching a different generation of people. I mean, this is, I think this is the job of the teacher ongoing, right? And I know it's exhausting. And maybe Some of the difference is what feel like grand cultural sweeps are happening pretty quickly in the last five years, right? [00:12:41] Lance Eaton: I, I think there's, I think there's that, and I think there's, just because of the nature of this tool, and again, I understand why it's happening, but this is, this is what's contributing to it, is a lot of institutions just not responding quickly enough to give faculty and students something. You know, I, I go back to like giving a policy, even if it's the most barest bones policy, [00:13:11] Jason: Mm-hmm. [00:13:11] Lance Eaton: Gives the faculty clarity. About how they're going to be supported, gives the students clarity or a bit more clarity about like what the institution's stance on it is, and allows the faculty member to still have full academic freedom, but to know that like now that there's a policy, that policy at least frames what the institution's disposition is that they better structure how they're going to work with that within their courses. [00:13:39] Jason: Mm-hmm. [00:13:40] Lance Eaton: There's still, I, I have not done the count, but I would guess there's still thousands of, in institutions that have no formal policy or have given very limited guidelines that are either like. Just in the Center for Teaching and Learning or the instructional design team's website like distributed once a year ago via email. And I think to me there's like, that's part of where faculty struggle a lot is like, much do they want to invest? How much time can they give to something? A time in which they don't even know, like they can do, because you can have policy and, you know, you can have policy not interfere with academic freedom and cover things that we know should be covered. Like, you know, stuff around FERPA. You know, those types of things or, or appropriate and, you know, that level of appropriate, inappropriate use. But without that there's a lot of like, I guess this is my policy, but I don't know if my institution's going to back me on, like the decisions I'm making. so, I think that plays into it as well as the, like looking for some leadership and there's, there's a, there's some vacuum there. And again, I said there's a reason for that because again, a lot of the leadership is also dealing with all the political stuff that's going on, all of the now economic stuff that, that's going on. So, like there, there's real demand to try to figure out, you know, all these different things. But I think that's where there, that's what I've seen a lot of faculty struggling with. [00:15:17] John Nash: I wanted to, I have sort of two other things to talk about and then Jason, you've probably got some ideas. One Lance is a little more sort of philosophical and prognostications and the other's kind of practical around faculty development. So, but let me go philosophical first, because in this, such as learning designers and teachers and faculty have moments, we're in this moment now where there's some discussion about what the presence of ag agentic browsers means for student work inside LMSs and things like that, and there's instructional design implications for that. There's also sort of policy implications for that, and I don't think there's a shortage of discussion in LinkedIn and other circles on Substack about what that means and what happens transactionally. I wanted to go one step further, if you'll go with me in that if this doesn't get addressed smartly. What do you think the cost is of agentic browsers to higher ed graduates who move into the workforce? I mean, I think, I think about the fact that if a, if a Silicon Valley or a San Francisco company wants to release an agentic browser, this is a company that has coders and product managers and other marketing and salespeople who got degrees somewhere to do that work, and now they're putting a tool in place that may actually undermine the kinds of learning that has to happen so that those people can get those jobs. Do you buy into that idea and what's, what are your think, what's your thought here? [00:16:47] Lance Eaton: So, this is a, this is a post that I am, I'm working my way through so I can engage with a bit here. I think like I, I have no have no trust of Silicon Valley to like, they can see it as making a buck, they're going to make it, you know, there's, there's, there's self-justification and we've seen that with the AI of just how easy they want to sell us, all these threats, which is why they need this money to figure out ai. And so, we don't have the singularity, even though there's a lot more harms directly being done. So, there's like. That piece of it that I, yeah, I have no, I have no doubt. You know, you have folks like Peter Thiel and the like who are investing money to do things that will undermine not just higher education, but like public education. So, there's like that piece of it. I mean, to me, the thing that I'm, I'm grappling with or, or trying to sketch out in my mind is to, to articulate this, this argument and this concern. So, as I said, I've, I've been, I've been, I took my first online course in, in 2000. You know, worked at, at a variety of places and engaged with lots of different folks and like it is fascinating to me that the discussion board for the majority of courses has not changed in 25 years. The online asynchronous discussion: post once reply twice. You know, and that's not just, it's not all faculty, it's not all courses, but like that still is, a major mechanism of, of interaction and is a very limited method of, of interaction. So. When I think about that, one of the things that I have seen most institutions, they've been playing this game of, you know, we're offering online learning for access to improve access and learning on one side, and then on the other side, framing it as their money maker. They are framing it as this is the thing we are doing to make income to support other things. And they have made intentional decisions, very business-oriented decisions to cut out all the fluff or whatever they want to, like, everything they to consider is not about the bottom line. And education is not supposed to be a business. It is supposed to be a nonprofit. It is supposed to be an endeavor that not run as a business because it's supposed to be a public good. Unfortunately, because of like neoliberal practices over the last 50 years of in continuingly to dismiss or undervalue this idea of a public good, a thing that we share together, we increasingly lose public services. We increasingly lose a public center higher ed has, is trapped in this and has seen. It's online learning as that mechanism to make money, and so it has not actually thoughtfully engaged with what. Dynamic deep online learning can be, instead of like how quickly and easy can we do it? And we can see that in the majority of the courses, like, I'm not saying there isn't thoughtful pedagogy. I'm not saying like people aren't caring about it, but you know, the cookie cutter templated course that dominates the vast majority of online learning. Like there is a way in which we are about to pay, like we are about to pay the tax of privatizing and emphasizing the individual, value versus the communal value as agentic AI emerges. Because I do really think if we looked at the last 25 years and we really pushed ourselves to not just make it about saving money, but to think about. In this new space, in this cyber space, in all of the tools that have continued to emerge over the last 25 years in this space, like what could dynamic learning look like? And all we've come up with by and large is, oh, you go into an LMS, right? You go into a closed environment, then you go into your course, which is its own closed environment. have no real control over that space. It is basically just like you follow all the things that are laid out to you, hopefully. Effectively in a very linear way, and you just complete these assignments and like the same kind of meaning making that happens in face-to-face classes but also happens in other online communities. I mean, you can go to places online where there is rich meaning making and learning and exchange happening. We forego almost all of that. I think that's what we're about to pay the price for is like we have; we have made it into assembly line. So, when a tool comes along that knows how to maximize the assembly line, I really do wonder how some of the big, massive online institutions are going to navigate it, because I can't imagine. Just as they've been gaining over the last 15 years. Legitimacy, like every student now that graduates in the last year through the next few years, is going to have to somehow articulate to a degree, like they had to do back in the early two thousands. That, "No, no, no, I really did this work. AI didn't do this work." Right? Like there was a period certainly in, you know, late two thousands early in, in through the 2010s where it was becoming more legitimate. And I feel like this is the moment where, unless there's a really good solve for that, like we're paying the cost of kind of shortchanging that space as a learning space. [00:22:41] Jason: Yeah. Great historian, Otto Peters said, even though he was positive towards online learning, said it was probably the most industrialized form of learning. And I, I think that there's always that temptation because it is such a fantastic platform for just delivering content to people, creating a structure that people can't escape from, and then just delivering them, [00:23:12] John Nash: Yeah [00:23:12] Jason: content in this close structure. So, I think that I think that temptation is always going to be there. [00:23:18] John Nash: You touched on it, Lance. I think I was thinking the same thing. I would hate for my public land grant institution, flagship of my commonwealth to decide that they will just sort of with a wink and a nod, say, "yeah, we'll take the tuition thank you." Because actually this will get the, maybe we'll even, oh my God, I just thought of this, but it's it know I, now I kind of shudder. " We will make our graduation goals now. We'll hit now four years. Instead of a five year, we'll actually get four-year graduation rates." [00:23:51] Lance Eaton: That, that's a scary scenario. And, and I think that, you know, it's that dynamic of, I've just seen so much of you know that, to your point, Jason, of the industrialized version, like I've kept thinking about over the years, what does the parity of experience online look like? And people, you know, there's, there's a reasonable discourse of like, well, they don't need the same experience because many of them are, are fully involved in their own context and lives and things like that. And there's a part of that I get. But there's a piece of this that is still, I think they're still missing a lot of possibility and exploration. And again, there's, there's reason, there's some good reasons for that. Like accessibility is also about consistency. So, but I feel like that becomes a hammer to then mute out or to just like, not allow for any real creativity in these spaces to occur. And so, yeah, that I, I do worry about agent browsers in this in the next few years. Because even if they come up with, think, I guess my concern is even if the browsers themselves or the LMSs themselves come up with workarounds, like, I don't know, 10 minutes on TikTok, I'm sure you'll find, you know, using the right hashtag, you will find all the workarounds. [00:25:16] John Nash: Yeah, definitely. Cheater's going to cheat. Yeah. Well, so let's, we'll step away from the doom and gloom for a second and go to the practical, because there are bright spots out there. I saw your National Center for Faculty Diversity and Development webinar. That was so cool. And we will put a link to it in the show notes, What I loved about your webinar is that with several hundred in the room, you still kicked off by saying, I'm going to give you voice and choice and here's three things we could do today, you guys pick which one we're going to do. So, what are your go-tos right now in faculty development around generative AI? Jason and I are in circles now where we're also kind of supporting our local units. We want to be helpful. What are you recommending now? What are you doing? What are you liking? [00:26:03] Lance Eaton: I mean it's, I appreciate the, the mentioning of that workshop. 'cause that's something I've started to do regularly. I also did it in a workshop this week where it's basically, I have like three different approaches and I try to think about those approaches of highly engaged, middle engaged and less engaged. And that to me is like part of the larger process of like trying to meet people where they're at. Because again, people are coming in its year three, and for some folks they're just starting to, to think about it. Other folks are much further along like where they are stepping into any workshop, it's good pedagogy to, to allow them agency in that space. also like. It's, it's also me trying to reflect the idea of, we want to be engaged with, with our students. And when we do that, we can surface stuff. So, a lot of the stuff I do now is usually trying to do activities where they're surfacing ideas and information and like helping clarify with one another. I certainly step in and add my, you know, my insights the points that like I've been able to figure out or the things that like I can speak more directly to. But it is often, like get faculty in a room with opportunities to either share their ideas, their questions, or their discoveries. Because, especially where we are now, I think there that just needs to keep happening till there's a critical mass. I think this is the thing. I worry about it, you know, three years. in there still doesn't feel like there's a critical mass at most institutions of faculty who. Are thinking about it, thoughtfully engaged with it in a way that isn't just, I'm banning it because I'm hypercritical of it, and I understand there's places for that, but I don't know. I think constraints as opposed to abstinence is a better strategy. And so, I'm not, I I'm still not seeing institutions that have a critical mass of faculty doing that. And it, I Whenever I step into these spaces, that's what I try to make my goal. It's like how to surface the things that are happening to help others feel like they can do something. Because often as an outsider coming in, like I'm an outsider. Like I can, I can give ideas and stuff like that, but it's really their peers that I think are going to be most helpful in, in figuring it out. [00:28:25] John Nash: Yeah, Jason. [00:28:27] Jason: Yeah, other than the solution, obviously, to get you to come in and talk to our faculty and meet them exactly where they're at, why do you think this is? Because I, I think I see this at my own institution as well. There's obviously, you know, we're, we've all studied the diffusion of innovation and how, you know, we're going to have those early adopters and so on. And I'm seeing that for sure. But I really do feel like that I expected us to be further along three years into this and, I am surprised by the number of beginning conversations we're having about AI. Partly because, you know, it feels like the moment this came out, John was texting me about it, you know, and we were, right in there figuring out what this new technology was and thinking about it. Implications for education and then it seems to filled at least part of the content of almost every one of our podcasts since. February 20 So, what do you think from an outsider's perspective or even at your own institution's, what is keeping us from moving forward? [00:29:35] Lance Eaton: I mean, it goes back to, it's a couple moves I've seen of like, it's the policy piece and then the tool piece. So, like having a policy, gives people a sense of like boundaries, but also a policy without an actual tool to say like, here's our institutional tool, in that it's a legitimate institutional tool. And so, when I say that like a lot of institutions have turned on like the basic Copilot, and I often, for those of us that are of a certain age, I'll, I'll use the framework of. Saying you're an institution that has turned on AI and you have that basic version of Copilot is like talking to somebody in 2005 who's really excited that they're on the web, you just come to find out that they're on AOL. This idea of like, eh, that's, that's like, again, it's this little cordoned off, quaint piece of, of the internet. So, I do think that that's, the challenge we're navigating is this tool came out, in some ways it's ubiquitous and it's everywhere in three short years, unlike a lot of other technology, which was a longer span of time. But it's still not affordable for institutions to solve the equity piece of they're either working with a version that they know some students are going to be able to buy the paid version and be working with a very different set of tools. And I think until that gets solved, it, it just does, doesn't feel quite as useful. Identifying a tool, but also like having to, to plan for it. And this is where like I think looking back is there's really good points of comparison. We know all institutions. You know, were, were had computer labs and the internet on their campus instantly. It was a gradual layout. It required, you know, figuring out the funding pipelines. And we know also that like lots of things became more affordable. So, at one point the idea of giving every student access to like the Microsoft Office Suite was unheard of. nowadays, like you're either getting that or you're getting your Google suite. This robust set of tools, and that's because the price points have like figured themselves out. So, we're stuck in this place where it's, the technology has like become hyper fast and not everybody has access to it, but there's an expectation of access. So, of that turns into like what does it mean to be in the classroom and trying to figure this out? And I do think it is they, where they can. Get the most out of this is just having honest conversations in general, but really just looking about and figuring out how is this showing up and being used in the industry, in the fields. Because again, if we look at the internet as a point of comparison, the internet it's changed how every discipline operated. It's changed how every industry operated and changed in different ways. You know, we can be historians without ever having to go now to locations to go into archives 'cause those are archives are made available. And how we engage in archival research is different and understood differently. So, I think there's, there's something there of like continuing to, to surveil the horizons within their disciplines about what are the real ways this is going to be integrated or going to impact them and using that as a basis of discussion. Like it doesn't have to be just about AI in general. It can just be focused on what does AI mean for history, for nursing, for take your pick. [00:33:22] John Nash: And kind of seeing that in here we are, as we say, we sort of keep talking about how we're about almost three years in and in the last 45 days in my academic department, we've had the most substantive conversations about what it means to have gen AI in our classes and what we believe about it. And the tenor of the conversations are people who are just now starting to grapple with it along with people like me in the room who've been grappling for 36 months and thinking, and we have some institutional policies, but my colleagues sort of asked like, what should I put in my syllabus and what should I do? They almost wanted institutional direction, so they could, I don't, I don't want to say that they wanted to stop thinking about it, but it, but they wanted something kind of hard and fast. Like, what do I say and do? And I was maybe to some chagrin, you know, on their part sort of shrugging my shoulders because it's, it's not. it's not that exactly that simple. I can point you to our university's sort of stoplight protocol of a green, yellow, red, and you decide how you want that. But when it comes to, say, doctoral work or in dissertation work, or if you're a mentor to a dissertation student, then what will be your own personal ethos around this? And the, and then we go, "oh, well that's a bigger issue, isn't it?" " Yes, it is." And so, part of me wants to ask you what would you say, do these two things on Monday? But also, it's, it's also not always that simple, but you know. Yeah. [00:34:50] Lance Eaton: I agree, like, it is a deeply personal question of their own teaching ethos, their, their, pedagogical philosophy. and I think that can and should be the grounding. And also, like double checking of is there anything about that that needs to be updated. I was in a conversation recently with a faculty who was still insistent on, like, was drawing this, if they don't know how to do the citations properly and perfectly, like they're not going to be prepared for the boardroom. And I was just like; I haven't been in a lot of boardrooms. But I'm pretty sure nobody is worrying about APA. So, I think the first thing, Monday morning is recognizing. There may be things they care deeply about that may not be as important, and we're still trying to figure that out. So, it's not “throw everything out,” but have your own critical eye about what are things you're deeply connected to, 'cause they were meaningful for you, but they're not necessarily as useful in the world we are today. You know, I think about that with, with writing, like physical writing. Some people are, you know, really stuck on that going back to Blue Books and as somebody who writes horribly in my educational experience, right. But my educational experience... [00:36:10] Jason: there with you? Yes, absolutely. So bad. [00:36:13] Lance Eaton: my educational experience changed how I was perceived as a student changed when I went from handwriting to the computer. My thoughts are still the same, but how it was perceived, so, you know. Wanting folks to just recognize there's going to be things they may have to let go. And that's hard. That sucks. If you hold to it even tighter some things, you're going to lose more students. So, I think that's one thing. And then the other is like, play with ai. Until it does something that surprises you, that you find is really useful, that just like, huh, and it's not that then you have to go and use it that way, but like that experience and hold that in your head to just understand like why it would be powerful and appealing to others and to help you think about what are, you know, what. If it can do something meaningful and helpful to you, like where else might you be able to transfer that into the teaching and learning that you're doing? [00:37:16] John Nash: Yes, yes. [00:37:18] Jason: That's great. Yeah. And we're going to start to kind land this plane here, the. Some of the themes that I'm hearing from today that I'm hearing from you, and you can add anything you want to this, but, you know, we were talking about really, again, coming back to good pedagogy in the classroom. [00:37:39] John Nash: Yah [00:37:39] Jason: Just like, you know, we've been talking about this book, the opposite of cheating is learning. The opposite of AI is not. No ai, it's also learning, right? It's good pedagogy. It's, it's not just banning it, although policy could be part of that, but it's also, making sure that we are engaging with our instructors and our teachers on the design side and the teaching side Be having the conversation to talk about how to integrate it and like you were saying here, your example of just getting into it, letting it surprise you, learning about it. Whether, and I was talking to a, a group at Kentucky State Department just a week ago. And I was like, you know, I'm not saying you have to use ai. I'm not even here as an advocate necessarily of you using AI in your job. However, I do think that every one of you needs to understand ai, at least what it can do, what it's doing in the populations that you're serving, understanding sense of. Of not either overestimating or underestimating where it's at in the world right now, so that you can make your own decision on this and maybe something that you just leave behind and don't use, but at least you'll understand, right? [00:38:49] Lance Eaton: Yep, that's it. Exactly. [00:38:51] Jason: Well, Lance, thank you so much for joining us. This has been a great conversation. Yeah, we've really enjoyed this and hope to continue to connect with you. I think we all connected with you on LinkedIn was kind of a main place. And what's the best way Is LinkedIn probably the best way for our listeners to keep up on the, the happenings of Lance Eaton? [00:39:11] Lance Eaton: It would probably be my Substack, the AI+EDU=Simplified Substack that I'm often sharing recent talks, workshops, any materials that I'm building, just I like to put out there for others to draw upon, adapt. It's usually all, all of it is published with Creative Commons License so that other folks can, can build upon and like we can help. This out. So, thank you so [00:39:36] John Nash: Beautiful. Love that. [00:39:38] Lance Eaton: Thank you so much for having me. [00:39:38] Jason: Great. Yeah. Thank you. We'll put those links in the show notes. And for those listening, you can always find our show notes@onlinelearningpodcast.com [notes@onlinelearningpodcast.com]. That's online learning podcast.com [http://podcast.com]. Always put our show notes in transcript. John usually draws a funny cartoon and puts it in there. Just joking. [00:39:57] John Nash: No. [00:39:58] Jason: in there and [00:39:59] John Nash: well, now I have to start. Now I have to start. [00:40:02] Jason: See a sampling of my bad handwriting. We'll just put it all in there for everybody to see. Yeah. Thank you [00:40:08] John Nash: Cool. Yeah, Lance, thanks a ton. [00:40:10] Lance Eaton: Have a great day.

EP 38 - Canvas, Credentials, and the Agentic AI Classroom: Ryan Lufkin, VP of Global Academic Strategy Instructure

In EP 38, John and Jason talk with Ryan Lufkin of Instructure about the evolution of online learning, the impact of Agentic AI on education, and how Canvas is shaping the future of digital classrooms. See complete notes and transcripts at www.onlinelearningpodcast.com [http://www.onlinelearningpodcast.com/] Join Our LinkedIn Group - *Online Learning Podcast [https://www.linkedin.com/groups/14199494/] (Also feel free to connect with John [https://www.linkedin.com/in/jnash/] and Jason [https://www.linkedin.com/in/jasonpauljohnston/] at LinkedIn too)* Guest Bio: Ryan Lufkin is the Vice President of Global Academic Strategy at Instructure, where he works to enhance the academic experience for educators and learners, worldwide. With over two decades in the edtech world, Ryan has experience with every major technology platform that institutions use to deliver education, from the LMS to the SIS, and all the systems in between. A well-known thought leader in the edtech industry, Ryan is a podcast co-host, frequent media spokesperson, and speaker at industry conferences and webinars. Ryan earned a Bachelor of Science degree in Public Relations/Communications from the University of Utah and certificates in Data-Driven Marketing and Brand Management from eCornell. Resources: * Canvas LMS https://www.instructure.com/canvas [https://www.instructure.com/canvas] * EduCast3000 Podcast https://www.instructure.com/resources/podcast [https://www.instructure.com/resources/podcast] * Chole 10 Report https://qualitymatters.org/qa-resources/resource-center/articles-resources/CHLOE-10-report-2025 [https://qualitymatters.org/qa-resources/resource-center/articles-resources/CHLOE-10-report-2025] Theme Music: Pumped by RoccoW [https://freemusicarchive.org/music/RoccoW/contact] is licensed under an Attribution-NonCommercial License [https://creativecommons.org/licenses/by-nc/4.0/]. Transcript: We use a combination of computer-generated transcriptions and human editing. Please check with the recorded file before quoting anything. Please check with us if you have any questions or can help with any corrections! [00:00:00] John Nash: You, you ready? Jason? Anything else? [00:00:02] Jason: Nope. Just taking a drink, that's all. [00:00:04] John Nash: Alright, I'll let you do another one. [00:00:06] Jason: Yeah, that's [00:00:07] Ryan Lufkin - Instructure: Do the vocal exercises, you the [00:00:08] John Nash: yeah, me, Mimi. Red leather. Yellow leather. Yeah. [00:00:12] Ryan Lufkin - Instructure: Yeah. [00:00:13] John Nash: I'm John Nash here with Jason Johnston. [00:00:15] Jason (2): Hey, John. Hey everyone. And this is Online Learning in the second half, the Online Learning podcast. [00:00:20] John: Yeah, we're doing this podcast to let you in on a conversation we've been having for the last three years about online education. Look, online learning has had its chance to be great, and some of it is, but some of it still isn't. And so how are we going to get to the next stage, Jason? [00:00:35] **Jason:**John, that's a great question. How about we do a podcast and talk about it? [00:00:39] John Nash: Perfect. What do you want to talk about today? [00:00:42] Jason: Honestly, and I'm not just saying this 'cause Ryan's in the room, but one of our favorite ed tech tools, Canvas. And we're here today with Ryan Lufkin from Instructure to talk to us. Welcome, Ryan. [00:00:56] Ryan Lufkin - Instructure: Thanks for having me. I love these conversations. Looking forward to it. [00:00:59] Jason: Good. Why don't you just kind of describe the role that you play at Canvas? [00:01:04] Ryan Lufkin - Instructure: Yeah so I'm the Vice President of Global Academic Strategy which means I, I spent a lot of time talking about the trends that are impacting education across the globe. In that role, I travel all over the globe. Honestly, I was in Singapore and Meine Columbia and me. City and all over the United States this year. Talking about exactly the topics that you all focus on as well. How does technology impact learning experience good and bad? And what does that look like? And I've been within Instructure it's funny 'cause I always say Instructure the makers of Canvas because everybody, Canvas is a household name. Fewer people know the company name. [00:01:35] Jason: Right. [00:01:36] Ryan Lufkin - Instructure: But I've been there for seven years now. I've been an ed tech for over 25 years, and I just love the company, love our mission. I love the focus and so it's, it truly is a pleasure to be able to come out and have these conversations. [00:01:47] Jason: Do you, do you work with anything other than Canvas at Instructure? Are you kind of over multiple things there, or? [00:01:55] Ryan Lufkin - Instructure: It's honestly our entire suite. So I think a lot of people know that we bought Mastery Connect, which is an assessment tool. We bought Parchment which is a credentials tool which I, I've watched my kids use I've used myself to send your transcripts when you're applying for college and university and things like that. We've bought Badger the Credentials program. We bought Portfolio, which is a portfolio program. So we, we really, over the last 14 years have grown from just a single product company to a real ecosystem of solutions. And unlike, other companies, we don't buy our competitors we, we buy our closest partners and extend that, that that ecosystem. [00:02:29] John Nash: Yeah, so that's why you didn't buy Blackboard, you just decided to just destroy them. [00:02:34] Ryan Lufkin - Instructure: We, in doing so, we evolved the market, right? And I always say Canvas came along when we were still having arguments about whether or not education over the, put data in the cloud and never moved towards SaaS solutions, right? And whatever be, open source. We're technically commercial open source. We publish our core code on GitHub as well. So to, to give people that peace of mind that, you own the code, that kind of [00:02:56] John Nash: Mm-hmm. [00:02:57] Ryan Lufkin - Instructure: it was, it. Truly transform the market from that, your LMS systems that would go down every month when you got your little update disc in the mail to, 99.99% uptime. And when we have an outage, it's a big deal now across the industry, [00:03:10] John Nash: Right. [00:03:10] Ryan Lufkin - Instructure: Used to be super common. So it's been a bit of fun ride. [00:03:13] John Nash: Yeah. Yeah. Just that little bit you just described there, it really encapsulates how much has changed in just, , a decade or even, , 15 years. [00:03:22] Ryan Lufkin - Instructure: been in it, yeah, if you've been in that change, it you take it for granted. [00:03:24] John Nash: Mm-hmm. [00:03:25] Ryan Lufkin - Instructure: how much it's fundamentally changed over the last [00:03:27] John Nash: Yeah. [00:03:28] Ryan Lufkin - Instructure: And, education has the reputation for moving slow. And in a lot of ways, we're moving faster than people think. [00:03:32] John Nash: Yeah, you're right. When you're in it, you forget. And again, we didn't have you on, so I could talk about all the other tools that are out there that we didn't like, but we used Adobe Connect for a long time and that was a, an abysmal failure for us that we, we just couldn't get it to go. And so I mean, we're just thankful to have tools that are stable and up and, and then even now thoughtful about instructional design. Well, that killed the conversation [00:03:55] Jason: we say all that to say we, . We're kind of, we're kind of fanning on Canvas today because we do, because we, as you said, we have been around. I mean, I think the first, not, I think the first LMS I used was, I it was Blackboard in that early stage 20 years ago. and at that point it was so bad that I just made my own websites basically for my for my students, right? And we're kind of left to that and a lot of, a lot of institutions were doing the same thing. It's like, these are so bad, we're going to make our own LMS system. At the time Canvas and others too have stepped into that gap and created products truly that work well. They're responsive both in the web sense of things, but also responsive to the users. Continue to get better. Run without a lot of outages are secure. , There's a lot of, lot of things that go into these LMS systems that, as John said, , it's easy to take, take for granted in in 2025. [00:04:55] Ryan Lufkin - Instructure: Yeah. And , there were four kind of key pillars when they founded the company, they were simple, engaging, open, and reliable. And the reliability piece was really the software as a service co-hosted right. Partnership with AWS. But that openness piece is one that, that still remains a massive differentiator for us, we have. I think publicly we see we've got 600, over 600 open APIs. We actually have 847 or something crazy like that. And then we, comply with the common standards like the LTI standard, right? To make sure that if oh, we've got over a thousand partners that have developed LTI apps that plug directly into Canvas. And, we were very clear. We don't own the data. We are caretakers of the data. We don't own the experience. We facilitate the experience for schools as much as possible. We've done everything possible, not kind out of. Create that walled garden that I think, other vendors that you may have mentioned earlier, I really tried to protect and box out competitors. We work with coopetition on a daily basis, right? And embrace terms like that, right? And so it's, for us, it's all about how do we facilitate our universities and K 12 institutions to, to build the. learning experience that they want. And whether that's working with our friends or more tenuous friends, we, we support that. [00:06:04] John Nash: Yeah, so Ryan, we're excited to talk about instructional design, thinking about online assessments thinking about where AI is going to play a role to lighten the workload, but not turn courses into some automated thing. But I think we would be remiss if I didn't start off with something that's kind of like the elephant in the room, which is where, , instructors are asking us all the time about AI agents and we've even done an episodes on it, , logging in, doing coursework. And in a recent episode of your podcast EDUcast 3000, put a little plug in there for you, I listened to your converse. Yeah. You and Zach Pendleton talked about what you learned at EduCause 2025. And I heard you talk about something I hadn't really thought about, which is this something called model context protocol, MCP as a way to give AI a safer, more controlled access to systems. Is that something you think could play a role in the LMS world, at least in addressing these concerns that teachers have about agents logging in? [00:06:59] Ryan Lufkin - Instructure: Yeah, so you're not familiar with the term MCP it, it essentially rolls those 847 open APIs that I talked about into a more cohesive and again, I'm not, zach's the super technical one, but into a more cohesive approach. And when more about AI, it's garbage in, garbage out, right? The more organized your data is, the better access those AI tools have to that data. The better your results are. And from a security standpoint, from a. From an accuracy of output standpoint, from a, from an agent standpoint, making sure that they can accomplish the tasks they need to, having an MCP is really important. It just organizes the entire experience. It also makes integration with third parties easier. And again. One of the things I always like to clarify and it's something that Zach and I came up with three years ago, right on the cusp of, the AI revolution, November 30th, 2022 and when they launched OpenAI we really came up with the, this concept of eating our vegetables, right? We have all this regulation in place and there was this immediate. We got to pass new regulation, we got to do this. Nope. We've got privacy, student data privacy covered, we've got accessibility covered, we've got security covered. We just need to make sure that we eat our vegetables and make sure these tools align with those existing regulations. We don't need another layers of regulation specific to AI' cause it exists within the technology we're already using. And so that's something that we've really focused on. And so when we talk about MCP and we talk about integrations. All of that is done with the permission and the full disclosure of our universities, even down to the educator level. So you can turn on AI features within your course. Things like that. We don't ever turn them on for everyone and plug data in. I think that's everybody's fear is that we're going to plug ChatGPT into Canvas and it'll suck all the data out. [00:08:33] John Nash: Right. So, but when we think about this concern that instructors have about agents logging in and impersonating students, is, is MCP something that an instructor would invoke, or is it something that that instructor or the LMS has to put in place to say, we won't let agents log in on behalf of students or say, like Comet browser is going to, , roll in there and take the course? [00:08:57] Ryan Lufkin - Instructure: Yeah. It's funny because before, before we started this conversation around AI the thing was, we were having Chinese Foreign Exchange students log in and take our courses for us, right? This has been a problem as long as, and it was a problem when we had a physical classroom. We had people show up. You would have to show your student ID to make sure that you were who you said you were when you were taking your final. Things like that. This is not a new problem. And I think there's a lot of times where there's a, there's a level of anxiety or belief that this is happening more than it needs than it is within education. The vast majority of students actually want to learn. And I think one of the things that AI is driving is this explanation of the why. Why do we do these things right? And if we explain the why, if we understand that like this, knowledge builds on itself. Students are less likely to turn to cheating. But back to your question. Yes. our goal is to catch it at the system level. Right? If we're catching these bots coming in at the system level and performing these tasks, there's we've had some institutions that have had problems with bots coming in and applying for financial Aid, right? And they're. The AI tools are good enough to actually enroll in the course and go in and do enough activity in the course to pass those standards. And so it's a constant cat and mouse game. The technology advances, we figure out ways to try to block those or identify those users. But at some level the, it's an ongoing battle that we hope to get the upper hand on all the time. [00:10:16] John Nash: We understand that this is not all at the feet of a company like Instructure. That if a, if a bot is going to log into a course and succeed at it, and I played with a little demonstration project to show how that could happen. It's really a more, a matter of is what's the instructional design of that course such that a bot can get in there and succeed in it and get As. Also, I think, Jason, you've pointed out that in a smaller course, this would be sort of catchable but in a larger course, it could be a bit of a challenge if there were 60, 70, 80, 90 folks in there and it was sneaking in, but as you point out, this could, there also could be a proxy human being doing it too, depending on where they sit, but. [00:10:56] Ryan Lufkin - Instructure: Yeah. Ultimately if a student wants to cheat and do that, it, they're just robbing themselves of that experience, unfortunately. And so is that kind of ongoing battle. But it's existed since education started, somebody sending in a proxy and having them learn for them. And it's an ongoing effort. It's something that, that we spend a lot of time, one, one of the funny things about AI is we continue to uncover new use cases that we wouldn't have imagined before. And so somebody will come with a, with an example and we're like, "Wow, I hadn't thought about that one. That's a good one. We'll go track that one down." And so we, we have a constant, a team that constantly works on that. [00:11:28] Jason: Yeah. And we've been talking about AI since we've had this podcast, it's been a constant topic. But, this fall particularly, we've been talking a fair bit about agentic AI, So as John mentioned, , we've done some testing. So currently right now, so this is for those listening in the future. This is December 5th, 2025. We know things change very rapidly. We've currently done some testing with , both the Perplexity Comet as well as Atlas, in Canvas specifically, where it can go in and this point you don't even need to do a workaround with these browsers. It used to have to do some sort of work around kind of like. Oh, this, I'm just testing this course and you're going to do it. , So it doesn't refuse to cheat. They'll just do it for you now. To respond to, , a quiz. Get nine outta 10 or 10 outta 10, do discussion post email the professor through Canvas. So this is current right now, so I guess my question is then what do you expect to see right now with it? Kind of what your stance is, what's in place, and then where do see this going? Are there changes happening in terms of, of Canvas, , in terms of their approach practically? As I think about from a teacher standpoint and a, in a student standpoint. [00:12:49] Ryan Lufkin - Instructure: Like I mentioned, we are, we're constantly working on this and we have really great relationships with Google and Microsoft, and we work with them as well. 'cause some of the, one of the first things we saw was Google Chrome plugins that were, "Hey, we'll take your quiz for you." And so then we've got to reach out to Google and say, Hey, maybe you could disable this plugin because it is not even hiding the fact that it's cheating, and so it's not even within our product itself, because if they're plugging those things into the browsers that that's giving access into the system. And. There's still, if you I'm also a student I'm doing a master's program at Arizona State University, and so I'm actually looking at how they're tackling some of this from the student side as well, which is just fascinating. But ultimately, within a lockdown browser experience for finals, you've that's still, you've got a turn on lockdown browser write your paper directly within that tool. There's other tools that are coming out where they plug directly into Canvas, and then you have to write your paper directly in there and it tracks every aspect of it. And that's really hard to game with an AI tool because it's very clear, it's being written by AI. and so there's a number of tools. It's a constant cat and mouse game of us trying to battle this. But what's interesting is when you're also in the workplace where like the exercise that you just went through to build a agent to go in and do this test, of people are doing that within the workplace to accomplish their work more quickly. Building that tool is actually like really interesting skill that we're also probably should be teaching students, which is, seems self-defeating except that these are tools they're going to going to need in the workplace. And so there's nothing I hate more when I hear educators say, "we're going to go medieval on 'em, we're going back to blue books and pencils" because that's just not preparing students for the future. And we'll continue to play our cat and mouse game and try to block the, the. Use of these tools and the fair use of these tools. But we also need to evolve on, and John, I think you mentioned this a little bit, we also need to evolve how we design our courses. And if they can be gamed by AI in a lot of ways, we've got to figure out how to prevent that and turn that on its head. And I think that's what we've been a little slow with is. How do we give the resources to educators to understand what these tools can do? Basically, AI literacy, so they understand that this is a problem, can be a problem. How do we actually help them redesign their courses and give them instructional design resources to modify how their course is designed to help prevent this type of thing? And then how do we the tool provider, continue to fight this battle? So it's a multi multilayered evolution that we're all going through [00:15:08] John Nash: Mm-hmm. [00:15:09] Jason: Yeah. And we just released a podcast with a guest who was talking about kind of a multi-pronged approach, which we are completely behind, like redesign. This is, I work with a group of instructional designers at University of Tennessee, and we're talking about this every, every week, right? And we're talking with faculty every week. And the great thing about that aspect of this is a, this is actually an. Incredible opportunity. we find that over and over again, we are coming back to strong pedagogical principles, right? [00:15:41] Ryan Lufkin - Instructure: 100%. Yep. [00:15:43] Jason: just want to build in anyways. How do we make it more engaging? How do we make more [00:15:46] Ryan Lufkin - Instructure: Yeah. [00:15:47] Jason: How do we scaffold more? How do we focus on process? How do [00:15:51] Ryan Lufkin - Instructure: How do we make it more personal? Yep. [00:15:53] John Nash: Yes, yes. [00:15:54] Jason: Exactly. How do we connect better? , So those are, I believe in those, but I also wonder about this kind of multi-prong, so focusing on those aspects as well because , Canvas's business, kind of like my business, is online learning and right now I don't see a really strong blocking approach, a technology like blocking approach right now. And I don't know if it's because it's just not possible to do it in this cat mouse kind of thing or is it from more philosophical, or is there, I get concerned sometimes 'cause I know Canvas is also has agreements with OpenAI and so on, and I wonder about some of those kind of things. Can you speak to any of that? [00:16:36] Ryan Lufkin - Instructure: That's what I'd love to dispel that last one aggressively. We work with every AI provider right from Anthropic to Microsoft to Google to OpenAI. We don't sell data, they don't have access to our student data. That's one of those things where hard and fast, and it limits the ways we can use AI within the classroom quite a bit. Because you cannot train, we will not let you train through our technology we will not let you train. AI model on student data. Period. Period. Hard stop. And also where I worry about some of the startups that, we go to ASU GSV every year in San Diego and there are startups making promises that you're like, "you can't deliver that functionality unless you are training on student data. And that's a FERPA violation. And do you what FERPA is?" Like the basic things there and that, so that's why we're always very. We are the gatekeeper in a lot of ways for, all of those vendors that want to work with us because we have over half the, students in North America using Canvas on a daily basis. And so how do we make sure that we are the gatekeeper to make sure those aren't, those tools aren't being used nefariously and we are absolutely the gatekeeper there, right? And we take that role very seriously. But I think to your point is it's not, it's not philosophical. If you look at lockdown browser, like lockdown browser makes it so you can't access anything. But that test, and it is a great tool for eliminating the ability to cheat. It is not a great tool in providing any sort of engagement or interactivity or a good experience. So for us it's, we're caught in this conundrum of like, how do we track these, how do we block the use of these tools nefariously, but also allow the freedom to create really engaging courses and personalized courses. And it's. It's not binary. We've actually, and that's where that cat and mouse comes in of constantly going back, building the MCP. One of the reasons we built the MCP is because we thought people were using some of our APIs in ways that they shouldn't. Maybe we are being too open and we need to shut some of that down and control some of that a little bit. And not to prevent our schools from using it, but to prevent any type of nefarious use in the future. Things like that. And so we are constantly looking ahead and saying, okay, what is possible? What is the next step? What is the, maybe the things we haven't thought of? And how do we make sure that we're protecting for that? But then how do we also make sure that we're, were encouraging the use of these tools in productive ways within the courses. And I was at the, ASU Agenic AI and the Student Experience Conference a couple of months ago, month ago. I travel so much. It's a big blur. But it was so interesting to see what some of these schools were doing. Like Florida State University was embedding a NotebookLM in every course to enable. Yeah, if you're familiar with NotebookLM, it allows you to, it organizes all of the content within that course and then allows students to actually create a podcast to create study cards, right? To listen to elements as a stream, as a narrative, right? It provides a level of, personalization of content consumption, right? That really is the future of learning, right? You're meeting every learner where they are because you're letting them choose how they consume some of that content. Schools like Arizona State has actually created their I think it's called Creator Up. And it's the ability to create an agentic for every educator to create an agent of their own and apply that in different ways. And so we're starting to see all these different usage of the tools in creative ways within this kind of secure context that Canvas can provide, be the gatekeeper for. But we can't get there and we can't start exploring those tools if we're, like the letter that came out earlier this year, but it was signed by 7,000 educators that said, AI is corrosive to learning, and it was, it was, so we're going to bury our heads in the sand and pretend that we can make this thing go away. And that's just not realistic. And it actually prevents us from finding these positive uses because we're so fixated on the negative aspects of these tools. [00:20:14] John Nash: Mm-hmm. Jason. And I think a lot about how to improve instructional design broadly, and so many instructors. Know their content so well, but they don't always know what constitutes strong instructional design. And I think as we're thinking about what the future of Canvas looks like, and I'll speak just personally for myself, I would love to see ways in which Canvas might even support my ability to do instructional design. So what's, , what's the landscape look like there, the, horizon for that kind of thing? [00:20:47] Ryan Lufkin - Instructure: Yeah. So one of the things, and we've rolled out some features like translation and rubric development and different features that save educators time, that's where we've really focused. But one of the things that, Zach Pendleton, who you mentioned earlier, who's our chief architect and one of the smartest people I've ever met I meet with him once a week and I walk away from every one of those meetings smarter for it. But one of the things he kind of realized early on is, look, we have an ecosystem of solutions that we are supporting all the time, but we are not. The AI creators. And so if we actually go out and partner with all, with Anthropic, with OpenAI, if we become agnostic in that approach, can only develop, we can only progress so fast 'cause we're supporting all these different tools, but they're really just focused on their AI tool and so they're progressing so much faster than we could if we partner with them and then allow our schools to work seamlessly through them, that's amazing. And that's one of the reasons that we partner with AWS because AWS has their bedrock models, right? They've got, I think, 27 different large language models, Claude Anthropic, all the ones that we've talked about Gemini, all of those that you can plug in through AWS that provides that additional layer of security and control, right? And so that's. AWS, Mary strain from AWS and I are besties, and you just spend a lot of time trading information and then covering these like different pedagogical use cases. She just sent me one from a school in Maryland. And we are, we're constantly looking to uncover that and help show schools what good looks like. I think John, what you were talking about with it seems very. What's, anything's possible go out and do these things, but if you show someone good looks like, or what some examples are, that really is a good starting point for how do I start applying these tools in interesting ways. And so, at some point in the future, there, there are partners that are almost virtual instructional designers and things like that. [00:22:32] John Nash: Yeah. [00:22:33] Ryan Lufkin - Instructure: And right now what we've done is let them run. 'cause they're going to evolve faster than we probably could with the resources we have, but then that fits into our model of you. We might buy them down the road and integrate those pieces in. [00:22:43] John Nash: Yeah. [00:22:43] Ryan Lufkin - Instructure: And you see great partners like Praxis. I'll give a plug to Dave from Praxis. They've done a great job with that and using it in different ways. And they're half. They're inside the tent. They're in, they're part, they're one of our partners already. They work really closely. They're at all of our events. And it, that's probably the model we'll follow is find those best of breed tools that are already halfway inside the tent and then pull them all the way in. [00:23:04] John Nash: Yeah. 'cause I guess, and Jason probably knows more about it than I do, but I think, and now I'm just speaking as my, just my user hat on. It would be nice, I think to have a way for Canvas to help a, a novice, face-to-face instructor coming to the online space to avoid producing shovelware and having an adept instructional designer riding alongside as they bring that course online. And I think that for me, this John Nash talking, I think that would be kind of cool if it could work well. But Jason, you, you run a whole shop of instructional designers. I don't know what, what would you like? [00:23:42] Jason: Yeah, I would welcome that because, , we always say there's always more work to be done, right? Like, [00:23:48] Ryan Lufkin - Instructure: Yeah. [00:23:50] Jason: I have no concern about our longevity in these roles and also for my instructional designers because partly 'cause I have really good ones, but also because so much work to be done. One, there's a lot of faculty that would just prefer to work with a person. Instructional designers are really good. They see around things in different ways. there's a group of faculty that would prefer just to basically do it on their own and sometimes do it between 11:00 PM and 2:00 AM in the morning, right? [00:24:21] Ryan Lufkin - Instructure: When you'd rather be asleep? Yes, yes. [00:24:23] Jason: And we're not going to be there. Our IDs are not being scheduled for one-on-ones during those times. And I think that, especially if they are tuned in a good way. And as you said, not just doing shovelware and able to give feedback. , We joke sometimes about Clippy, , about having a little Clippy, kind of pop up and say, did you mean to do that? [00:24:45] John Nash: Right. [00:24:46] Ryan Lufkin - Instructure: I like the, what you just said about they need to be tuned, they need to be monitored. I think there's a lot of, I, this kind of misconception that AI can just be set and it's off on its own running and you don't have to look at it anymore. You still need somebody a, an expert, an instructional designer to be monitoring. What the outputs are. Is it staying on task? Are we seeing drift? Do we need to update the parameters around this? Oh, we didn't think about this aspect of it. We got to add that, right? These are tools that need monitors that need bosses, right? And so I think that's often funny. We're like, "oh, fire all instructional designers and. And replace 'em with AI." That's not realistic. That's just not something that, that we see happening anywhere in the near future. They're about the scale, just as you said. They're scaling and providing that, that support when there's not a human available. [00:25:30] John Nash: Yeah. Yeah, yeah. I think a big question for instructors is how AI can lighten the workload without turning courses into automated experiences. How do you think Canvas is thinking about that balance between. In being efficient and maybe, , while we're all about the preserving the, the human aspect of teaching and learning. [00:25:51] Ryan Lufkin - Instructure: Yep, what's interesting, first pass grading is one that we've talked about a lot, right? And then we demoed the feature at InstructureCon this year, and it is, you set up your rubric and then the tool run, the AI tool runs the assignment against that rubric, right? And it gives a first pass of did they meet the criteria? Here's some basic feedback. And the goal there really is to provide a first pass, and especially for a large course, if you have 90 people on a course, thinking about the time saving of having it go through and do a basic first pass for you, and then you as an instructor being able to go through and say, oh, actually I'm going to change this. I'm going to change that. I don't, I think it gave them too much credit for this. I think I'm going to reduce that and, but it provides that framework as a starting point. So it reduces grading fatigue. It reduces bias. We see a lot of, if your letters, if your name starts with Z, your instructor's really tired by the time they get to your paper, and they're not necessarily going to give you as thorough an overview. But what we don't want it to be is just that robots grading robot submitted work. And so what it does. You even have data, so you can actually look at, from a program level, you can look at are your instructors actually just going submit? Or are they actually going through and changing them? Because it's designed to save time and not to just replace that process. [00:27:03] John Nash: A comment and then a follow up question on that. The, the comment is, I was reading in the, this says a lot about me and maybe how dangerous I am, but I'm in the subreddit for professors on Reddit ,and there's they, in that subreddit, professors complain a lot about AI submission and that, not that students are necessarily lazy, but that's, there's a tone of that there. But something a professor said that I that struck me is it said, I think that I'm going to ask my students when they turn something in, I'm going to ask them how many minutes they would like me to spend grading it. And it and it sort of made me think about, well that's interesting because you're right. If they're just going click, click, click because I'm getting a bunch of AI slop and so I'm just going to, , or I'm going to spend time thinking about this with you. And so it'd be interesting if the tools could really help professors really get into, we talk to a lot of professors who love that aspect of marking papers because they get to learn about their students more. They learn about their interests more. That's just my comment. I think that my question kind of gets into the weeds that it's neat that it'll make a rubric, but doesn't that presume that you also have good learning outcomes? And so is there developmental work that can happen for professors before they get to that point? 'Cause it's easy to think, "oh. I'll just make a rubric." But I, I don't have good learning outcomes, then the rubric's no good, right? Is, yeah. [00:28:22] Ryan Lufkin - Instructure: And there's a number of things that we do to support outcomes, mapping outcomes, and, yeah, what's interesting is you can actually use if you have your course content and you can actually use some of these tools to help you define outcomes [00:28:34] John Nash: Yeah. [00:28:34] Ryan Lufkin - Instructure: some of those things, [00:28:35] John Nash: Yes. [00:28:36] Ryan Lufkin - Instructure: And so there are, this is where AI is both the cause of, and the solution to all of our problems. Not to pull a Homer [00:28:42] John Nash: That's a Homer Simpson. Yeah. [00:28:44] Ryan Lufkin - Instructure: Yeah, exactly. But it is in so many ways, right? And because it is, I think Jason, you mentioned the Socratic method and this like getting back to this, like one-to-one approach that real human connection of learning. And in a lot of ways AI is able to help us get closer to that if we do it right. That personalization piece I think is important. And yeah, it can actually help you create course content. But again, one of those things are, is that a tool that we should build within Canvas that we may not be able to update as fast? Or is there a really good third? Is actually Anthropic's Claude's Learning Mode does some really interesting things around Socratic, where it does won't give you the direct answer, but it answers, Now, OpenAI has their, I think, student mode that does the, something similar, right? There's these tools that they're innovating faster and that's why that open architecture allows us to move more quickly realistically than if we were. Building all of these features ourselves especially as we move into more agentic phase. And these tools are more powerful. They can plug into the, into Canvas a little more deeply. And then it'll be I say a little bit like some of these features that we've developed will be we built them in Flash, right? They'll, we'll retire those and it'll just be the agent throughout the tool that is supporting that. And I really do think we'll get there. We've got to, we've got to build the trust. We've got to. Build the understanding of what it's capable of. But I also think AI is great at detecting other AI. If we get to the point where Canvas for has never really done we've always relied on our partners for academic integrity. That was a decision early on that we were not going to, be aggressive with academic integrity. We've got great partners like Turn It In and things like that, that that really focus on that CopyLeaks and some of those others. And we have not focused on that as much, but in the future with, these different models, you may see that evolve a little bit. [00:30:26] John Nash: Yeah. What are you thinking, Jason? [00:30:29] Jason: Yeah, I, and I love that idea of that first pass, especially as I'm thinking about larger classes, and the load that that takes both for teachers and TAs and was in a seminar earlier this summer are with some people from ASU and talking about some of these larger classes asking the question, , what if we could focus on the students that really want our feedback, right? And, , those are some of the things I really like . About AI from a grading standpoint. Thinking about from a teaching standpoint, because my students don't need me to correct another one of their commas or split infinitives, right? I don't need to be doing that. They don't need me to be doing that. It's not personal. It's not anything. It's something they should be learning. And they should, especially if they're going to go on higher education, they should be correcting themselves and learning along the way. But I don't need to be spending my time with that. However, I do want to be interacting with them over ideas. And this is where the line comes from me, is like, , I played a little bit with AI creating comments for students, and it just felt kind of icky. That's my technical word for it. It [00:31:39] Ryan Lufkin - Instructure: Honestly, soulless. [00:31:41] Jason: Yeah. Yeah. [00:31:42] Ryan Lufkin - Instructure: I've heard soulless is a version of that? Yeah. [00:31:43] Jason: It just felt wrong. Like now I was like, I was a being of an imposter teacher, , that I wasn't really, [00:31:52] Ryan Lufkin - Instructure: Yeah. [00:31:53] Jason: For me, it kind of crossed over from that, oh, yeah, here's some, here's some technical things I've now given you feedback for from AI, and this works fine points on this and that, and using a rubric, kind of like that aspect. But now that I get talking to him, Hey, good job, Billy, or whatever, it all, all of a to feel a little weird. [00:32:10] Ryan Lufkin - Instructure: Feels insincere. Yeah. The other piece too is there's, we talk a lot and I see this, I, I have a, I mentioned my daughter's a junior at university, just as we were coming on the show. And my son's a freshman in high school. And they both use Canvas. And I actually, what's interesting is some of the frustration they get around the timeliness of the feedback, right? I submitted my paper two days ago and my professor hasn't given me my grade yet. How why not? Why not, right? They're a generation that's, everything is so automated and so on demand and immediate satisfaction. And it's really frustrating for them when it takes that long. The other aspect too, and I've seen it not to call out my professor at my master's program, but where I didn't get feedback on my, my. First paper, and I'm already submitting my second paper, so how can I provide a second, am I going to get ding for what he graded me for, but I haven't gotten it before I write it, right? And so there's some simple basic realities that we need to make sure we are addressing. And instead of villainizing AI or saying it's robots providing robots responses. That timeliness and that is, providing feedback in a useful way. So I'm building on that feedback is really important and I, it's not always easy for educators. It's, I don't blame necessarily my professor, but it is, it's frustrating on the student end. [00:33:21] John Nash: This is not the solution, but what you just said in the experience you're having and your children are having, reminds me of, of what happens if any of either of you have used the app on Domino's to order a pizza, but when you, you order the pizza and then it says, Jane has received your order, and then it says, Bill has put it in the oven. And then it says Sam has stuck it in the car. And so I can imagine like John has received your paper and then John has opened your paper and then, [00:33:50] Ryan Lufkin - Instructure: John Expletives about being tracked by this device. Yes. [00:33:54] John Nash: yeah. Ryan, you mentioned badging and Badger was the tool that you bought. Our institution's been talking about badging. I've heard you, I think in other spaces talk about micro-credentialing about credentialing being detached from courses. [00:34:10] John Nash: I think about it in my course, I teach a relatively complex project-based course that teaches students how to be design thinkers. And so there's opportunity I've seen for micro badging inside that. So even if you didn't finish the course, you could have left being an empathetic interviewer or a good brainstorm or what have you. So talk to us a little bit about what direction that's going in and where we might leverage that as instructors. [00:34:34] Ryan Lufkin - Instructure: I love that. And we talk a lot about skills-based learning, [00:34:36] John Nash: Mm-hmm. [00:34:37] Ryan Lufkin - Instructure: which is, we hear pushback on skills-based learning. 'cause people are like, we're not a, we're not a vocational school. Our job is not to prepare people for jobs, right? There's still some of that mentality floating around out there even in the face of some of the, federal requirements now around are people getting jobs with these degrees? Things like that. And I don't think that's the end all. I, I don't want to come down on either side of that debate. But, 40% of college students don't graduate within six years in a traditional four year program. And they leave with essentially nothing. But like you said, John, the, they could actually leave with, I actually am a good presenter. I am a good, I've got a badge and evidence-based decision making, things like this. [00:35:13] John Nash: Yes, yes. [00:35:15] Ryan Lufkin - Instructure: And I don't think it's we've got in no other industry would somebody spend potentially two or three years of their lives and leave with nothing for what they pay. That's just, it's just insane. And how do we break that down? How do we actually start providing that incremental credit or things like that. But even then my daughter, she's a strategic communications major, she's getting a minor in psychology and she wants a certificate in data analytics, so she can show look, it's not just soft skills. Let's, I've got some hard skills. what's interesting is that certificate in data analytics is an, is non-credit program, right? Because we are caught up in this traditional model of your program is accredited, right? Because it is a full program. Most certificate programs are not accredited if they stand separately, right? They're for adult learners they're not, not for credit or they're treated separately. And so we've got to get to this model where we're looking at it all as a skills-based framework and giving students credit for that so they can then properly show to potential employers or other educational institutions that they have those skills. They've achieved those skills in measurable ways. And there's a lot of changes within the infrastructure of education that need to happen. But what we've seen is this massive growth since COVID of. Of demand for credentials, demand for, Hey, if I'm going to grow in my job, if I'm going to, if I'm going to switch careers, if I'm going to change something, I need something. And it doesn't need to be a four year degree necessarily. It could be a six course, certificate that would help me get a better job or help me get better pay. And we're seeing that across the globe. We've seen initiatives for that in the Philippines around tech jobs. We've seen in Mexico a very similar program where they're going to try to drive more non four year degree programs to upskill their labor force. And so this is a global initiative and we have the ability to be leading it just as we've lead the, the model for education, we've led the model for education across the globe, for the last 150 years. We have the opportunity to be leading this and we need to kind of lean into that. [00:37:09] John Nash: Could I now 'cause I'm too lazy to go look it up and I have you here. Could I now have a students in a full 16 week course that would complete a module and be badged for that module? Yeah. [00:37:20] Ryan Lufkin - Instructure: Totally. You can set up with, we've rebranded it was Canvas credentials. They've just rebranded it as Under Parchment, so it's Parchment Digital Badges. But yeah, you can actually set that up at whatever level you want as granularly or that could be six courses and you get a badge. Could be a badge within every module within a Canvas course. Things like that. [00:37:38] John Nash: And I'm the authority on that badge? That doesn't have to be my institution. I mean, I'm just like, I think I've looked squinty eye at all this, you look good on this. You get a badge. Yeah. [00:37:47] Ryan Lufkin - Instructure: Yes. And the goal really is to provide a level of flexibility around, because every institution's kind of trying something different. The challenge is the value of that badge then comes from your reputation and your institution's [00:38:00] John Nash: Right. [00:38:01] Ryan Lufkin - Instructure: and as opposed to a unified skills taxonomy [00:38:03] John Nash: Yes. [00:38:04] Ryan Lufkin - Instructure: or something like that, where there's an agreed upon [00:38:06] John Nash: Yeah. [00:38:07] Ryan Lufkin - Instructure: we talk a lot about that currency, [00:38:09] John Nash: Right. [00:38:09] Ryan Lufkin - Instructure: that there was a reason that I went and took a GoogleAI essentials certificate because it has Google on it. [00:38:15] John Nash: Right? [00:38:15] Ryan Lufkin - Instructure: I could have done that at, Salt Lake Community College or somewhere else. And, but it's the value [00:38:20] John Nash: Yes. [00:38:21] Ryan Lufkin - Instructure: that's what I think is so in flux right now. [00:38:22] John Nash: Mm-hmm. Yeah. It's like, oh, they got that from Nash, that "hack," Nash. So that badge not worth it. [00:38:28] Ryan Lufkin - Instructure: Or, oh my gosh, you got that from Nash? Okay, [00:38:30] John Nash: right? Yeah. Okay. Yeah. Right. [00:38:32] Ryan Lufkin - Instructure: that's the, [00:38:33] John Nash: Yeah. Yeah. [00:38:35] Jason: like only Gotten, have achieved that from Nash. [00:38:38] John Nash: That's right. [00:38:39] Jason: Yeah, I think that's an exciting direction. , I was at a session with if you're, you're probably familiar with Chloe 10 the most recent Chloe report. And there's just an enormous among leadership universities, an enormous focus on pouring resources in, into non-degree credits. And they didn't say badging, but I'm assuming that badging is going to be a big part of that. How do represent those? [00:39:05] Ryan Lufkin - Instructure: Yeah, badging is the. Badging is the external representation of those non-graded programs. And you hear a lot about stackable credentialing and that's why that analogy of my daughter, they really are looking at it as a stackable way as opposed to a degree that is just two dimensional in a lot of ways. How do I stack up these smaller groups that shows, look, I'm really good at data analytics and I lead more on the creative side. 'cause look what I've done some visualization stuff, right? It's just a. It's a more granular way to define your skills or show your skills. And it's exciting. I think it's nice to see some, we've been talking, Badger, I think we bought Badger six years ago or something like that. Like it it's, we've been working towards this and it's just really gained momentum. I think post COVID is, everything's moving a little faster. [00:39:49] Jason: Well, it seems like an appropriate time to make the UHF joke. Have you guys watched UHF "Badgers? We don't need no stinking badgers." Anyways, great films side comment, but... Maybe as we're kind of wrapping up here a little bit, I would love to know from you, from your viewpoint, where are we going? This is online learning in the second half. A lot of what we talked about here is, know, we've, we've been at this for 20 years. We're looking at where online learning has been. We can, we can get information in front of students, we can have them connect with it securely. Now there's ways to interact. do you think we're going in the next year, five years, 10 years? When it comes to online learning in general, or specifically with Canvas, some of the aspirational goals you have. [00:40:37] Ryan Lufkin - Instructure: Yeah. I'll go back and say we are, we've all been in education long enough to remember, or and many institutions still have that same, we are not a business, we are academia. They are not customers. We owe them nothing. And it's been really interesting to see this evolution towards more student-centered approach. And I think what we're going to see is a more personalized experience for students, more student agency in what they're able to take. We see this in things like the California community college system, right? Where you can actually take a course from any California community college and they've got a great pathway builder that shows how you take these different courses from any physical institution, right? And that more of that flexibility and interoperability and our goal, canvas today has about, about half of all college students in North America, about a third of all K 12 students in North America, and we're supporting more businesses with their development as well. And so when you actually look at the benefits of, it sounds a little self-serving, but it is also when you look at the benefits of that technology just disappearing in the background and the cognitive load of learning a new technology being removed and providing a very seamless experience. is really powerful. And then when you actually had tools like, okay, we'll support dual enrollment. And so you're having a seamless experience when between your high school and college courses and oh, we make it super easy to apply for college directly within Canvas and actually maybe recommend what that looks like and where do your aptitudes lean towards? The more we can use technology to. Keep students engaged in education, the better off we will be. The dropout rates that we've seen, and again, I'm old, I, my first weed out course, they said, look to your left, look to your right. One of these people isn't going to be here at the end of the semester. And they took pride in that. I love this the shift that we're like, how do we provide services to really keep students on track and moving towards success. I think education has the biggest impact on the positive. Aspect of the globe, right? The positive growth of our society, everywhere in the world. And so I, I think it's incredibly important. So you'll see more of that. How do we pull together an ecosystem that supports universities? How do we help universities work together? How do we help create corporate? Corporate university partnerships to make sure that the programs that universities are offerings are the ones that map to, to paying jobs and in demand jobs, things like that. And so it's hard to say what it looks like 'cause it's, I wouldn't have predicted AI five years ago, but and. We see so many of that quick evolution, but it's amazing to be part of this, like the most tumultuous but also the most transformative time in, in the history of education. We all get to affect that, and that there's a, there's kind of an excitement and a power in that that I, I love. [00:43:03] Jason: Love that and obviously the excitement in your voice. See it in your face [00:43:08] John Nash: Mm-hmm. [00:43:09] Jason: our podcast listeners can't see that, but I can tell you're passionate about this and it's been great to talk to you about this. 'cause we, we line up, and this is why we like our edtech partners, right? We don't pretend to be able to manage all this alone. , If it was, honestly, if it was left up to the institutions to the LMS, we, I don't know John, we'd probably still be having students dropping things into Microsoft office files or some, or [00:43:35] John Nash: yes, [00:43:36] Jason: like that on the [00:43:37] John Nash: we'd be, we'd be using Evernote and Dropbox. [00:43:40] Jason: yep. [00:43:41] Ryan Lufkin - Instructure: Dropbox. Oh, yes, I, And I love these conversations and thank you for having me on the show. I just think the more that we, we have these conversations and we explore different ideas the better off we are as a community. And so this is, I appreciate what you're doing. [00:43:52] Jason: Yeah. Well, thank you. [00:43:53] John Nash: Oh yeah. Thank you. [00:43:55] Jason: For those listening, we're online learning podcast.com [http://podcast.com]. That's online learning podcast.com [http://podcast.com]. So you'll see the full transcript notes there. We'll also put some links in to EdCast 3000, which love the reference there. And if you don't know the reference folks well, [00:44:10] Ryan Lufkin - Instructure: Special kind of nerd reference that we like. [00:44:12] Jason: a bunch of nerds here. Basically, we've pretty much admitted that, [00:44:15] John Nash: Totally, totally. [00:44:16] Jason: is in a sub sub Reddit on on But. [00:44:20] Ryan Lufkin - Instructure: John also got my Homer Simpson, my Homer Simpson quote, so I appreciate that as well. [00:44:26] Jason: We'll put a link to that and I, I guess we'll put it a, a link to Canvas. I think probably most people listening know about Canvas, but [00:44:34] John Nash: Yeah. [00:44:35] Jason: we might as well put a link in [00:44:36] John Nash: No. Yep. [00:44:37] Jason: check you out. They've never heard of this thing called the LMS, but yeah. Thank you so much for joining us. Really appreciate it. [00:44:46] John Nash: Yeah. Thanks Ryan. Yep. [00:44:47] Jason: Yeah, thanks.