Leaders in the Loop – Episode 12 – Rethinking Assessment in the Age of Generative AI
Dan [https://www.linkedin.com/in/danjenkinsphd/] and Gaurav [https://www.linkedin.com/in/gkhanna1/] are joined by three researchers who are doing some of the most focused, longitudinal work in higher education on how assessment professionals are actually experiencing generative AI — not in theory, but in practice, in policy, and in the profession itself:
* Dr. Ruth Slotnick [https://www.linkedin.com/in/ruthslotnick/] is Director of Assessment at Bridgewater State University and a co-lead of a national GenAI pulse survey for higher education assessment professionals that has now run across four administrations since early 2025.
* Dr. Will Miller [https://www.linkedin.com/in/willmillerphd/] is Associate Vice President for Continuous Improvement and Institutional Performance and SACSCOC Liaison at Embry-Riddle Aeronautical University. He oversees institutional effectiveness and accreditation across more than 100 sites globally and serves on the SACSCOC Board of Trustees.
* Dr. John Hathcoat [https://www.linkedin.com/in/john-hathcoat-771a19352] is Associate Professor of Graduate Psychology and Associate Assessment Specialist in the Center for Assessment and Research Studies at James Madison University, where his work focuses on measurement theory, validity, and the emerging challenge of assessing AI literacy.
Episode Overview
Dan Jenkins [https://www.linkedin.com/in/danjenkinsphd/] and Gaurav Khanna [https://www.linkedin.com/in/gkhanna1/] open the episode by framing assessment as one of higher education's most persistent challenges — and one of the most consequential sites for navigating generative AI. Ruth, Will, and John bring longitudinal survey data, institutional case studies, and sharp theoretical perspective to a conversation that moves from the origins of their research collaboration, through what the data actually show about adoption and institutional readiness, into deeper questions about what it means to assess student learning when AI is always in the room. Dan also brings firsthand perspective from his own institution's AI task force and from a recent program review he conducted using AI tools for the first time.
Topics Covered
* How Ruth, Will, and John each came to generative AI independently — and how they found each other at the Assessment Institute in Indianapolis
* Ruth's early 2022–2023 experiments using Bing, Bard, and ChatGPT for qualitative data analysis alongside colleague Joanna Boeing
* John's shift from skepticism about early automated scoring systems to genuine excitement after a JMU book club read Ethan Mollick's Co-Intelligence
* Will's conviction that AI disruption — like COVID before it — creates rare windows for meaningful change in institutions that otherwise resist it
* The difference between programmatic/institutional assessment and course-based assessment — and why that distinction matters for how AI enters the work
* Gaurav's framing: adoption has won; institutionalization has not
* The GenAI Pulse Survey: four administrations since early 2025, more than 278 assessment leaders in Spring 2026, longitudinal tracking of use, tools, policy, training, and influence
* Key survey findings: 79% regular or occasional users; 87% report efficiency gains; 87% self-taught; 71% report ongoing privacy and ethical concerns
* What "self-taught" reveals about the gap between individual adaptation and institutional support
* The policy question: why moving from individual use to institutional guidelines is harder than it looks — and why guidelines may serve institutions better than formal policy
* The "tragedy of the commons" dynamic when institutions lack shared AI expectations
* John's three-frame model for assessing in an AI-enabled world: what students can do without AI, with AI, and across new contexts
* Will's challenge: we know how to verify math before allowing calculators; we don't yet know how to verify critical thinking before letting students go all-out with AI
* John's concept of "co-constructed performance" — when learning and assessment no longer happen separately, but in real time alongside AI
* Ruth's argument for innovation space: "skunk works" thinking, cross-functional data collaboration, and the role of curriculum design
* Dan's firsthand experience using AI (Copilot and Claude) during a program review at JMU — taking stakeholder notes and generating a first-draft report
* Faculty reluctance, faculty openness, and what happens when assessment professionals arrive carrying both the assessment agenda and the AI agenda
* Lee Shulman's concept of signature pedagogies and why AI integration must reflect disciplinary tradition
* Will's analogy: higher ed still gives students summers off because they once needed to work the fields — and AI is forcing the same kind of reckoning with inherited structure
* Reasons for optimism: growing AI use in the assessment profession, a tenfold increase in AI-focused presentations at the Assessment Institute, the boot camp Ruth, Will, and John built for the field
* The resource gap: many assessment professionals are using free tools because institutions haven't yet funded access
* The sphere of influence finding: assessment professionals have substantial influence within their immediate team — but that drops sharply at the department, institution, and external levels
* Lightning round: AI tools (Gamma, Claude, Codex/Claude Code), a co-constructed AI wish list, and Ruth's "velvet hammer" prompt approach
Key Takeaways
*
Adoption has outpaced institutionalization — and that gap has consequences. The survey data are clear: assessment professionals are using generative AI, often daily, and mostly on their own. But institutions have not kept pace with training, policy, or tool access. The result is fragmented, inconsistent practice that puts students, faculty, and practitioners in difficult positions.
*
The cheating frame is a distraction from a harder question. John argues that higher education needs to move past the academic integrity panic and toward a more serious reckoning with what we're actually trying to assess. Three distinct frames matter: what students can do without AI, what they can do with AI, and how they transfer skills across contexts. Most institutions haven't fully confronted any of these.
*
Co-constructed performance changes what assessment means. When AI is available in real time — prompting students, monitoring thinking, offering feedback — the act of learning and the act of being assessed begin to collapse into each other. John sees this as one of the most urgent conceptual challenges in the field, and one that current measurement frameworks weren't built for.
*
Assessment professionals are caught in a double bind. Ruth describes the dynamic with characteristic directness: assessment practitioners are already navigating limited sphere of influence on campus. When they also arrive carrying AI advocacy, some faculty "doubly don't like" what they represent. That reality shapes what change is possible and how it has to be introduced.
*
Guidelines may serve institutions better than policy — for now. The survey and the guests converge on a shared concern: formal policies take years to pass through governance, and the technology changes faster than institutions can ratify. Guidelines — clear but flexible — may allow institutions to move like speedboats instead of tankers.
*
The field needs to train itself. With 87% of assessment professionals self-taught on generative AI, and with institutions slow to fund either tools or professional development, Ruth, Will, and John built their own boot camp. The field is not waiting — but it is under-resourced.
*
The window for change is open, but it won't stay open. Will is explicitly optimistic about higher education's discomfort. Discomfort creates opportunity. Faculty are rethinking entire programs — not because an accreditor is coming, not because enrollment is down, but because AI is forcing a real look at what curriculum is actually for. That pressure, he argues, is a rare gift.
Resources & Mentions
Research & Survey
* GenAI Pulse Survey for Higher Education Assessment Professionals – Spring 2026 Results [https://genaiassessment.neocities.org/Spring2026/Spring2026] — Ruth Slotnick, Joanna Boeing, & Bobbijo Grillo Pinnelli
* GenAI Assessment main site [https://www.genaiassessment.com/]
Book Referenced
* Co-Intelligence: Living and Working with AI — Ethan Mollick (Amazon [https://www.amazon.com/Co-Intelligence-Living-Working-Ethan-Mollick/dp/0593716744])
Conference
* Assessment Institute at Indiana University Indianapolis [https://assessmentinstitute.indianapolis.iu.edu/] — chaired by Dr. Stephen Hundley; described as the oldest and largest U.S. higher education assessment event
AI Tools Mentioned
* Gamma [https://gamma.app/] — AI-powered presentation tool (Will's recommendation)
* NotebookLM [https://notebooklm.google.com/] — Google AI research and synthesis tool (Dan mentions new slide editing capability)
* Claude [https://claude.ai/] — Anthropic (Dan and John mention; John references Claude Code)
* Codex [https://openai.com/codex] — OpenAI coding tool (John mentions alongside Claude Code)
* Microsoft Copilot — Dan used during program review at JMU
Frameworks Referenced
* Lee Shulman — Signature Pedagogies (landmark concept in discipline-specific teaching)
* Metacognitive practice in AI-integrated curriculum (Ruth references work by Mike Kent — affiliation unverified; Needs Human Review)
* "Skunk works" / innovation sandboxes in higher ed — attributed to Nick Bero Jones at Northeastern (name spelling uncertain; Needs Human Review)
Organizations
* Bridgewater State University [https://www.bridgew.edu/] — Ruth Slotnick's institution
* Embry-Riddle Aeronautical University [https://erau.edu/] — Will Miller's institution
* James Madison University – Center for Assessment and Research Studies [https://www.jmu.edu/assessment/] — John Hathcoat's center
* SACSCOC [https://sacscoc.org/] — Southern Association of Colleges and Schools Commission on Colleges (Will serves on Board of Trustees)
* Walden University [https://www.waldenu.edu/] — Ruth teaches here; AI use required in curriculum
* American University – Kogod School of Business [https://www.american.edu/kogod/] — cited by Ruth as example of visible, external AI integration commitment
* University of South Florida [https://www.usf.edu/] — where Dan and Ruth completed their PhDs together
Forthcoming
* Book chapter by Ruth Slotnick on AI and assessment, forthcoming in a volume edited by Will Miller (title and publisher TBD)
Stay curious. Stay human.
Kommentit
0Ole ensimmäinen kommentoija
Rekisteröidy nyt ja liity Leaders in the Loop-yhteisöön!