The Method to Our Madness

Training Tomorrow’s Evaluators in a Changing World

What does it take to prepare the next generation of evaluators to meet the challenges of a changing world? This episode dives into the shifting demands on the field, from federal funding uncertainty to the growing influence of AI tools, and the skills future evaluators will need to rise to the challenges. Our guests from the University of Minnesota discuss how future evaluators should be trained to think systemically, challenge assumptions, and act as collaborative problem-solvers, as well as possibly be prepared for an employment model based on project work, rather than extended tenure at one organization. Whether you're in education, public policy, or nonprofit work, this conversation offers timely insight into how evaluation is being redefined for the future.   Our guests this week from the University of Minnesota are: John Hitchcock, Ph.D. [https://olpd.umn.edu/people/john-hitchcock], Professor of Evaluation Studies and Marilyn Sime Professor for Educational Excellence, hitch141@umn.edu Jodi Benenson, Ph.D. [https://olpd.umn.edu/people/jodi-benenson], Associate professor in the Department of Organizational Leadership, Policy, and Development in the College of Education and Human Development, benenson@umn.edu Stuart Yeh, Ph.D. [https://olpd.umn.edu/stuart-yeh], Professor and Coordinator of Graduate Programs in Evaluation Studies, yehxx008@umn.edu [yehxx008@umn.edu]   We cover a lot of topics in this episode, so if desired, a written summary can be found here: Episode 12 summary [https://drive.google.com/file/d/10oMJXq6lEOWU8_77MssFHYM4fkDQUOaE/view?usp=sharing]   Podcast music written and produced by Stacey Merola. Podcast artwork by Zoe Targino.

Logic Models: Keeping them Simple, Useful, and Dynamic

Logic models don’t have to be complicated or static. In this conversation with guest Deborah Levy [https://www.linkedin.com/in/deborah-levy-7996a85/], Stacey and Bill discuss why simplicity matters, how logic models should evolve over time, and what happens when programs face unexpected challenges like shifting audiences or external disruptions. The episode also offers practical tips for engaging stakeholders, collecting data, and using logic models as living tools throughout an evaluation.    Podcast music written and produced by Stacey Merola. Podcast artwork by Zoe Targino.

The Practice and Judgement Behind Measuring Fidelity of Implementation

Fidelity of implementation is often talked about but rarely understood. In Episode 10 of The Method to Our Madness, Allan Porowski, Principal Associate at Abt Global, brings clarity to this critical concept in program evaluation. Drawing on years of experience and large‑scale evaluation work, Allan explains why fidelity metrics must be tailored to each program, how planned‑offered‑received frameworks reveal meaningful variation, and why imperfect implementation can still drive valuable insight. This episode offers practical wisdom for evaluators seeking to interpret results with context, nuance, and purpose.   Podcast music written and produced by Stacey Merola. Podcast artwork by Zoe Targino.

Keeping Statistics Useful: From Descriptive Data to Meaningful Impact

In this episode of The Method to Our Madness, Stacey and Bill unpack the different types of statistics commonly used in program evaluation—and why “simple” doesn’t mean unsophisticated. They explore how evaluators balance accessibility with rigor, discussing when descriptive statistics, t-tests, and ANOVA are the right tools, and when more advanced methods like hierarchical linear modeling or propensity score matching make sense. Along the way, they dig into the hidden complexity behind clean-looking results, the limits of common data visualization tools, and why effect sizes often matter more than p-values. Drawing on real research experiences, disciplinary perspectives from sociology and psychology, and lessons learned from both practice and academia, this conversation highlights what it really takes to produce statistical findings that are both credible and useful. The episode wraps up with a reflection on communicating results clearly—and an open invitation for listeners to suggest future topics they’d like to explore in greater depth.   Podcast music written and produced by Stacey Merola. Podcast artwork by Zoe Targino.

Different Types of Evaluation

What’s the difference between formative, process, outcome, and impact evaluation? When should you actually use each one? In this episode, Stacey and Bill break down the why and when of evaluation, drawing on real-world examples from education, training programs, and policy work. They talk implementation fidelity, stakeholder engagement, logic models, and why not every evaluation is about proving causality. The takeaway: good evaluation is about learning, improvement, and usefulness—not just measurement.   Music written and produced by Stacey Merola. Podcast artwork by Zoe Targino.