Writing Good Survey Questions - Part 3. Open-Ended Questions

Designing Surveys for Multilingual Populations - Part 2. An Interview with Rubén Ángel Arias Rueda

In this episode of Total Survey Design, Dr. Azdren Coma sits down with Dr. Rubén Ángel Arias Rueda, Project Manager and Lead Researcher at Washington State University's Social and Economic Sciences Research Center, for the second part of a two-part series on multilingual surveys. Dr. Arias Rueda specializes in mixed-methods equity research, bilingual data collection, and community-engaged fieldwork, and has led statewide qualitative and survey studies for Washington State agencies and commissions—including research on opportunity gaps for Hispanic and Latino students and on agricultural labor conditions. Before joining the SESRC, he spent 15 years teaching Spanish and Spanish literature at all levels of the curriculum. The conversation explores Rubén's approach to designing surveys that work in both English and Spanish, beginning with the core goal that guides his bilingual instruments. He weighs in on whether tools like Google Translate and generative AI are sufficient for translating a survey into a second language, and discusses the strengths and limits of back-translation as a quality check. From there, the discussion turns to the practical challenges of designing and administering multilingual surveys in the field, the methodological questions involved in combining data across language versions, and the considerations researchers should keep in mind when interpreting results from multilingual samples. The episode closes with Rubén's broader advice for researchers and practitioners taking on multilingual survey work, drawing on lessons from his experience of bilingual data collection with Spanish-speaking communities across Washington State. Support the show [https://ko-fi.com/totalsurveydesign] Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign [https://taplink.cc/totalsurveydesign?fbclid=PAZXh0bgNhZW0CMTEAAabcPg7NrYGB42pSmIOr0z7LFw74QyF4WxUk_LnWpepucEqV1U4fC0OwLQc_aem_TrjCUlqu_RIsBay_aPPUaA]

Designing Surveys for Multilingual Populations - Дизајнирање анкети за повеќејазични популации

In this episode of Total Survey Design, Dr. Azdren Coma explores what it takes to design surveys for populations that speak two or more languages. The episode explains why measurement equivalence (ensuring responses mean the same thing across languages) matters more than literal translation, and why the goal should be conceptual equivalence rather than word-for-word matching. Drawing on a personal case study of the 2021 North Macedonia Census, Azdren walks through real-world translation pitfalls, including untranslated Cyrillic dropdown menus and grammatically awkward translations from Macedonian into Albanian, and closes with practical takeaways for reducing cognitive burden and bias across language groups. This is the first of a two-part series on surveying multilingual populations. Support the show [https://ko-fi.com/totalsurveydesign] Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign [https://taplink.cc/totalsurveydesign?fbclid=PAZXh0bgNhZW0CMTEAAabcPg7NrYGB42pSmIOr0z7LFw74QyF4WxUk_LnWpepucEqV1U4fC0OwLQc_aem_TrjCUlqu_RIsBay_aPPUaA]

Personal Appeals in Contact Letters: How More Appeals Means More Responses

In this episode, I explore research by Lena Le and Thom Allen at Washington State University on the use of personal appeals in survey contact letters — the persuasive statements that encourage people to participate, like explaining how results will be used or telling respondents their opinion matters. Analyzing 190 surveys, their data shows response rates climb from 21.6% with no appeals to 51.6% with nine. I also ran my own statistical analysis on the data, finding each additional appeal is associated with a 4.1 percentage point increase in response rate (p = 0.010). Ideal for survey researchers and anyone responsible for designing contact strategies to maximize participation. Support the show [https://ko-fi.com/totalsurveydesign] Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign [https://taplink.cc/totalsurveydesign?fbclid=PAZXh0bgNhZW0CMTEAAabcPg7NrYGB42pSmIOr0z7LFw74QyF4WxUk_LnWpepucEqV1U4fC0OwLQc_aem_TrjCUlqu_RIsBay_aPPUaA]

How the Dunning-Kruger Effect Skews Self-Reported Skill Ratings

In this episode, Dr. Azdren Coma explores how the Dunning-Kruger effect distorts self-reported skill ratings in hiring and surveys. A seemingly straightforward question like “On a scale of 0 to 10, how skilled are you in Microsoft Excel?” can produce misleading data because undefined scale points allow respondents to interpret them differently. Beginners often overestimate their abilities due to limited awareness of complexity, while more skilled individuals rate themselves more conservatively as they recognize what they don’t know—potentially reversing the ranking of candidates. This episode breaks down the psychological mechanism behind the bias, illustrates it with real-world examples, and offers practical fixes for better data quality, including fully labeling scales, using clearer anchors, and shifting to task-specific questions. Ideal for HR professionals, hiring managers, and anyone relying on self-assessments in talent acquisition or people operations. Support the show [https://ko-fi.com/totalsurveydesign] Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign [https://taplink.cc/totalsurveydesign?fbclid=PAZXh0bgNhZW0CMTEAAabcPg7NrYGB42pSmIOr0z7LFw74QyF4WxUk_LnWpepucEqV1U4fC0OwLQc_aem_TrjCUlqu_RIsBay_aPPUaA]

Writing Good Survey Questions - Part 3. Open-Ended Questions

In this episode, I will be talking about open-ended questions. Along with closed-ended questions, open-ended questions are the other major type of question used in surveys. This episode covers the purpose of open-ended questions, their pros and cons, the different types of open-ended questions, several examples, and finally some tips and best practices for implementing them effectively in surveys. The episode also includes an important ethical cautionary tale, as well as a warning about the dangers of over-relying on generative AI to analyze open-ended questions.  Support the show [https://ko-fi.com/totalsurveydesign] Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign [https://taplink.cc/totalsurveydesign?fbclid=PAZXh0bgNhZW0CMTEAAabcPg7NrYGB42pSmIOr0z7LFw74QyF4WxUk_LnWpepucEqV1U4fC0OwLQc_aem_TrjCUlqu_RIsBay_aPPUaA]