Assessing survey questions to measure populism from a cognitive testing perspective

Cognitive testing is a helpful pre-testing technique for survey questions. It is aimed to identify issues related to understanding and interpretation of words and phrasing used in a survey question. Also, it helps to identify issues related to memory retrieval (i.e., memory erosion), rationale and decision making when determining an answer, and assessing how respondents map their thinking into response categories. The International Social Survey Programme (ISSP) is developing a new survey module (National Identity and Citizenship) which covers a variety of relevant topics, including questions on populism. These questions will be fielded across the globe by ISSP members. The USA team conducted a cognitive test on newly designed question on populism, covering topics such as: will of people, people vs. politicians, inaction of politicians, political compromise, perception of people by politicians, political elitism, perception of economic policy. This presentation will discuss results from the testing of these items and implications for survey measurement properties; that is, how are questions understood, what elements may be more salient in memory as respondents answer questions, how respondents decide on what responses to provide, and whether response categories capture the intended response.