Who gets Lost, and what Difference it Makes? Mixed Modes, Survey Participation and Nonresponse Bias

While the bulk of the literature on turnout bias in post-election surveys has focused on vote overreporting, recent work shows that voter overrepresentation among survey respondents (nonresponse bias) contributes to a greater extent than measurement error (overreporting) to the overestimation of turnout (e.g. Sciarini and Goldberg, 2016 and forthcoming). Nonresponse bias thus deserves closer attention. In the present Paper, we contribute to the stream of survey research looking at the influence of mode effects on nonresponse bias (Voogt and Saris 2005, Atkeson et al. 2014). Empirically, we rely on a unique data set of validated votes collected in the context of two post-election surveys in the canton of Geneva (in 2012 and 2015). The data set offers information on official turnout (and basic socio-demographic variables) for both sampled citizens who participated in the survey, and those who did not. Both surveys used the same sampling frame (drawn from the official vote register) and they both relied on a mixed mode design (telephone and internet with very similar response rates (about 45%)). In addition, both surveys included a short written questionnaire among non-respondents, again with very similar response rates (about 35%). However, the surveys differ from one another with respect to the relative weight of telephone and online surveys. The first survey was predominantly a telephone survey, complemented with online interviews, whereas the second was primarily an internet survey, complemented with telephone interviews. We first address the question "who gets lost", by comparing the socio-demographic characteristics of the initial sample with those of the two sub-samples of survey respondents and respondents to the written questionnaire. For the two types of respondents, we further compare the impact of political attitudes on survey participation. We then turn to the analysis of "what difference it makes", i.e. to the analysis of nonresponse bias (voter overrepresentation). Using again the initial sample as base-line we analyze the size and the determinants of voter overrepresentation among survey respondents and respondents to the written questionnaire. Overall, our results highlight the added value of the written questionnaire among non-respondents, which in both surveys increases the representativeness of the realized sample, reduces the nonresponse bias, and enhances the accuracy of turnout determinants.