VAAs typically include two-dimensional visualizations of the ideological space. These maps perform a crucial function: not only do they position the user but they also provide, at least implicitly, a form of voting advice, analogous to proximity-based spatial models of voting (Louwerse & Rosema 2011). Given the importance of these maps, the research community has begun to address some of the methodological concerns raised by the use of such ideological scales (Gemenis 2013; Louwerse & Otjes 2012). However, existing assessments of measurement quality focus on the supply side (i.e. political parties) and are all related to the supranational setting of the EU. Whilst revealing, such a focus is insufficient for both methodological and practical reasons. First, the political maps ought to be meaningful to the users, which cannot be guaranteed by solely drawing on the supply side. Second, unlike the EU setting, the number of parties in a national VAA is typically too small for rigorous scale validation. To overcome these problems, we propose a demand side analysis based on actual user data. From a methodological viewpoint, we examine two important measurement properties: unidimensionality and test-score reliability. In practical terms, we propose that VAA designers dynamically adjust the ideological scales. We explore the plausibility of this recommendation by drawing on subsets of early users to scrutinize the scales that emerge and whether the latter hold over time. We illustrate our method by drawing on user datasets from two different electoral settings, the Swiss smartvote and the Greek choose4greece. Our findings suggest that a scale adjustment in an early phase would have implied considerable adjustments to the ex-ante defined scales and led to divergent placement of users in the political space and significant changes to the implicit voting advice.
