Synthetic Policy Spaces Using Voting Advice Applications

Voting Advice Applications (VAA) have been shown to outperform expert surveys and text-analytical methods in their predictive power with respect to elite and party behaviour. The main reason is that VAAs are designed to be election-specific in order to identify salient policy dimensions in a given election. However, there are two central caveats regarding their viability for comparative research: First, while many VAAs exist across different countries and for a wide variety of elections, they are often not fully comparable. Second, VAAs are a relatively recent phenomenon compared to projects based on election manifestos. Consequently, they are not available for all elections and parties. This creates problems for direct comparisons as well as for dimensionality reduction: reducing asymmetric matrices with missing values is problematic for obtaining valid estimates of low-dimensional party positions. Recent developments in transformer models help to alleviate these problems. We argue, first, that although items are not completely identical, semantic similarity between them can be estimated to emulate theoretical answers to very similar questions. Second, we argue that entirely missing information on distinctive questions and parties can be recovered from election manifestos. We use a text-analytical approach to impute missing answers. By encoding all manifesto statements and all questions ever asked in any available VAA using a BERT transformer, we connect manifesto sentences and VAA items in a common embedding space. We then reduce the dimensionality of the VAA statement vector representations and party-election entities using canonical correspondence analysis. This procedure yields a map of statements and party positions in a joint policy space. We apply this approach to all German federal and state elections between 1990 and 2020. Since VAAs have only been available around 2002, whereas manifestos and surveys have existed since 1990, this constitutes an ideal use case to keep moderate variability in context. We validate our findings using three validation strategies, mirroring the approach’s main use cases: First, we assess performance in coalition prediction. Second, we rely on a harmonised dataset of state election surveys (Landtagswahlstudien), which contains variables of ideological self-placement and placement of parties across different spatial and temporal contexts. Finally, we compare our approach to expert surveys and manifesto-based results (wordscores).