Measuring Populism in Social Media Data: A Supervised Machine Learning Approach using Party Communication

Populism is a crucial concept in the study of political parties and the most recent electoral gains of parties and candidates frequently labelled as populist even amplified public and scientific debates surrounding it. On the one hand, the scholarly literature on populist parties has established a broad consensus concerning the definition of the term populism and what must empirically be observed to speak of political actors using populist rhetoric (Kriesi, 2014; Mudde, 2013). On the other hand, however, efforts to measure the concept empirically are scarce (but see Rooduijn and Pauwels, 2011; Jagers and Walgrave, 2007; Pauwels, 2011), which imposes limits on comparative research on usage patterns of populist rhetoric in party politics, its driving forces and its consequences. Against this background, we introduce a new measure to identify populist rhetoric employed by political parties and their candidates using data from the social media website Facebook. We argue that this data source is well suited for this task for three reasons. First, all parties and candidates in contemporary democracies actively use this platform. Second, it enables researchers to explore direct party communication since no gatekeepers need to be circumvented to make statements visible to a wider audience. Thus, it is arguably the most relevant platform for populist rhetoric. Third, the data source can be analyzed on a daily basis. All in all, it allows exploring populism comparatively, unfiltered and over time for a wide range of political actors. To test the merits of this approach, we study the official Facebook accounts of the most important parties and their lead candidates in Austria, Germany and the United Kingdom between 2014 and 2016. In total, we classify more than 30000 Facebook posts from 45 different accounts using a supervised learning approach to document classification based on support vector machine to identify posts containing populist rhetoric. We validate the derived indicator with data from the Chapel Hill Expert Survey 2014 (Bakker et al., 2015). Our results show how prevalent populist rhetoric is in party communication on Facebook.