Get to Know Your Data: Possibilities and Limits of Analysing Parliamentary Speeches with Wordfish

Panel: Methodological innovation in parliamentary research Quantitative text analysis (QTA) for parliamentary research has been an intriguing approach in the past years. It promises fast and efficient data processing, convincing graphical outputs and the exploitation of data sources that were previously only available at high cost. Useful sources in this context include protocols of parliamentary speeches, party press releases or draft laws. Especially speeches provide a large data source that, until now, has not been comprehensively tapped into. To make use of these sources, QTA utilizes various methods, such as numerous approaches to categorizing text sections or sentiment analysis as well as ideological scaling of texts. The latter is of particular interest for parliamentary and party researchers as it is an attractive addition to estimates from election manifestos or expert surveys. The goal of this paper is to discuss the possibilities and limitations of parliamentary debates as a data source for QTA methods, with a particular focus on Wordfish. The starting point of these considerations is my own research, which looks into the ideological party positions in the German Parliament (Bundestag). The positions were estimated using Wordfish and plenary debates on refugee policy during the so-called refugee crisis in 2015. In dealing with QTA methods and parliamentary debates it is essential to think about the implications of the processed texts and methods. The authors of existing studies do reflect on their own case characteristics, but a systematic collection of all these insights and their implications is still missing. To do this, the paper takes into account three main features: 1) the technical requirements of QTA methods, such as Wordfish, 2) the characteristics of parliamentary speeches as a data source and 3) the implications for a possible application of the analysis method and the interpretation of their results. More specifically, the goal of this paper is to reflect systematically on the characteristics of parliamentary debates as a data source and the quantitative approach to deduce ideological positions from this sort of texts. Is this not just measuring word preferences? How do the rules and restrictions that shape the nature of parliamentary debates influence the results of Wordfish estimates? What exactly are the rules that govern speaking length, timing and frequency of Member of Parliament and party contributions? Other topics are the opposition and governmental affiliation of parties in parliament, which determine their actions, their language and may thus also impact on the analysis of parliamentary debates as text. And if the aggregation of individual speakers to a party level position is a permissible procedure for aggregating the overall party position. At this point the influence of party group discipline has also to be taken in account. Additionally it will be discussed to what extent practical or technical approaches provide solutions for the methodological and theoretical restrictions of QTA. For technical language processing this are e.g. lemmatization, stemming and the removal of stopwords, as well as for the analysis itself, the required minimum word count for QTA.