Detecting Democratic Backsliding in Assessment Reports Using Computational Social Science Tools

The combined influence of COVID-19 emergency measures, toxic polarization as well as the rise of illiberal democratic regimes have put a halt to democratic advances. Many mature and young democracies have experienced democratic backsliding, defined as the deterioration of qualities associated with democratic governance. Despite initially promising signs of liberalization, countries like Russia, Turkey, Poland and Hungary have slid back into authoritarian rule. Mature democracies such as France and the United Kingdom have also recorded loss of democratic quality. In such a context, democratic backsliding has attracted the attention of international agencies or consortia (e.g., Freedom House, V.Dem), which regularly assess the quality of democracy in different countries. Nevertheless, such attempts suffer from subjectivity bias, as they mostly rely on qualitative expert judgments. Yet, we lack a comparative view of the dimensions and quality of democratic assessments. To address this gap, our paper focuses on the following questions: 1. To what degree assessment reports vary in grading countries by traits of democratic quality and over time? To address this question, we develop and apply computational text analysis tools that: 1) map dimensions of democratic quality in texts and 2) assess the precision of democratic assessments. Theoretically, we focus on three well-established dimensions of democracy: “electoral”, “participatory” and “liberal”. More precisely, we distinguish between country features related to free and fair elections (electoral dimension), ‘positive’ political rights contributing to pluralism (participatory dimension) and ‘negative’ civil rights protecting institutions and individuals from the state (liberal dimension). Empirically, we propose a taxonomy of indicators for democratic quality using the individual country reports produced by the European Commission, Freedom House, and the Bertelsmann Foundation. The reports cover all Council of Europe countries between 1999 and 2022. The rich data allows us to train a computational text analysis algorithm that detects the emphasis (i.e. coverage) of democratic quality indicators in different countries and across time. Based on the analysis, we discuss and reflect on the merits and limits of computational approaches for the study of democracy.