Detecting Democratic Backsliding in Assessment Reports Using Computational Social Science Tools.

The combined influence of COVID-19 emergency measures, toxic polarization and the rise of illiberal democratic regimes have put a halt to democratic advances. Many mature and young democracies have experienced democratic backsliding. Despite initially promising signs of liberalization, countries like Russia, Turkey, Poland and Hungary have slid back into authoritarian rule. Mature democracies such as France and UK have also recorded loss of democratic quality. In such a context, democratic backsliding has attracted the attention of international agencies or consortia, which regularly assess the quality of democracy in different countries. Nevertheless, such attempts suffer from subjectivity bias, as they mostly rely on qualitative expert judgments. Yet, we lack a comparative view of the dimensions and quality of democratic assessments. To address this gap, our paper addresses the question: To what degree assessment reports vary in grading countries by traits of democratic quality and over time? We develop and apply computational text analysis tools that map dimensions of democratic quality in texts and assess the precision of democratic assessments. Theoretically, we focus on three well-established dimensions of democracy: “electoral”, “participatory” and “liberal”. We distinguish between country features related to free and fair elections, ‘positive’ political rights contributing to pluralism and ‘negative’ civil rights protecting institutions and individuals from the state. Empirically, we propose a taxonomy of indicators for democratic quality using the individual country reports produced by the European Commission, Freedom House and the Bertelsmann Foundation. The reports cover all Council of Europe countries between 1999 and 2022. The rich data allows us to train a computational text analysis algorithm that detects the emphasis (i.e. coverage) of democratic quality indicators in different countries and across time. Based on the analysis, we discuss the merits and limits of computational approaches for the study of democracy.