Are expert assessments on democracy biased? A comparative study of democracy dimensions in Europe

Many mature and young democracies experience democratic backsliding, defined as the deterioration of qualities associated with democratic governance. Despite initially promising signs of liberalization, countries like Russia, Turkey, Poland and Hungary have slid back into authoritarian rule. Mature democracies are not immune to democratic backsliding either. France and the United Kingdom have also recorded loss of democratic quality. In this context, democratic backsliding has attracted the attention of international organizations and consortia (e.g., Freedom House, V.Dem, the European Union). Yet, these reports are often inherently biased. For example, the EU is criticized to predominantly focus on executive compliance with democratic principles with less emphasis on the role of civil society actors. Yet, we lack a comparative view on how democratic assessments vary across time, countries and sources. To address this gap, we develop and apply computational text analysis tools that map dimensions of democratic quality in texts. Theoretically, we focus on three well-established dimensions of democracy: "electoral", "participatory" and "liberal". Empirically, we propose a taxonomy of indicators for democratic quality using the individual country reports produced by the European Commission, Freedom House and the Bertelsmann Foundation. The reports cover all European and neighbouring countries between 1990 and 2023. Our findings show that country experts put extensive emphasis on democratic quality in CEE countries across various dimensions. Furthermore, there is increased emphasis on rule of law as a primary dimension for democratic quality. Based on the analysis, we discuss and reflect on the merits and limits of computational approaches for the study of democracy.