ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

What can go wrong? Using LLMs to extract networks of political actors from multilingual cross-platform search results on Swiss popular votes

Voting
Methods
Quantitative
Technology
Big Data
Mykola Makhortykh
Universität Bern
Vihang Jumle
Universität Bern
Mykola Makhortykh
Universität Bern
Maryna Sydorova
Universität Bern
Victoria Vziatysheva
Universität Bern

To access full paper downloads, participants are encouraged to install the official Event App, available on the App Store.


Abstract

Today, large parts of the population in Western democracies rely on AI-driven systems, such as search engines and, increasingly, chatbots, to acquire information about politics. While such systems are far from being the only source of political information, the way they inform individuals about politics has been shown to have an impact on politics-related attitudes and behaviours, including voting decisions. An important part of this informedness regards how these systems present the relationships between specific issues and actors, due to it determines the degree to which individual voters perceive actors’ alignment with their own beliefs and also to what degree the actors can be held accountable for specific developments in relation to the issues. A major challenge of the AI-driven political information ecosystem is the extensive amount of digital content that can be retrieved and prioritized by AI systems, potentially containing information about networks of actors and issues. While named entity recognition has long been used to address this problem in the context of political network data extraction, it has also faced multiple challenges related to generalization to new domains, scalability, and language ambiguity. Many of these problems can be addressed with the advancement of large language models (LLMs) due to their improved capacities for natural language understanding, but their application for network extraction still requires much empirical assessment, particularly regarding the integration of LLM-based approaches into existing data extraction and processing pipelines, including the ability of LLMs to perform network analysis-related tasks for multilingual cross-platform digital corpora. In our study, we examine the possibilities and limitations of leveraging an LLM-assisted analysis for extracting networks of relationships between political actors and issues in the context of Swiss popular votes from AI-driven system outputs. Our interest in Switzerland is attributed to it being a (semi-)direct multilingual democracy, where the informedness of citizens on political issues is essential for the functionality of the political system due to the regular rounds of popular votes. Using two large datasets of search engine and AI chatbot outputs for user prompts related to a selection of popular votes in 2024 and 2025, we examine the degree to which commercial (Cohere) and publicly available (Llama) LLMs are capable of extracting relations between political actors and vote-specific issues from multilingual cross-platform data. We also examine how the extracted data can be used to conduct network analysis of AI-driven representations of actors whose opinions matter in relation to popular votes. We aim to make two contributions in this study. First, we offer a methodological contribution that examines the effects of data processing (specifically, HTML cleaning) on the LLM-based extraction of political actor networks from search results scraped from various platforms appearing in search engine results, and discuss scalability challenges associated with this process. Second, we provide empirical insights into the performance of two LLMs for political network data extraction in different languages and discuss the implications of similarities and differences in the resulting networks for how AI systems represent Swiss politics to their users.