Navigating Ideological Currents: A Machine Learning Exploration of Populist and Eurosceptic Rhetoric in Prime Minister’s Questions

In the context of the British political landscape, populism and Euroscepticism are intricately interwoven. This study introduces a novel methodology for identifying and measuring populism and Euroscepticism within the framework of Prime Minister's Questions (PMQs) in the United Kingdom (UK), spanning the years 1997 to 2023. The period in question witnessed pivotal events, including the ratification of the Maastricht Treaty, several regional and global crises facing the European Union, the rise of the UK Independence Party (UKIP), the Brexit referendum, and the eventual withdrawal of the UK from the EU. These milestones significantly influenced the discourse surrounding European integration and the role of the EU, prompting increased use of populist rhetoric by politicians to resonate with public sentiment on matters of national identity, immigration, and sovereignty. Against the backdrop of the populist and Eurosceptic movements and the evolving discourse in the UK during this timeframe, understanding the prevalence of these ideological stances in political rhetoric has become of paramount importance. In this regard, PMQs, serving as the "shop window" of the House of Commons, offer a concentrated platform to engage in direct and often confrontational exchanges. The intense verbal sparring in these weekly sessions provides a rich dataset for linguistic analysis, making it an ideal source for studying ideological trends. The diverse array of topics covered during PMQs also reflects the broader political landscape, ensuring that the analysis captures a comprehensive spectrum of issues and sentiments. Our study covers all PMQ sessions since they have taken their current 30-minute, weekly structure and started to be televised under the Blair government. We employ state-of-the-art natural language processing techniques to classify topics into (hard vs. soft) Eurosceptic or neutral, and to estimate levels of populism using text scaling approaches. The supervised models are trained on an annotated dataset enriched with linguistic features and contextual information. This approach enables more fine-grained analyses at weekly intervals and at the individual level while ensuring robust accuracy and interpretability. Overall, the study sheds light on the nuanced interplay between populism and Euroscepticism in the British political context over two decades, opening avenues for understanding how linguistic patterns in parliamentary debates reflect and influence broader ideological shifts in UK politics. As such, it makes valuable contributions to the interdisciplinary fields of political communication, computational linguistics, and machine learning.