Classifying ideological policy frames: Introducing the ILLFRAMES codebook and the Babel Machine pipeline

The growing illiberal movement poses a significant global challenge to democracy, with policy crises often serving as critical opportunities for illiberal leaders to advance their agendas through illiberal policy frames (IPFs). This paper introduces a comprehensive methodological framework for analysing the emergence and prevalence of illiberal policy frames (IPFs) across various policy domains—including migration, COVID-19, LGBTQ rights, and the Ukraine war—using advanced quantitative methods and large language models (LLMs). At the core of this framework lies the ILLFRAMES codebook, a novel tool designed to identify and categorise ideologically charged frames in legislative speeches, parliamentary debates, and political communication. Complementing this is the Babel Machine classification pipeline, an innovative system capable of classifying illiberal frames across diverse linguistic, cultural, and political contexts. The Babel Machine employs state-of-the-art natural language processing (NLP) techniques, automated translation tools, synthetic data generation, and fine-tuned LLMs to ensure robust cross-linguistic and cross-contextual applicability. This approach is empirically validated using data from four countries—Austria, Germany, Hungary, and the United States—offering insights into the cross-national variation in how IPFs manifest in response to policy crises. Policy crises are operationalised using statistical data such as migration trends, COVID-19 mortality rates, and geopolitical events, with parliamentary speech corpora as the primary data source. Additionally, this research addresses critical methodological challenges, including linguistic variability, data availability, and the creation of synthetic datasets for fine-tuning LLMs. The proposed framework ensures replicability and scalability across diverse contexts, making it a vital resource for scholars examining political framing and crisis-driven narratives. By integrating the ILLFRAMES codebook with the Babel Machine pipeline, this study provides a robust, scalable, and replicable method for examining political framing and crisis-driven narratives by focusing on the methodological rigour necessary for studying illiberalism in varied policy contexts.