Leveraging Crowdcoding and LLMs for the analysis of gender mainstreaming: gender- sensitive recovery policies in the EU

Gender mainstreaming, the transversal consideration of gendered implications in all policy-making, has become a widespread policy commitment in Europe and is a cornerstone of the EU approach to the promotion of gender equality. Measuring gender mainstreaming presents several challenges. Firstly, its transversal nature implies it is potentially relevant in all policy documents. At the same time, gender-sensitive content is usually rare. An additional challenge of comparative analyses rests on the multi-lingual context they generally imply. This paper addresses these challenges in the analysis of gender mainstreaming in the EU National Recovery and Resilience Plans (NRRPs) by developing a novel methodological approach that combines crowd coding and LLMs. Automated translation and bag-of-words approaches revealed geographical differences in gender equality saliency across Member States. Yet these approaches have limitations in the nuanced classification of text across policy domains and — especially — in distinguishing between rhetorical mentions and policy commitments. This paper develops a scalable classification of gender-sensitive policy-relevant content. The approach refines multilingual dictionaries for the identification of gender-related content in the 27 plans with crowd-coding in their original languages, distinguishing as well across policy area and policy relevance, which serves as high-quality training data. The analysis compares traditional supervised approaches, zero-shot classification and few-shot learning across the three progressively more complex tasks. The paper contributes to advancing the analysis of gender equality, bridging nuanced qualitative coding and scalability, extremely valuable in the analysis of a transversal approach such as gender mainstreaming in the EU with low-frequency content across large multilingual corpora.