Towards an Automated Approach to Identify Policy Frames: A Comparative Analysis of Climate Change Adaptation Policy Documents in Five Dutch Municipalities

Policy frames are representations of policy issues that policy actors use to make sense of policy realities that are rife with ambiguity. Policy frames selectively communicate the most salient aspect of a policy issue, including particular problem definitions, causal interpretations, moral evaluations, and proposed solutions. Frames shape policy agenda setting, policy design and implementation, as the framing of a policy issue narrows down the policy solutions to alternatives that fit such frames. For instance, given the climate risk of urban heat, one could propose different alternatives, such as greening urban environments, housing renovation, or reduction of car use. The likelihood of an alternative being chosen depends largely on the dominant policy frame (e.g., reduction of car use may fit a frame in which congestion and fossil fuels are considered causal mechanisms for the problem). Qualitative research methods, such as interviews or desk research, are typically used to investigate policy frames. In this paper, we present a novel approach, leveraging recent advancements in Natural Language Processing, to identify frames in a (semi-)automated way, based on policy documents. Our approach consists of using zero-shot classification to select relevant segments from the policy documents, using a Large Language Model (LLM) to retrieve and generate relevant frame elements from these segments. Next, we group these elements using word embeddings and we map the relations between the grouped elements to identify problem and solution combinations that can be interpreted as policy frames. Applying this approach, we compare policy frames on climate change adaptation over time from the municipal councils of five Dutch cities, namely Amsterdam, Delft, Dordrecht, Rotterdam, and Zaanstad. To do so, we collected documents from their municipal council records addressing administrative and political decision-making on climate adaptation. These are publicly available in open government platforms. After removing duplicates and selecting only relevant documents, our final corpus consists of 4000 municipal records. Current findings show interesting results. First, our approach shows that municipalities are embracing different frames regarding climate adaptation, leading to different policy responses to a similar problem. Secondly, our approach provides consistent results in different experiments. With this, we can replicate our approach in different settings. Thirdly, our word-embeddings clustering results in an explainable group of elements, which is useful for further manual inquiry by researchers. Finally, our analysis reveals interesting associations between different frame elements, between perceived policy problems and proposed solutions.