Advancing Content Analysis Foreign Policy Analysis Through Semantic Triple Extraction

The increase in available political data has led to a need for reliable innovative approaches to analyse this information effectively. This is especially the case in the domain of Foreign Policy Analysis (FPA), which traditionally relies heavily on historical methods, qualitative text analysis and case studies. With the increase in available data regarding the beliefs and actions of states and leaders, FPA scholarship has started to combine qualitative and quantitative methods, including computer assisted content analysis. However, scholars face the challenge of combining the efficiency of automated content analysis with the need for in-depth analysis. We explore the methodological advancements in Semantic Triple Extraction and how these may help to extricate the in-depth and nuanced information needed to answer common FPA research questions. Semantic triples consist of subject-predicate-object structures, which capture the relationships between entities in a text. This method allows for a deeper understanding of the underlying semantics and context, making it a valuable tool for political data analysis. We focus specifically on extracting causal relations (cause-effect-signal triplets) as has been used within FPA in cognitive mapping research. Although automated content analysis has evolved significantly with the advent of large language models (LLMs) and more advanced rule-based extraction techniques, automatic extraction of causal relations remains complex. Methods leveraging LLMs have shown remarkable capabilities in identifying patterns and extracting meaningful information from unstructured data. However, LLMs struggle with the reliable extraction of triples and nuanced or context-specific information. Moreover, automated methods excel in handling large datasets and uncovering hidden patterns but may require significant computational resources and training data. Rule-based methods are more transparent, provide in depth and interpretable results, and consume far fewer computational resources but may lack the flexibility to adapt to diverse datasets as they can be time-consuming for individual research programs to develop and maintain. In our paper, we introduce and compare two methods to extract causal semantic triples from a text: a rule-based method using Profiler Plus and an LLM-based method using Llama 3.1 70B (upgrading to better Llama models as they become available). These methods will be tested by analysing all available speech-acts by Russian President Putin and US President Trump (respectively 9236 and 4835 to date). The performance and usefulness of the methods will be evaluated based on several criteria, including accuracy, scalability, adaptability, and resource requirements. The findings of this paper will provide insights into how the different variants of automated Semantic Triple Extraction can be effectively utilized in political research and contribute to the understanding of new methodologies in political data analysis in general and FPA in particular.