Large Language Models and the Political Science Research Process: Insights on Potentials and Risks from Reflexive Self-Ethnographic Case Studies

Large Language Models (LLMs) have become widely used in Political Science, but their use is often accompanied by uncertainty. How can we use LLMs? Should we use them? In what ways might they limit us in our research? In our paper, we explore how LLMs can mitigate common pain points in the Political Science research process and where they should be avoided. We employ a reflexive self-ethnographic methodology to examine the role of LLMs in our research practice. Rather than collecting external data, we draw on our experiences and concrete interactions with LLMs. This approach allows us to develop case-based insights into how LLMs can shape reasoning, writing, and decision-making throughout the different stages of the research process. We plan to adopt a two-fold comparative approach: In addition to comparing the uses of various models for specific research tasks, we will compare our uses of LLMs as junior and senior scholars. More specifically, we provide case studies of LLM applications in political research on autocracies’ security ties, the Chinese military’s thinking about future wars, and the use of LLMs in cyber conflicts and cybersecurity. Based on these examples, we will reflect on how drawing on LLMs impacts the research process. Finally, we aim to synthesize our findings in a summary table that provides a clear overview of use cases, benefits, and potential pitfalls. We argue that a structured and reflexive use of LLMs can enhance political science research, but must be accompanied by careful and ethical decision-making and verification. By providing a reflective account of how we use these tools and hands-on advice on how to use them and for what purposes, we contribute to the discussion of the impact of LLMs on political science and the ethical rules that should govern their use.