Text analysis of international organizations: A new framework to leverage large-scale data

International organizations’ everyday operations leave a paper trail—speeches, reports, agreements—which scholars increasingly scrutinize, owing to advances in computational methods and the digitization of archival documents. Methods may have evolved, but I argue we also need to refine and adapt our theoretical toolkit to fully leverage these large-scale data. In this article, I introduce a framework for analyzing textual data based on critical discourse analysis. It varies across two dimensions: a) type of author; and b) the function a text serves. The configuration of these two aspects determines which research questions are amenable to the study of text (e.g., x- or y-centered research design) and theoretical requirements. To illustrate how the framework matters in practice, I discuss the data-generating process of speeches in the IMF’s decision-making body based on semi-structured elite interviews. In addition, I demonstrate the sensitivity of quantitative text analyses by comparing observed speeches with simulated data from GPT-3, the most advanced text autocomplete program to date. Taken together, this study advances scholarship in international politics by showing how to best leverage large-scale textual data. At stake is not merely a question about the data-generating process of texts, but one about the scope for permissible inquiries with such data.