The Rational Design of International Institutions in the Age of AI: Turning Continents into Islands?

In this manuscript, I replicate key data points gathered by Barbara Koremenos (2016) for her seminal study on the rational design of international institutions known as the “Continent of International Law” (COIL). Originally, collecting these data required a multi-person, multi-year effort to conquer a metaphorical “continent”. Today, roughly a decade later, to what extent is it possible to replicate these data with Large Language Models (LLMs)? I find that key elements of the COIL project related to treaty-design features can be reproduced by a single researcher in a matter of hours and at a fraction of the cost, illustrating how the empirical study of international cooperation has literally been transformed from continents—into islands. Still, replicating the presence of the cooperation problems at the heart of COIL proves less promising. I end with a note of caution on how proprietary models still hold a qualitative edge today, putting researchers in the uncomfortable position of having to choose between replicability and data quality.