Ranking Business Trade Preferences Using GPT

This article evaluates Large Language Models' (LLMs) advantages and limitations for analyzing firms’ and associations’ trade preferences. Drawing on 1,557 public positions from US trade negotiations, I pair comments into dyads and use OpenAI’s GPT to determine which actor more explicitly supports investment protection. I then aggregate these pairwise choices into a Bayesian Bradley–Terry index to rank organizations by inferred support intensity. The resulting ranking aligns with theory and evidence, opening new avenues for hypothesis testing. GPT’s classifications are generally robust, with uncertainty remaining acceptable even when sampling under 2\% of all comparisons. Yet, GPT classifications based on training data poorly correspond with those derived from public comments, and small prompt variations cause significant output changes. Within-cluster correlation also influences the confidence in the estimates. In sum, LLM can be useful, but with caution. I end the article with recommendations for future works.