Learning or contestation: Analyzing political biases in international organization evaluation reports

Evaluation has become a significant part of international organizations’ (IOs) policy cycles and decision-making procedures. Every year, hundreds of costly evaluation reports are produced by major international organizations (IOs) and the number is growing. These reports inform IO decision-making procedures and are generally perceived as a ‘gold standard of institutional assessment’. Yet there is virtually no comparative empirical research on the substance of IO evaluations and how the findings are presented. In this paper, we conceptualize, operationalize and measure political biases in over 1,000 evaluation reports across 9 major IOs. We argue that powerful IO stakeholders – member states and IO bureaucracies – can use evaluations to utilize their pre-defined strategic interests. Drawing on qualitative and automated text analysis, such as automated sentiment analysis, we assess both the existence and variation of such political biases in IO evaluation reports.