OECD and IEA Country Reviews and Peer Pressure: The Types and Determinants of Policy Impact

The OECD country reviews, carried out at regular intervals in various fields of public policy, and the energy policy reviews conducted by the OECD’s sister organisation, International Energy Agency (IEA), are among the most established examples of international peer reviews, and an example of the increasingly popular ‘soft’ governance tools. Lacking regulatory authority, the capacity of the OECD and the IEA to exert policy impact relies on ‘peer pressure’, as the reviews draw attention from the media, civil society, and other member country governments towards the reviewer country’s policy. Earlier research on OECD reviews has shown that peer pressure generates mainly indirect, subtle and long-term ‘conceptual’ impact, by shaping frameworks of thought and by setting agendas. Sometimes the reviews manage to provoke ‘political’ impacts, yet the extent to which the reviews manage to do so depends on the ways in which the review process and the contents of the reviews – primarily their conclusions and recommendations – interact with a host of crucial contextual, situation-dependent and country-specific factors. Drawing on empirical evidence from three international peer review processes – the OECD Environmental Performance Reviews, the OECD reviews of sustainable development (conducted in 2001-2004 as part of the OECD economic surveys), and the IEA energy policy reviews – this paper suggests a framework for analysing the significance of international peer reviews, and explaining variations in this significance. The paper employs two main strands of literature: 1) the “knowledge-use” and “utilisation of evaluation” literatures; and 2) the French pragmatic sociology, in particular its concept of “portée de l’argumentation”, which could be translated as the reach and the impact of an argument. The concept of “portée” is usually applied to the analysis of public controversies, yet in this paper the term is translated into the analysis of the “portée” of international peer reviews. The paper underlines the distinctions and interaction between learning and accountability as two main rationales of policy evaluation, distinguishes between instrumental, conceptual and political influence, and emphasises the various contextual and power-related determinants of the impact and significance of peer reviews.