The organization of evaluations: The influence of the ministry of finance on evaluation systems

In recent years, governments worldwide have sought to further the systematization and institutionalization of expert knowledge in processes of policymaking to increase the use of such knowledge by policymakers. One example of the institutionalization of knowledge is the establishment of policy evaluation systems. Evaluation systems can be understood as institutionalized evaluation cultures, which come with certain epistemic standards, regulations and processual guidelines for the evaluation of planned, ongoing or completed policies. Research on knowledge utilization have thus increasingly turned toward the question of how to develop and maintain such evaluation systems for them to increase evaluation use. However, we still know little of the actual effects of these evaluation systems and whether they further some forms of evaluation use rather than others. This paper addresses this gap by analyzing how the organization of evaluation systems within the central administration of the Netherlands and Denmark influences the function and use of evaluations in policymaking processes. We specifically focus on the role of the Ministry of Finance, as this ministry provide the institutional anchorage of the evaluation system in the two countries, as well as in many other OECD countries. We apply the concept of the constitutive effects of evaluations to elucidate how the ministry of finance not only influence the function and use of the evaluation system through formal (regulatory) means, but also through informal means (e.g. norms and discourses) which create unintended consequences. The two cases are analyzed through a combination of document analysis (evaluation guidelines and evaluation reports), and a series of interviews with centrally placed civil servants in the Ministries of Finance, Court of Audit, and the Ministries of Labor and Public Health in both countries. While it is often assumed that the systematization and institutionalization of procedures for knowledge production, -dissemination and -use within policymaking organizations and processes will naturally increase the influence of such knowledge on policymaking, our study clearly nuances this assumption. We highlight how the anchorage of the evaluation function in the Ministry of Finance constitutes a specific economic outlook on evaluation, which 1) narrows down the applied evaluation methods and criteria; 2) changes the primary evaluation user from the parliament to the ministry of finance; and 3) inserts accountability rather than learning as the main form of use. Both evaluation systems thus mainly supports "defensive" functions of keeping checks and balances on cost spending – which ultimately hinders the systems in supporting more significant policy changes and forms of learning.