Evaluating the Quality of Policy Changes

The ACF and competing policy process theories tend to conceptualize policy content in the form of policy change, which involves policy alteration but not necessarily problem solving. Scholars aim to reveal the conditions under which long periods of stability (during which policies do not change) are interrupted by short phases of change. To this end, typologies have been developed which distinguish major, minor, or incremental policy changes. While such typologies categorize the magnitude of policy alterations between two points in time, they are less specific about the quality of change in terms of a positive or a negative direction of change. Nevertheless, it is an inherent aim of policy scholars to enhance our understanding of policy improvements in terms of their ability to alleviate societal issues. Towards this goal, the present article addresses the question: How can we evaluate the quality of policy changes in terms of improvements (or deterioration) in addressing public problems? The paper conceptualizes policies as more or less comprehensive solutions to an underlying policy problem. To evaluate the performance of policies from a problem-solving perspective, the policy comprehensiveness index is introduced here as a new analytical tool. The index can be employed to study a change from ‘less comprehensive’ to ‘more comprehensive’ policy designs over time. Based on lessons from existing policy instrument literature, six indicators feed into the proposed index. Such indicators enable comparison of policy variation across policy fields, countries, or time, and hence, the study of policy changes. In order to render complex, multi-dimensional aspects of the empirical reality, such as policy design, measureable, indexing is employed as a method for quantifying policy designs by means of qualitative data. The paper applies the policy index to the case of water protection policy to evaluate the performance of the studied policy designs in reducing pollution.