Is Evidence-Based Evaluation of Deliberative Mini-Publics Methods Possible?: An Impossibility Result and its Silver Lining

[Summary] This paper investigates the logical consistency of evaluations over the methods of mini-publics based on the evidence of the expected functions of deliberation. Diana Mutz (2008) has called for evidence-based evaluation of deliberative democracy that allows us to utilize evidence from empirical research both for practice and normative research. I conversely propose that amalgamating varieties of evidence-based evaluation does not allow us to know whether certain procedures and institutions lead to fruitful deliberation. This conclusion is derived from Amartya Sen’s liberal paradox argument (1970) in social choice theory and is especially crucial for researchers and practitioners because it suggests that they hardly utilize empirical evidence to form/select appropriate forms of citizens’ deliberative participation in public policy. To alleviate the problem, I propose “issue-specific theories of deliberation” that allow researchers and practitioners to have specifiable norms and policy goals across contexts. This paper depicts how specification of normative arrangements across contexts facilitates evidence-based policy in deliberative democracy. The argument of this paper aids the recent Bächtiger and Parkinson’s claim on a contingent adaptation of the various forms of deliberation based on different deliberative goals (2019). [Literature] Current research on deliberative democracy has accumulated a long list of conditions and consequences of deliberation to date. Varieties of mini-publics methods and other forms of deliberation have been proposed and implemented without systematic evaluation of effectiveness between forms. Therefore, in her article “Is Deliberative Democracy a Falsifiable Theory?,” Mutz has called for evidence-based evaluation for deliberative democracy. Following what she calls “textbook” orthodoxy of good empirical research, Mutz encourages us to (1) streamline conditions of deliberation down to its essential elements, (2) accumulate empirical evidence (causes and effects) by testing, and (3) evaluate forms of deliberation based on empirical evidence of its functions that normative theorists anticipate. My project is to offer a critical evaluation of her third proposal from social choice theoretic perspectives, which has not been discussed in detail by Mutz or any other researchers. [Methodological Framework] My concern is that Mutz’s proposal shares the same logical structure of liberal paradox problem discussed by Amartya Sen (1970). Although originally discussed as a framework of privilege of liberty, Sen himself suggests its interpretation is open to other issues that share a similar logical structure. Thus, I translate privilege of liberty framework into privilege of evidence framework. In the same vein as Sen’s logic of the liberal paradox, I propose the impossibility of forming a consistent evaluation of various deliberation forms by amalgamating evidence-based evaluation of deliberation. [My Proposal] Instead, I argue issue-specific normative arrangements and value ordering formation known as the “specification” method in applied ethics will work well for the framework of the institutional arrangements governing evidence use in different settings. In particular, I propose “issue-specific theories of deliberation” such as climate change and immigration as an instance of specification. They allow researchers and practitioners to start the general framework of evaluation including setting up goals, specifying informational basis, specifying value standards, and eventually utilizing evidence to form consistent evaluation.