Benchmarks of Deliberation

In the EU and around the world, democratic innovations have often taken the form of deliberative public engagements. Varying in design, these forums provide an environment in which participants can come together under conditions of equality and mutual respect to discuss policy decisions or inform potential policies in emerging areas of concern. The core feature that makes these forms of public participation deliberative is the exchange of reasons which, in combination with other criteria, is meant to ensure that the ‘force of the better argument’ is the only form of influence at work. With the spread of deliberative public engagement, researchers have begun to ask how to evaluate the quality of deliberation. Procedural approaches to measurement and evaluation have generally relied on in-depth coding and analysis of transcripts or web-posts. Although these methods offer detail and precision, they also require a significant investment of resources and time on the part of trained researchers in order to draw conclusions about the quality of deliberation. A more manageable evaluation method would render deliberation more accessible as a policy tool while enhancing claims to legitimacy by demonstrating the presence of an equal and respectful exchange of reasons. in this paper, we suggest a mixed-method framework for observing and evaluating deliberation, relying in large part on structured observation by dedicated evaluators. This evaluation system will balance the need for precision and validity that motivates transcript analysis with a need for practical usability. Drawing on theories of deliberative democracy, the existing literature on evaluating deliberation, and the results of four deliberative public engagements in the field of science, technology and genomics, we will put forward an operationalized set of “benchmarks” for deliberation and suggest a series of strategies for applying them to identify whether, and to what extent, deliberation has taken place.