"Hi voter, I can assist you in filling out this Voting Advice Application!" How CAVAAs may assist voters in informing themselves about elections

Theoretical Background: Voters who fill out a Voting Advice Application frequently encounter comprehension problems (Kamoen & Holleman, 2017). Instead of looking up missing information, for example, about the meaning of a term, voters make assumptions about the statement's meaning and provide an answer nevertheless. These answers are disproportionally often middle-option or no-opinion answers, which hampers the validity of the voting advice. To help users to inform themselves, Conversational Agent Voting Advice Applications (CAVAAs) have recently been introduced (e.g., Kamoen & Liebrecht, 2022). In these tools, users can request additional information from a conversational agent while filling out a VAA. While first studies show that CAVAAs are indeed perceived to be more useful than regular VAAs these studies can be critiqued as they were conducted outside a real-life election context. Moreover, the construct validity of the experimental materials in these studies can be improved as previous VAAs and CAVAAs also differed in their layout. We, therefore, conducted two new experimental studies comparing a VAA to a CAVAA that address both issues: 1) the studies were conducted in the run-up to the Dutch National Elections of November 22, 2023; 2) the CAVAA was added as a widget to an existing VAA, guaranteeing a comparable layout. Materials: The regular VAA was inspired by the Dutch Stemwijzer, and included 30 political attitude statements about current Dutch politics. In the CAVAA condition, a chatbot was added at the bottom right corner of the screen. Here, users could ask questions about 1) semantic information, 2) pragmatic information, 3) pros and cons, 4) party positions towards the VAA statements. Participants could obtain this information either by clicking on predefined buttons or by typing in their questions in an open chat window.AI was used to recognize the intent of the CAVAA user, but no (generative) AI was used for the CAVAAs’ responses as we researchers always wrote these to ascertain that the answers were not biased towards a particular political party. Study 1: In a laboratory setting, 149 university students were randomly assigned to one of the tool versions. After obtaining voting advice, they filled out a survey evaluating the tool. We also content analyzed the chatbot conversations and as well as the proportion of non-substantive answers provided to the VAA statements. Preliminary findings show that students perceive the tools equally high in ease of use, and playfulness. However, users of the CAVAA indicated the tool to be more useful and felt more informed to vote than users of the regular VAA. In addition, CAVAA users provided fewer non-substantive answers as compared to VAA users. Study 2: We replicated study 1 among a more diverse population of citizens (N = 150) and the findings were very comparable: a CAVAA is perceived to be more useful than a regular VAA and also the proportion of non-substantive answers is lower in a CAVAA condition. At the ECPR conference, we will discuss the implications of these findings for theory and practice.