WA102 - Introduction to Conjoint Experiments

Instructor Details

Alberto Stefanelli

alberto.stefanelli.main@gmail.com

Institution:
Central European University

Instructor Bio

Alberto Stefanelli is a PhD candidate at the Institute for Social and Political Opinion Research (ISPO), KU Leuven, Belgium.

His research interests include voting behaviour, populism, radical belief systems and conspiracy theory.

Methods-wise, he is particularly interested in graphical causal models, standardisation techniques and matching algorithms, text analysis, experimental and semi-experimental design, and machine and deep learning.

He is currently co-principal investigator of the Belgian National Electoral Studies and of the Belgian Minority Study.

@sergsagara

Course Dates and Times

Friday 14 February 13:00–15:00 and 15:30–18:00

Saturday 15 February 09:00–12:30 and 14:00–17:30

Prerequisite Knowledge

The course assumes intermediate familiarity with the basis of experimental design, survey experiments and regression analysis.

The empirical analysis will be implemented using R. While example datasets and full syntax codes will be provided, intermediate knowledge of R is expected.

You should know how to:

read datasets in R
work with data frames
perform basic data manipulation
run basic statistical analyses such as linear or logistic regression.

More advanced knowledge of statistical computing, such as writing functions and loops, is helpful but not essential.

Short Outline

This course is an introduction to the logic and basic notions of conjoint analysis (CJA). Particular attention will be paid to Choice-Based design: the analysis, the interpretation of the results, and the limitations of conjoint experiments. We will apply these notions by looking at different applications of conjoint analysis within the field of behavioural sciences. The course will focus in particular on:

1. What is a conjoint experiment?
2. The different type of conjoint analysis and their applicability in social sciences.
3. The advantages of conjoint experiments compared to other designs.
4. How to design and deploy a conjoint experiment
5. How to analyse and interpret the results of a conjoint experiment

Tasks for ECTS Credits

1 credit (pass/fail grade). Attend at least 90% of course hours, participate fully in in-class activities, and carry out the necessary reading and/or other work prior to, and after, class.

Long Course Outline

Estimating causal effects is a central aim of quantitative empirical analysis in social sciences. Recently, Conjoint Analysis and Choice-Based Conjoint Experiments have gained interest among social scientists to understand and predict people's preferences in a multi-dimensional and multi-choice environment. This course offers an applied introduction to Choice-Based Conjoint, along with hands-on experience in lab sessions.

By the end of the course, you will:

1. Have a basic understanding of the structure, logical underpinnings, basic notions, and analytical goals of conjoint analysis.
2. Identify areas of application where conjoint analysis could be successfully implemented.
3. Critically evaluate conjoint experiment applications and understand the advantages/disadvantages compared to traditional methods.
6. Implement your own conjoint experiment into an (online) survey platform.
7. Understand and be able to apply different techniques to analyse conjoint experiments.
8. Be able to efficiently visualise your results
10. Be well prepared for more advanced conjoint and factorial design courses or workshops.

The course is structured around six key topics:

I present the general idea of conjoint experiments. I introduce the logic underlining conjoint experiments, their development, and the reasons behind the rising interest in them within the social sciences.
I briefly introduce the 'utility' function at the base of Choice-Based Conjoint analysis. In particular, I give an overview of the Random Utility Theory and discuss it within the framework of social and behavioural science.
I present different ways to measure individual preferences in a conjoint experiment. The focus will be on Choice-Based Conjoint measurement but we will briefly discuss other measurements (e.g. Rating, Ranking, Combined, and Adaptive).
I give an overview of different types of conjoint design, their use, and their limitations. I present and explain Alternatives, Choice Sets and Context design, paying particular attention to the design of conjoint alternatives.
I focus on the construction and implementation of a conjoint experiment. Specifically, I show how to design alternatives with attributes/levels constraints and randomisation using a ready to use JavaScript/Python program and R.
I give an overview of different methods to analyse a conjoint experiment. Specifically, I focus on AMCEs, marginal mean and omnibus F-test. We will briefly discuss subgroup differences and visualisation.

Note The goal of this workshop is to give an applied introduction to conjoint experiments. If you are already familiar with conjoint analysis or you are interested in the broader theory behind conjoint and factorial experiments, this is not the right course for you.

Day-to-Day Schedule

Day

Topic

Details

Introduction
History
Utility Function and Design

1.5h – Session I

Course Overview and Introductions
A quick refresher on experimental design and causality
A brief history of conjoint analysis (CJA)

1.5h – Session II

Translating theories into experiments: Random utility theory (RTU) and utility function in CJA
Different types of CJA and their applicability in social sciences

1.5h – Session III

Design a CJ experiment and dealing with attributes/levels constraints
Implementing attributes/levels constraints and randomisation using JavaScript and Python
Designing HTML tables with randomised CJ profiles

Deployment, Analysis and Visualisation

1.5h – Session IV

Implement a conjoint experiment on Qualtrics
Internal and External validity in a CJA: recommendations and guidelines for social scientists

1.5h – Session V

Analysing a conjoint experiment: strategies and modelling approaches
Brief introduction to AMCEs, marginal mean and omnibus F-test
Subgroup differences and moderators

1.5h – Session VI

Plotting and visualisation
Carryover and Left/Right Diagnostics
Advances in CJA

Day-to-Day Reading List

Day

Readings

Morton, R.B. & Williams, K. (2010)
Experimental Political Science and the Study of Causality: From Nature to the Lab Chapters 2 and 7
Cambridge University Press

Gustafsson, A., Herrmann, A., Huber, F. (Eds.) (2010)
Conjoint Measurement, Methods and Applications Chapter 2
Springer

Auspurg, K. & Hinz, T. (2015)
Factorial Survey Experiments Chapter 3
Series: Quantitative Applications in Social Science
Sage Publications

Knudsen, E., & Johannesson, M. P. (2018)
Beyond the Limits of Survey Experiments: How Conjoint Designs Advance Causal Inference in Political Communication Research
Political Communication, 0(0), 1–13

Hainmueller, J., & Hopkins, D. J. (2015)
The Hidden American Immigration Consensus: A Conjoint Analysis of Attitudes toward Immigrants
American Journal of Political Science, 59(3), 529–548

Hainmueller, J., Hangartner, D., & Yamamoto, T. (2015)
Validating vignette and conjoint survey experiments against real-world behavior
Proceedings of the National Academy of Sciences, 112(8), 2395–2400

Horiuchi, Yusaku, Daniel M Smith and Teppei Yamamoto. 2015
Measuring Voters’ Multidimensional Policy Preferences with Conjoint Analysis: Application to Japan’s 2014 Election
Available at SSRN 2627907

Strezhnev, A., Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2013)
Conjoint Survey Design Tool: Software Manual

Leeper, T. J., Hobolt, S. B., & Tilley, J. (2018)
Measuring Subgroup Preferences in Conjoint Experiments
Political Analysis 55

Kaczmirek, L. (2015)
Conducting web surveys: Overview and introduction
In Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research, Chapter 13
Routledge

Toepoel, V. (2016)
Doing Surveys Online, Chapters 6 and 15
Sage

Callegaro, M., Manfreda, K. L., and Vehovar, V. (2015)
Web survey methodology, Chapters 5, 6 and 7
Sage

Software Requirements

Please bring your laptop with the latest version of R (3.6.x, Planting of a Tree), R Studio (1.2.x) and Python 2.7 installed. Note that Python 2.7 and Python 3.7 are not the same. Unfortunately, the softwere required by the randomisation mechanism is still running on Python 2.7.

Make sure they work and that you can run a script before coming to class since we will have no time to resolve technical issues. If you have already collected data, bring it along. If not, you’ll get a toy dataset to play with. Be sure to have installed in R the cjoint and cregg packages together with any other package that you use for data management/cleaning/visualisation (e.g. dplyr,ggplot, etc).

This course will use R, which is a free and open-source programming language primarily used for statistics and data analysis. Although you are allowed to use other solutions, we will also use RStudio, which is an easy-to-use interface to R.

If you encounter any problems with R Studio on your local machine, use R Studio Cloud.

Hardware Requirements

Please bring your laptop.

There are no specific requirements other than being able to use a browser (for dataset downloads and R Studio Cloud) and having installed R Studio and Python 2.7.

The following other ECPR Methods School courses could be useful in combination with this one in a ‘training track .

Recommended Courses After

Survey Design

Logistic Regression and General Linear Models

Methods of Modern Causal Analysis Based on Observational Data

Multi-Level Modelling

Additional Information

Disclaimer

This course description may be subject to subsequent adaptations (e.g. taking into account new developments in the field, participant demands, group size, etc). Registered participants will be informed in due time.

Note from the Academic Convenors

By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach these prerequisite items. If in doubt, contact the instructor before registering.

Share this page

Install the app