ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

Your subscription could not be saved. Please try again.
Your subscription to the ECPR Methods School offers and updates newsletter has been successful.

Discover ECPR's Latest Methods Course Offerings

We use Brevo as our email marketing platform. By clicking below to submit this form, you acknowledge that the information you provided will be transferred to Brevo for processing in accordance with their terms of use.

Experimental Research: Theory Development, Hypothesis Generation and Empirical Tests

Course Dates and Times

Monday 5 – Friday 9 August 

14:00–15:30 / 16:00–17:30, ending slightly earlier on Friday

 

John Jost

john.jost@nyu.edu

New York University

This course will involve little lecturing and will focus instead on students’ development of their own theories, hypotheses, and research programs. By leveraging William J. McGuire’s philosophy of science and the perspectivist method of planning research in behaviouural science, we will emphasise creative hypothesis generation and theory building.

You will complete exercises designed to clarify the conceptual foundations of your own work and help you develop multiple hypotheses concerning empirical relationships among independent, dependent, mediating, and moderating variables. We will develop useful manipulations and measurement strategies that overcome common methodological problems.

On several occasions, I'll ask you to present hypotheses, research designs, and experimental procedures to the rest of the class, and we will engage in group discussion and constructive evaluation. You can incorporate this feedback to refine your plans, with the long-term goal of conducting sophisticated programs of research on interesting ideas of your own choosing.

ECTS Credits for this course and below, tasks for additional credits:

3 credits Prepare an in-class presentation based on your own research (mandatory for all participants).

4 credits As above, plus complete a take-home paper – deadline will be set during the course.


Instructor Bio

John T. Jost is Professor of Psychology and Politics and Co-Director of the Center for Social and Political Behaviour at New York University.

His research, which addresses stereotyping, prejudice, political ideology, and system justification theory, has been funded by the National Science Foundation, has appeared in top scientific journals and received national and international media attention.

He has published more than 200 journal articles and book chapters and four co-edited book volumes, including Social and Psychological Bases of Ideology and System Justification (Oxford, 2009).

John has received numerous honours and awards, including the Gordon Allport Intergroup Relations Prize, the Erik Erikson Award for Early Career Research Achievement in Political Psychology, the International Society for Self and Identity Early Career Award, the Society for Personality and Social Psychology Theoretical Innovation Prize, the Society of Experimental Social Psychology Career Trajectory Award, and the Morton Deutsch Award for Distinguished Scholarly and Practical Contributions to Social Justice.

He has served on several editorial boards and executive committees of professional societies and is currently editor of the Oxford University Press book series on Political Psychology.

John Jost is a Fellow of the Society of Experimental Social Psychology and the Association of Psychological Science, and is Past President of the International Society of Political Psychology.

  @JohnJost1

This course is unusual in the sense that it will involve very little lecturing. Instead, we will focus squarely on students’ development of their own research ideas, interests, questions, theories, and hypotheses. We will begin by considering William J. McGuire’s post-Popperian philosophy of science and his perspectivist method of planning and implementing programmatic research in the social and behavioural sciences. The focus of this method is on creative hypothesis generation and theory building rather than on hypothesis testing and theory disconfirmation per se.

Specifically, I will ask you to complete a series of written exercises designed to clarify the conceptual foundations of your own research programmes and, in the process, develop multiple hypotheses concerning non-obvious empirical relationships among independent, dependent, mediating, and moderating variables that are of genuine theoretical interest. You will present your hypotheses to the rest of the class, and we will engage in group discussion and constructive evaluation.

We will discuss the major advantages of developing experimental programmes of research in the social and behavioural sciences, such as leveraging the method of strong inference, and the four stages of experimentation, namely:

  1. setting the stage for the experiment
  2. constructing the independent variable
  3. measuring the dependent variable
  4. planning the post-experimental follow-up.

We will work on developing meaningful operationalisations of variables, including manipulations as well as measurement strategies, and we will seek to avoid and overcome common methodological problems or 'impurities,' which can introduce interpretational (and rhetorical) threats and ambiguities. You will present your own experimental designs to the rest of the class, and we will engage in group discussion and constructive evaluation.

Based on feedback at several stages, you will work to refine your hypotheses, research designs, and experimental procedures, with the eventual long-term goal of independently conducting sophisticated, theory-driven programmes of research on interesting ideas of your own choosing.

You must have completed the first-week course Introduction to Experimental Research in the Social Sciences or obtained equivalent knowledge elsewhere.

I assume you will have already conducted your own empirical research (typically in the context of a social science PhD) and have ongoing plans to conduct research that lends itself to experimental methods.

I will ask you to present your own ideas for research, and to engage in theory building, hypothesis generation, and the design of careful, sophisticated experiments for investigating those hypotheses.

This is possible only if you are actively pursuing empirical research programmes in the social and behavioural sciences.

Day Topic Details
1 Scientific theories and criteria for their evaluation; post-Popperian philosophy of science; creative hypothesis generation; the perspectivist method

Both sessions will involve a mix of background lectures and discussion, as well as individual and group activities focused on students’ own theoretical development

2 Presentation and discussion of hypothesis generation exercise (perspectivist worksheets)

These sessions will involve student presentations of initial and contrary hypotheses (according to the perspectivist method) followed by group discussion/constructive evaluation

3 Presentation and discussion of (rough) experimental designs; operations and other logistical issues

These sessions will involve student presentations of initial experimental designs (and procedures) followed by group discussion

4 Strong inference; overcoming experimental 'impurities'; improving your manipulations and measures

In addition to methods of strong inference, we will discuss common problems with the conceptualisation and operationalisation of independent and dependent variables and other interpretational threats

5 Presentation and discussion of theoretical ambitions and (more polished) experimental procedures

Students will describe (and receive feedback on) their plans for conducting theory-driven experimental research programmes on interesting ideas of their own choosing

Day Readings
1
2
  • William J. McGuire (1989). Strategic planning of programs of research. Selection from: The Psychology of Science: Contributions to Metascience (pp. 224-245).
  • Prior to Day 2, participants will complete worksheets made available electronically and adapted from William J. McGuire (2004). Perspectivist worksheets for generating a program of research. In J.T. Jost et al. (Eds.) Perspectivism in Social Psychology (pp. 319-324).
3
  • Hendrick, C., & Jones, R.A. (1972). Postscript (on the method of strong inference).  Selection from: The Nature of Theory and Research in Social Psychology (pp. 355-360). New York: Academic.
  • Timoth D. Wilson et al. (2010). The four stages of laboratory experimentation. Selection from: Handbook of Social Psychology (Fifth Ed., Vol. 1, pp. 63-78).
4
5

Software Requirements

None.

Hardware Requirements

Participants are required to bring their own laptop.

Literature

Background (Foundational)

Druckman, J. N., Green, D. P., Kuklinski, J. H., & Lupia, A. (Eds.). (2011). Cambridge Handbook of Experimental Political Science. Cambridge University Press

Kittel, B., Luhan, W. J., & Morton, R.B. (Eds.). (2012). Experimental Political Science: Principles and Practices. Palgrave-Macmillan.

Morton, R.B. & Williams, K. (2010). Experimental Political Science and the Study of Causality: From Nature to the Lab. Cambridge University Press.

Foreground (Required)

Abelson, R.P. (1995). Impurities in experiments. Selection from: Statistics as Principled Argument (pp. 189-198). Hillsdale, NJ: Erlbaum.

Druckman, J.N., & Lupia, A. (2012, March 9). Experimenting with politics. Science, 335, 1177-1179.

Hendrick, C., & Jones, R.A. (1972). Postscript (on the method of strong inference).  Selection from: The Nature of Theory and Research in Social Psychology (pp. 355-360). New York: Academic.

McGuire, W.J. (1989). Strategic planning of programs of research. Selection from: B. Gholson, W.R. Shadish, Jr., R.A. Neimeyer, & A.C. Houts (Eds.), The Psychology of Science: Contributions to Metascience (pp. 224-245). New York: Cambridge University Press.

McGuire, W.J. (1997). Creative hypothesis generating in psychology: Some useful heuristics. Annual Review of Psychology, 48, 1–30.

McGuire, W.J. (2004). Appendix: Perspectivist worksheets for generating a program of research. In J.T. Jost, M.R. Banaji, & D. A Prentice (Eds.) Perspectivism in Social Psychology (pp. 319-324). Washington, DC: APA.

McGuire, W.J. (2013). An additional future for psychological science. (Edited with an introduction by J.T. Jost). Perspectives on Psychological Science, 8, 414-423.

Wilson, T.D., Aronson, E., & Carlsmith, K. (2010). The four stages of laboratory experimentation. Selection from: Handbook of Social Psychology (Fifth Ed., Vol. 1, pp. 63-78).

Supplemental (Optional)

Festinger, L., & Carlsmith, M. (1959). Cognitive consequences of forced compliance. Journal of Abnormal and Social Psychology, 58, 203-210.

Lord, C.G., Lepper, M., & Preston, E. (1984). Considering the opposite: A corrective strategy for social judgment. Journal of Personality and Social Psychology, 47, 1231-1243.

Recommended Courses to Cover Before this One

Summer School

Introduction to Experimental Research in the Social Sciences

Recommended Courses to Cover After this One

Summer School

Structural Equation Modelling
Multilevel Modelling

Winter School

Structural Equation Modelling
Multilevel Modelling