ECPR Winter School
University of Bamberg, Bamberg
2 - 9 March 2018




WD205 - Advanced Multi-Method Research: Principles and Practice

Instructor Details

Instructor Photo

Ingo Rohlfing

Institution:
University of Cologne

Instructor Bio

Ingo Rohlfing for Methods of Comparative Political Research at the Cologne Center for Comparative Politics at the University of Cologne.

He is doing research on social science methods with a focus on qualitative methods (case studies and process tracing), Qualitative Comparative Analysis and multimethod research.

He is author of the monograph Case Studies and Causal Inference (Palgrave Macmillan) and has published articles in Comparative Political Studies, Sociological Methods & Research and Political Analysis.

  @ingorohlfing

 


Course Dates and Times

Monday 5 to Friday 9 March 2018
25 hours over 5 days
09:00-12:30 and 14:00-16:30

Prerequisite Knowledge

A central element of the course is the in-depth discussion of the participants’ research projects. It is strongly recommended that participants are working on a mixed-methods study when taking the course and have advanced to a stage allowing them to present and discuss their study in class.

The course does not teach the basics of regression analysis, QCA, case studies, process tracing, or any other method one might use in multi-method research. Participants are expected to have acquired skills on these methods when taking this course because it specifically focuses on how to combine them. Having sufficient knowledge of the methods you aim to use is important because it is essential for understanding some of the principles of multi-method research.

Short Outline

This course deals with mixed-methods/multimethod research (MMR) in the social sciences. The course aims to reflect the diversity of MMR studies in the discipline to accommodate the participants’ projects. We will discuss the broader understanding of combining qualitative and quantitative methods and the more focused approach linking case studies and process tracing to a large-n method such as Qualitative Comparative Analysis (QCA) or regression analysis. The relative emphasis we put on specific variants of MMR designs will be adapted to the methods the participants are applying in their own research.

The goal of the course is to understand the different varieties in which MMR can be done. We discuss the unique advantages and methodological and practical challenges confronted in implementing multi-method designs. Topics include concepts in the qualitative and quantitative analysis, case selection for qualitative research, and the compatibility of theoretical expectations and inferences on causal effects and causal mechanisms. Method-centered discussions are illustrated with examples from different fields of political science and the participants’ projects in the afternoon sessions. At the end of the course, participants are able to realize their own mixed-methods study in a systematic manner and to critically evaluate published MMR studies.

Tasks for ECTS Credits

  • Participants attending the course: 4 credits (pass/fail grade) The workload for the calculation of ECTS credits is based on the assumption that students attend classes and carry out the necessary reading and/or other work prior to, and after, classes.
  • Participants attending the course and completing one task (see below): 6 credits (to be graded)
  • Participants attending the course, and completing two tasks (see below): 8 credits (to be graded)
  1. Participants get 4 ECTS points for attending the course, reading the readings and participating actively.
  2. Participants get 6 ECTS points for attending the course, submitting a daily assignment on day 1, 2, 3 and 4. The assignment is based on the discussion of the participants’ projects in class.
  3. Participants get 8 ECTS points for attending, submitting the daily assignments and a 10-15 page description of the mixed-methods research design as it stands after having taken the course.
Long Course Outline

Mixed-method research is an enduring topic in the social sciences (e.g., Creswell and Piano 2011), but multi-method research (MMR) more narrowly defined, a.k.a. as nested analysis, is a relatively new topic. After longstanding antagonistic discussions about the pros and cons of qualitative and quantitative methods, we now find a growing consensus that each method has its distinct advantages and that they work best in combination with each other. This course builds on the debate about mixed-methods and multimethod research and focuses on their unique advantages and challenges for empirical researchers seeking to combine two or more methods.

On day 1, we lay the foundation for the rest of the course. We learn about different conceptions of causation and how they can be integrated in a single mixed-methods analysis. We further introduce different varieties and dimensions of mixed-methods research such as the timing of the methods’ applications and their relative importance in the context of the broader design. In the afternoon session of day 1, participants briefly present their projects, locate their own study in the mixed-methods framework and are invited to present the challenges they currently confront.

On day 2, we begin with a reflection on concepts and concept formation in MMR as the cornerstone of all empirical research. The session is based on two interrelated claims one finds in the methods literature. First, it is argued that concepts are thin in quantitative and thick in qualitative research. Second, it is claimed that this discrepancy creates problems of conceptual stretching undermining causal analysis. We elaborate on whether these assertions are warranted and, to the extent that they are accurate, how concept formation can be improved. In the afternoon session, we discuss the participants’ concepts in detail and seek strategies for avoiding conceptual pitfalls in their designs.

The topic of day 3 is case selection. Depending on the mixed-methods design, the case selection challenges can be diverse and range from the recruitment of participants for focus groups to the residual-based choice of countries after a macro-comparative regression. We will discuss the current state of the debate about case selection in nested analysis (regression or QCA with follow-up process tracing) and reflect upon additional case selection strategies pertinent to the participants’ designs (which are not known yet, so I cannot say more on this at the moment). The general methodological discussion is blended with the discussion of the participants’ intended case selection strategies, their challenges and possible fixes.

On day 4, we turn to the question of generalization. Generalization is considered in parts of the mixed-methods literature and ignored in others. The main focus lies on the generalization of the inferences generated in the qualitative analysis. According to a common line of reasoning, generalization of qualitative conclusions is only possible under conditions that are difficult, if not impossible, to reconcile with the quantitative part. We will discuss generalization strategies for different variants of mixed-method designs and in application to the participants’ studies.

The last day, day 5, covers a wrap-up session. We take a closer look at published empirical studies with regard to the topics of day 1 to 4 and with a specific focus on what can be called causal consistency or causal coherence. This means that one’s theoretical expectations as regards the large-n results and process tracing insights should fit with each other. Similarly, the conclusions that one derives from quantitative and qualitative analyses should be coherent. For example, a lack of fit occurs when process tracing leads to the conclusion that multiple factors work in conjunction, while the regression analysis models the effect of covariates as independent from each other. We discuss several sources and manifestations of inconsistency and strategies for achieving coherence over the course of the analysis. In the afternoon, participants present what they learned over the week, how this influences their plan for the mixed-methods project and what new challenges they realized they confront.

Day-to-Day Schedule

Day 
Topic 
Details 
1Introduction to mixed-methods research

Morning:

  • Varieties of multi-method research (MMR)
  • Theories of causation and MMR
  • Course goals

Afternoon:

  • Presentation of participants’ projects
  • Location of studies in mixed-methods framework
  • Current challenges in MMR analysis
5Wrap-up session and lessons learned

Morning:

  • Applying insights to published mixed-methods studies
  • Coherence of qualitative and quantitative analysis

Afternoon:

  • First lessons learned by the participants and presentation of new ideas and challenges that came up during the week
2Concepts and concept formation

Morning:

  • Thin and thick concepts
  • Risks of conceptual stretching
  • Conceptual consistency in MMR

Afternoon:

  • Discussion and elaboration of participants’ concepts
  • Strategies for avoiding conceptual pitfalls
3Case selection

Morning:

  • Short intro to nested analysis (identifying types of cases and selection strategies)
  • General challenges in other mixed-methods designs of participants’

Afternoon:

  • What are cases in participants’ projects and how best to select them?
  • Practical and methodological challenges for case selection for the participants’ projects
4Case selection & generalization

Morning:

  • What is generalized in mixed-methods research?
  • Assumptions and risks in generalizing
  • Strategies to strengthen generalization claims

Afternoon:

  • Identifying generalization claims in participants’ projects
  • Determining ways how participants can narrow/broaden their claims and strengthen them
Day-to-Day Reading List

Day 
Readings 
1

Compulsory

Creswell, John W. and Vicki L. Plano Clark (2011): Designing and Conducting Mixed Methods Research. Los Angeles: SAGE Publications: chap. 3.

Seawright, Jason (2016): Multi-Method Science: Combining Qualitative and Quantitative Tools. Cambridge: Cambridge University Press: chap. 3.

 

Voluntary

Lieberman, Evan S. (2005): Nested Analysis as a Mixed-Method Strategy for Comparative Research. American Political Science Review 99 (3): 435-452.

Rohlfing, Ingo (2008): What You See and What You Get: Pitfalls and Principles of Nested Analysis in Comparative Research. Comparative Political Studies 41 (11): 1492-1514.

Small, Mario Luis (2011): How to Conduct a Mixed Methods Study: Recent Trends in a Rapidly Growing Literature. Annual Review of Sociology 37 (1): 57-86.

Ahmed, Amel and Rudra Sil (2012): When Multi-Method Research Subverts Methodological Pluralism—or, Why We Still Need Single-Method Research. Perspectives on Politics 10 (4): 935-953.

2

Compulsory

Coppedge, Michael (1999): Thickening Thin Concepts and Theories - Combining Large N and Small in Comparative Politics. Comparative Politics 31 (4): 465-476.

Ahram, Ariel I. (2013): Concepts and Measurement in Multimethod Research. Political Research Quarterly 66 (2): 280-291.

Welch, Catherine, Maria Rumyantseva and Lisa Jane Hewerdine (2016): Using Case Research to Reconstruct Concepts: A Methodology and Illustration. Organizational Research Methods 19 (1): 111-130.

 

Voluntary

Sartori, Giovanni (1970): Concept Misformation in Comparative Politics. American Political Science Review 64 (4): 1033-1053.

Collier, David and James E. Mahon (1993): Conceptual Stretching Revisited - Adapting Categories in Comparative-Analysis. American Political Science Review 87 (4): 845-855.

3

Compulsory

Gerring, John and Lee Cojocaru (2016): Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods. Sociological Methods & Research 45 (3): 392-423.

Weller, Nicholas and Jeb Barnes (2016): Pathway Analysis and the Search for Causal Mechanisms. Sociological Methods & Research 45 (3): 424-457.

Sharp, Julia L., Catherine Mobley, Cathy Hammond, Cairen Withington, Sam Drew, Sam Stringfield and Natalie Stipanovic (2012): A Mixed Methods Sampling Methodology for a Multisite Case Study. Journal of Mixed Methods Research 6 (1): 34-54.

 

[change of readings possible conditional on the participants’ mixed-methods designs and case selection requirements]

 

For QCA research (optional; depends on participants’ projects)

Schneider, Carsten Q. and Ingo Rohlfing (2013): Combining QCA and Process Tracing in Set-Theoretic Multi-Method Research. Sociological Methods & Research 42 (4): 559-597

 

Voluntary

Seawright, Jason and John Gerring (2008): Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options. Political Research Quarterly 61 (2): 294-308.

Rohlfing, Ingo and Peter Starke (2013): Building on Solid Ground: Robust Case Selection in Multi-Method Research. Swiss Political Science Review 19 (4): 492-512.

4

Compulsory

Kühn, David and Ingo Rohlfing (2016): Generalization of Process Tracing Inferences in Multimethod Research. Typescript.

Spillman, Lyn (2014): Mixed Methods and the Logic of Qualitative Inference. Qualitative Sociology 37 (2): 189-205.

 

Voluntary

Onwuegbuzie, AnthonyJ and NancyL Leech (2010): Generalization Practices in Qualitative Research: A Mixed Methods Case Study. Quality & Quantity 44 (5): 881-892.

Lieberson, Stanley (1991): Small Ns and Big Conclusions: An Examination of the Reasoning in Comparative Studies Based on a Small Number of Cases. Social Forces 70 (2): 307-320.

5

Compulsory

Lange, Matthew (2009): Lineages of Despotism and Development: British Colonialism and State Power. Chicago: The University of Chicago Press: chap. 1, 2, 3, 4, 6.

(As a wrap-up, we will discuss one multi-method study in detail, which requires it to read the intro, theory, the quantitative analysis and one in-depth case study.)

Software Requirements

None

Hardware Requirements

None

Literature

A useful introduction to mixed-methods research in a broader sense:

Creswell, John W. and Vicki L. Plano Clark (2011): Designing and Conducting Mixed Methods Research. Los Angeles: SAGE Publications.

A good, focused discussion of multimethod research:

Seawright, Jason (2016): Multi-Method Social Science: Combining Qualitative and Quantitative Tools. Cambridge: Cambridge University Press.

The following other ECPR Methods School courses could be useful in combination with this one in a ‘training track .
Recommended Courses Before

Summer School

Multivariate regression
QCA
Case study research
Process tracing
Intro to mixed-methods

Winter School

Process tracing (basic)

 

Recommended Courses After

Winter School

QCA (depends on what Carsten Q. Schneider is teaching in this course, but it should be appropriate)
Different advanced statistics courses

 

Additional Information

Disclaimer

The information contained in this course description form may be subject to subsequent adaptations (e.g. taking into account new developments in the field, specific participant demands, group size etc.). Registered participants will be informed in due time in case of adaptations.

Note from the Academic Convenors

By registering to this course, you certify that you possess the prerequisite knowledge that is requested to be able to follow this course. The instructor will not teach these prerequisite items. If you are not sure if you possess this knowledge to a sufficient level, we suggest you contact the instructor before you proceed with your registration.


Share this page
 

"Nothing in politics is ever as good or bad as it first appears" - Edward Boyle


Back to top