ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

Your subscription could not be saved. Please try again.
Your subscription to the ECPR Methods School offers and updates newsletter has been successful.

Discover ECPR's Latest Methods Course Offerings

We use Brevo as our email marketing platform. By clicking below to submit this form, you acknowledge that the information you provided will be transferred to Brevo for processing in accordance with their terms of use.

Advanced Multi-Method Research

Course Dates and Times

Monday 17 – Friday 21 February 2019, 09:00–12:30
15 hours over five days

Ingo Rohlfing

ingo.rohlfing@uni-passau.de

Universität Passau

This course deals with mixed-methods / multimethod research (MMR) in the social sciences. To accommodate participants’ projects, it aims to reflect the diversity of MMR studies in the discipline.

We will discuss the broader understanding of combining qualitative and quantitative methods and the more focused approach, linking case studies and process tracing to a large-n method such as Qualitative Comparative Analysis (QCA) or regression analysis. The relative emphasis we put on specific variants of MMR designs will be adapted to the methods participants are applying in their own research.

The goal of the course is to understand the different varieties in which MMR can be done. We discuss the unique advantages and methodological and practical challenges confronted in implementing multi-method designs.

Topics include concepts in the qualitative and quantitative analysis, case selection for qualitative research, and the compatibility of theoretical expectations and inferences on causal effects and causal mechanisms. Method-centred discussions are illustrated with examples from different fields of political science.

By the end of the course, you will able to realise your own mixed-methods study in a systematic manner, and to critically evaluate published MMR studies.

Tasks for ECTS Credits

2 credits (pass/fail grade) Attend 90% of course hours, participate fully in in-class activities, and carry out the necessary reading and/or other work prior to, and after, classes.

3 credits (to be graded) As above, plus submit a short daily assignment on days 1, 2, 3 and 4. The first two assignments are based on your own projects. If you are not working on an MMR project, an equivalent exercise will be assigned. The last two assignments are about general methodological issues and published empirical research.

4 credits (to be graded) As above, plus, after the course, submit a 10–15 page description of your mixed-methods research design as it stands. If a you are not working on an MMR project, an equivalent exercise will be assigned.

 


Instructor Bio

Ingo Rohlfing is Professor of Methods of Empirical Social Research at the University of Passau

He researches social science methods with a focus on qualitative methods (case studies and process tracing), Qualitative Comparative Analysis and multimethod research.

Ingo is author of Case Studies and Causal Inference (Palgrave Macmillan) and he has published articles in Comparative Political Studies, Sociological Methods & Research and Political Analysis.

 

Mixed-method research is an enduring topic in the social sciences (e.g., Creswell and Piano 2011), but multi-method research (MMR) more narrowly defined, also known as nested analysis, is a relatively new topic.

After longstanding antagonistic discussions about the pros and cons of qualitative and quantitative methods, we now find a growing consensus that each method has its distinct advantages and that they work best in combination with each other.

This course builds on the debate about mixed-methods and multimethod research, and focuses on their unique advantages and challenges for empirical researchers seeking to combine two or more methods.

Day 1
Laying foundations for the rest of the course, we learn about different conceptions of causation and how they can be integrated in a single mixed-methods analysis. We introduce different varieties and dimensions of mixed-methods research such as the timing of the methods’ applications and their relative importance in the context of the broader design.

Day 2
We begin with a reflection on concepts and concept formation in MMR as the cornerstone of all empirical research. This session is based on two interrelated claims one finds in the methods literature. First, it is argued that concepts are thin in quantitative and thick in qualitative research. Second, it is claimed that this discrepancy creates problems of conceptual stretching, undermining causal analysis. We question whether these assertions are warranted and, to the extent that they are accurate, how concept formation can be improved.

Day 3
All about case selection. Depending on the mixed-methods design, case selection challenges can be diverse, ranging from the recruitment of participants for focus groups to the residual-based choice of countries after a macro-comparative regression. We will discuss the current state of debate on case selection in nested analysis (regression or QCA with follow-up process tracing) and reflect on additional case selection strategies pertinent to participants’ designs.

Day 4
We turn to the question of generalisation. Generalisation is considered in parts of the mixed-methods literature and ignored in others. The main focus lies on the generalisation of inferences generated in the qualitative analysis. According to a common line of reasoning, generalisation of qualitative conclusions is only possible under conditions that are difficult, if not impossible, to reconcile with the quantitative part. We will discuss generalisation strategies for different variants of mixed-method designs.

Day 5
Wrap-up session. We take a closer look at a published empirical study with regard to the topics of days 1–4, with specific focus on what can be called causal consistency or causal coherence. This means that one’s theoretical expectations regarding the large-n results and process tracing insights should fit with each other. Similarly, the conclusions that one derives from quantitative and qualitative analyses should be coherent. For example, a lack of fit occurs when process tracing leads to the conclusion that multiple factors work in conjunction, while the regression analysis models the effect of covariates as independent from each other. We discuss several sources and manifestations of inconsistency and strategies for achieving coherence over the course of the analysis.

THIS IS THE WINTER SCHOOL 2019 OUTLINE

In-depth discussion of participants’ research projects is central to this course. You should be at an advanced stage of a piece of mixed-methods research, so you can present and discuss your study in class.

The course does not teach the basics of regression analysis, QCA, case studies, process tracing, or any other method one might use in multi-method research. You should know them already, because this course focuses on how to combine these methods.

Sufficient knowledge of the methods you aim to use is essential for understanding the principles of multi-method research.

Day Topic Details
Day 1 Introduction to mixed-methods research
  • Varieties of multi-method research (MMR)
  • Theories of causation and MMR
  • Course goals
Day 2 Concepts and concept formation
  • Thin and thick concepts
  • Risk of conceptual stretching
  • Conceptual and measurement consistency in MMR
Day 3 Case selection
  • Identifying types of cases in MMR
  • Strategies of case selection for different types of cases
Day 4 Generalization
  • Dimensions of generalization mixed-methods research
  • Assumptions and risks in generalizing
  • Strategies to strengthen generalization claims
Day 5 Wrap-up session and lessons learned
  • Applying insights to published mixed-methods study
  • Judging coherence of qualitative and quantitative analysis
Day Readings
Day 1

COMPULSORY

Seawright, Jason (2016)
Multi-Method Science: Combining Qualitative and Quantitative Tools, Chapter 3
Cambridge: Cambridge University Press

Johnson, R. Burke, Federica Russo and Judith Schoonenboom (2017)
Causation in mixed methods research: The meeting of philosophy, science, and practice
Journal of Mixed Methods Research advance access

VOLUNTARY

Lieberman, Evan S. (2005)
Nested Analysis as a Mixed-Method Strategy for Comparative Research
American Political Science Review 99 (3): 435–452

Creswell, John W. and Vicki L. Plano Clark (2011)
Designing and Conducting Mixed Methods Research, Chapter 3
Los Angeles: SAGE Publications

Small, Mario Luis (2011)
How to Conduct a Mixed Methods Study: Recent Trends in a Rapidly Growing Literature
Annual Review of Sociology 37 (1): 57–86

Ahmed, Amel and Rudra Sil (2012)
When Multi-Method Research Subverts Methodological Pluralism — or, Why We Still Need Single-Method Research
Perspectives on Politics 10 (4): 935–953

Beach, Derek and Ingo Rohlfing (2018)
Integrating cross-case analyses and process tracing in set-theoretic research: Strategies and parameters of debate
Sociological Methods & Research 47 (1): 3–36

Day 2

COMPULSORY

Coppedge, Michael (1999)
Thickening Thin Concepts and Theories – Combining Large N and Small in Comparative Politics
Comparative Politics 31 (4): 465–476

Ahram, Ariel I. (2013)
Concepts and Measurement in Multimethod Research
Political Research Quarterly 66 (2): 280–291

VOLUNTARY

Sartori, Giovanni (1970)
Concept Misformation in Comparative Politics
American Political Science Review 64 (4): 1033–1053

Collier, David and James E. Mahon (1993)
Conceptual Stretching Revisited – Adapting Categories in Comparative-Analysis
American Political Science Review 87 (4): 845–855

Welch, Catherine, Maria Rumyantseva and Lisa Jane Hewerdine (2016)
Using Case Research to Reconstruct Concepts: A Methodology and Illustration
Organizational Research Methods, Vol. 19, Issue 1

Day 3

COMPULSORY

Lieberman, Evan S. (2005)
Nested analysis as a mixed-method strategy for comparative research
American Political Science Review 99 (3): 435–452

Lange, Matthew (2009)
Lineages of Despotism and Development: British Colonialism and State Power, Chapters 1 and 3
Chicago: The University of Chicago Press

Weller, Nicholas and Jeb Barnes (2016)
Pathway Analysis and the Search for Causal Mechanisms
Sociological Methods & Research 45 (3): 424–457

VOLUNTARY

Seawright, Jason and John Gerring (2008)
Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options
Political Research Quarterly 61 (2): 294–308

Sharp, Julia L., Catherine Mobley, Cathy Hammond, Cairen Withington, Sam Drew, Sam Stringfield and Natalie Stipanovic (2012)
A Mixed Methods Sampling Methodology for a Multisite Case Study
Journal of Mixed Methods Research 6 (1): 34–54

Rohlfing, Ingo and Peter Starke (2013)
Building on Solid Ground: Robust Case Selection in Multi-Method Research
Swiss Political Science Review 19 (4): 492–512

Schneider, Carsten Q. and Ingo Rohlfing (2013)
Combining QCA and Process Tracing in Set-Theoretic Multi-Method Research
Sociological Methods & Research 42 (4): 559–597

Gerring, John and Lee Cojocaru (2016)
Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods
Sociological Methods & Research 45 (3): 392–423

Day 4

COMPULSORY

Kühn, David and Ingo Rohlfing (2016)
Generalization of Process Tracing Inferences in Multimethod Research
Typescript

Goertz, Gary (2017)
Multimethod Research, Causal Mechanisms, and Case Studies, Chapter 8
Princeton: Princeton University Press

VOLUNTARY

Lieberson, Stanley (1991)
Small Ns and Big Conclusions: An Examination of the Reasoning in Comparative Studies Based on a Small Number of Cases
Social Forces 70 (2): 307–320

Onwuegbuzie, AnthonyJ and NancyL Leech (2010)
Generalization Practices in Qualitative Research: A Mixed Methods Case Study
Quality & Quantity 44 (5): 881–892

Spillman, Lyn (2014)
Mixed Methods and the Logic of Qualitative Inference
Qualitative Sociology 37 (2): 189–205

Day 5

COMPULSORY

Lange, Matthew (2009)
Lineages of Despotism and Development: British Colonialism and State Power, Chapters 2, 4, 5, 6 and 7
Chicago: The University of Chicago Press

As a wrap-up, we will discuss one multi-method study in detail: the intro, theory, quantitative analysis and one in-depth case study.

We will discuss ahead of the Winter School who will read which of the case study chapters.

Literature

A useful introduction to mixed-methods research in a broader sense:

Creswell, John W. and Vicki L. Plano Clark (2011)
Designing and Conducting Mixed Methods Research
Los Angeles: SAGE Publications: chap. 2, 3

A good, focused discussion of multimethod research:

Seawright, Jason (2016)
Multi-Method Social Science: Combining Qualitative and Quantitative Tools
Cambridge: Cambridge University Press

Recommended Courses to Cover Before this One

Summer School
Multivariate regression
QCA
Case study research
Process tracing
Intro to mixed-methods

Winter School
Process tracing (basic)

 

Recommended Courses to Cover After this One

Winter School
QCA
Different advanced statistics courses