ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

Back to Panel Details
Back to Panel Details

Advanced Multi-Method Research

Leonce Röth
leonce.roeth@uni-koeln.de

University of Cologne

Leonce is Senior Researcher at the Cologne Center for Comparative Politics, University of Cologne.

He researches and teaches party politics and methods with a focus on multimethod and quantitative analysis. 

Leonce has published in European Journal of Political Research, West European Politics, European Political Science Review or Swiss Political Science Review.

 


Course Dates and Times

Monday 25 February – Friday 1 March, 09:00–12:30
15 hours over 5 days

Prerequisite Knowledge

In-depth discussion of participants’ research projects is central to this course. You should be at an advanced stage of a piece of mixed-methods research, so you can present and discuss your study in class.

The course does not teach the basics of regression analysis, QCA, case studies, process tracing, or any other method one might use in multi-method research. You should know them already, because this course focuses on how to combine these methods.

Sufficient knowledge of the methods you aim to use is essential for understanding the principles of multi-method research.


Short Outline

This course deals with mixed-methods / multimethod research (MMR) in the social sciences. To accommodate participants’ projects, it aims to reflect the diversity of MMR studies in the discipline.

We will discuss the broader understanding of combining qualitative and quantitative methods and the more focused approach, linking case studies and process tracing to a large-n method such as Qualitative Comparative Analysis (QCA) or regression analysis. The relative emphasis we put on specific variants of MMR designs will be adapted to the methods participants are applying in their own research.

The goal of the course is to understand the different varieties in which MMR can be done. We discuss the unique advantages and methodological and practical challenges confronted in implementing multi-method designs.

Topics include concepts in the qualitative and quantitative analysis, case selection for qualitative research, and the compatibility of theoretical expectations and inferences on causal effects and causal mechanisms. Method-centred discussions are illustrated with examples from different fields of political science.

By the end of the course, you will able to realise your own mixed-methods study in a systematic manner, and to critically evaluate published MMR studies.

Tasks for ECTS Credits

2 credits (pass/fail grade) Attend 90% of course hours, participate fully in in-class activities, and carry out the necessary reading and/or other work prior to, and after, classes.

3 credits (to be graded) As above, plus submit a short daily assignment on days 1, 2, 3 and 4. The first two assignments are based on your own projects. If you are not working on an MMR project, an equivalent exercise will be assigned. The last two assignments are about general methodological issues and published empirical research.

4 credits (to be graded) As above, plus, after the course, submit a 10–15 page description of your mixed-methods research design as it stands. If a you are not working on an MMR project, an equivalent exercise will be assigned.

 


Long Course Outline

Mixed-method research is an enduring topic in the social sciences (e.g., Creswell and Piano 2011), but multi-method research (MMR) more narrowly defined, also known as nested analysis, is a relatively new topic.

After longstanding antagonistic discussions about the pros and cons of qualitative and quantitative methods, we now find a growing consensus that each method has its distinct advantages and that they work best in combination with each other.

This course builds on the debate about mixed-methods and multimethod research, and focuses on their unique advantages and challenges for empirical researchers seeking to combine two or more methods.

Day 1
Laying foundations for the rest of the course, we learn about different conceptions of causation and how they can be integrated in a single mixed-methods analysis. We introduce different varieties and dimensions of mixed-methods research such as the timing of the methods’ applications and their relative importance in the context of the broader design.

Day 2
We begin with a reflection on concepts and concept formation in MMR as the cornerstone of all empirical research. This session is based on two interrelated claims one finds in the methods literature. First, it is argued that concepts are thin in quantitative and thick in qualitative research. Second, it is claimed that this discrepancy creates problems of conceptual stretching, undermining causal analysis. We question whether these assertions are warranted and, to the extent that they are accurate, how concept formation can be improved.

Day 3
All about case selection. Depending on the mixed-methods design, case selection challenges can be diverse, ranging from the recruitment of participants for focus groups to the residual-based choice of countries after a macro-comparative regression. We will discuss the current state of debate on case selection in nested analysis (regression or QCA with follow-up process tracing) and reflect on additional case selection strategies pertinent to participants’ designs.

Day 4
We turn to the question of generalisation. Generalisation is considered in parts of the mixed-methods literature and ignored in others. The main focus lies on the generalisation of inferences generated in the qualitative analysis. According to a common line of reasoning, generalisation of qualitative conclusions is only possible under conditions that are difficult, if not impossible, to reconcile with the quantitative part. We will discuss generalisation strategies for different variants of mixed-method designs.

Day 5
Wrap-up session. We take a closer look at a published empirical study with regard to the topics of days 1–4, with specific focus on what can be called causal consistency or causal coherence. This means that one’s theoretical expectations regarding the large-n results and process tracing insights should fit with each other. Similarly, the conclusions that one derives from quantitative and qualitative analyses should be coherent. For example, a lack of fit occurs when process tracing leads to the conclusion that multiple factors work in conjunction, while the regression analysis models the effect of covariates as independent from each other. We discuss several sources and manifestations of inconsistency and strategies for achieving coherence over the course of the analysis.

Day Topic Details
Day 1 Introduction to mixed-methods research
  • Varieties of multi-method research (MMR)
  • Theories of causation and MMR
  • Course goals
Day 2 Concepts and concept formation
  • Thin and thick concepts
  • Risk of conceptual stretching
  • Conceptual and measurement consistency in MMR
Day 3 Case selection
  • Identifying types of cases in MMR
  • Strategies of case selection for different types of cases
Day 4 Generalization
  • Dimensions of generalization mixed-methods research
  • Assumptions and risks in generalizing
  • Strategies to strengthen generalization claims
Day 5 Wrap-up session and lessons learned
  • Applying insights to published mixed-methods study
  • Judging coherence of qualitative and quantitative analysis
Day Readings
Day 1

Compulsory
Seawright, Jason (2016): Multi-Method Science: Combining Qualitative and Quantitative Tools. Cambridge: Cambridge University Press: chap. 3.

Johnson, R. Burke, Federica Russo and Judith Schoonenboom (2017): Causation in mixed methods research: The meeting of philosophy, science, and practice. Journal of Mixed Methods Research advance access.

Voluntary
Lieberman, Evan S. (2005): Nested Analysis as a Mixed-Method Strategy for Comparative Research. American Political Science Review 99 (3): 435-452.

Creswell, John W. and Vicki L. Plano Clark (2011): Designing and Conducting Mixed Methods Research. Los Angeles: SAGE Publications: chap. 3.

Small, Mario Luis (2011): How to Conduct a Mixed Methods Study: Recent Trends in a Rapidly Growing Literature. Annual Review of Sociology 37 (1): 57-86.

Ahmed, Amel and Rudra Sil (2012): When Multi-Method Research Subverts Methodological Pluralism—or, Why We Still Need Single-Method Research. Perspectives on Politics 10 (4): 935-953.

Beach, Derek and Ingo Rohlfing (2018): Integrating cross-case analyses and process tracing in set-theoretic research: Strategies and parameters of debate. Sociological Methods & Research 47 (1): 3-36.

Day 2

Compulsory

Coppedge, Michael (1999): Thickening Thin Concepts and Theories - Combining Large N and Small in Comparative Politics. Comparative Politics 31 (4): 465-476.

Ahram, Ariel I. (2013): Concepts and Measurement in Multimethod Research. Political Research Quarterly 66 (2): 280-291.

Voluntary

Sartori, Giovanni (1970): Concept Misformation in Comparative Politics. American Political Science Review 64 (4): 1033-1053.

Collier, David and James E. Mahon (1993): Conceptual Stretching Revisited - Adapting Categories in Comparative-Analysis. American Political Science Review 87 (4): 845-855.

Welch, Catherine, Maria Rumyantseva and Lisa Jane Hewerdine (2016): Using Case Research to Reconstruct Concepts: A Methodology and Illustration.

Day 3

Compulsory

Lieberman, Evan S. (2005): Nested analysis as a mixed-method strategy for comparative research. American Political Science Review 99 (3): 435-452.

Lange, Matthew (2009): Lineages of Despotism and Development: British Colonialism and State Power. Chicago: The University of Chicago Press: chap. 1, 3.

Weller, Nicholas and Jeb Barnes (2016): Pathway Analysis and the Search for Causal Mechanisms. Sociological Methods & Research 45 (3): 424-457.

Voluntary

Seawright, Jason and John Gerring (2008): Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options. Political Research Quarterly 61 (2): 294-308.

 Sharp, Julia L., Catherine Mobley, Cathy Hammond, Cairen Withington, Sam Drew, Sam Stringfield and Natalie Stipanovic (2012): A Mixed Methods Sampling Methodology for a Multisite Case Study. Journal of Mixed Methods Research 6 (1): 34-54.

Rohlfing, Ingo and Peter Starke (2013): Building on Solid Ground: Robust Case Selection in Multi-Method Research. Swiss Political Science Review 19 (4): 492-512.

Schneider, Carsten Q. and Ingo Rohlfing (2013): Combining QCA and Process Tracing in Set-Theoretic Multi-Method Research. Sociological Methods & Research 42 (4): 559-597

Gerring, John and Lee Cojocaru (2016): Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods. Sociological Methods & Research 45 (3): 392-423.

Day 4

Compulsory

Kühn, David and Ingo Rohlfing (2016): Generalization of Process Tracing Inferences in Multimethod Research. Typescript.

Goertz, Gary (2017): Multimethod Research, Causal Mechanisms, and Case Studies. Princeton: Princeton University Press: chap. 8.

Voluntary

Lieberson, Stanley (1991): Small Ns and Big Conclusions: An Examination of the Reasoning in Comparative Studies Based on a Small Number of Cases. Social Forces 70 (2): 307-320.

Onwuegbuzie, AnthonyJ and NancyL Leech (2010): Generalization Practices in Qualitative Research: A Mixed Methods Case Study. Quality & Quantity 44 (5): 881-892.

Spillman, Lyn (2014): Mixed Methods and the Logic of Qualitative Inference. Qualitative Sociology 37 (2): 189-205.

Day 5

Compulsory
Lange, Matthew (2009): Lineages of Despotism and Development: British Colonialism and State Power. Chicago: The University of Chicago Press: chap. 2, 4, 5, 6, 7.

As a wrap-up, we will discuss one multi-method study in detail: the intro, theory, quantitative analysis and one in-depth case study.

We will discuss ahead of the Winter School who will read which of the case study chapters.

Software Requirements

None

Hardware Requirements

None

Literature

A useful introduction to mixed-methods research in a broader sense:

Creswell, John W. and Vicki L. Plano Clark (2011)
Designing and Conducting Mixed Methods Research
Los Angeles: SAGE Publications: chap. 2, 3

A good, focused discussion of multimethod research:

Seawright, Jason (2016)
Multi-Method Social Science: Combining Qualitative and Quantitative Tools
Cambridge: Cambridge University Press

Recommended Courses to Cover Before this One

<p><strong>Summer School</strong><br /> Multivariate regression<br /> QCA<br /> Case study research<br /> Process tracing<br /> Intro to mixed-methods</p> <p><strong>Winter School</strong><br /> Process tracing (basic)</p> <p>&nbsp;</p>

Recommended Courses to Cover After this One

<p><strong>Winter School</strong><br /> QCA (depends on what Carsten Q. Schneider is teaching in this course, but it should be appropriate)<br /> Different advanced statistics courses</p> <p>&nbsp;</p>


Additional Information

Disclaimer

This course description may be subject to subsequent adaptations (e.g. taking into account new developments in the field, participant demands, group size, etc). Registered participants will be informed in due time.

Note from the Academic Conveners

By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach these prerequisite items. If in doubt, contact the instructor before registering.