ECPR Winter School
University of Bamberg, Bamberg
3 - 10 March 2017




WD204 - Advanced Multi-Method Research

Instructor Details

Instructor Photo

Ingo Rohlfing

Institution:
University of Cologne

Instructor Bio

Ingo Rohlfing is Professor of Political Science, Qualitative Methods, at the Bremen International Graduate School of Social Sciences (BIGSSS). He holds a PhD in Political Science.

Substantively, he is doing research on party competition and party organisations. Methodologically, he is working on the case study method, process tracing, QCA and multi-method research.

He has published in journals such as Comparative Political Studies, Sociological Methods & Research and West European Politics.

Ingo is the author of Case Studies and Causal Inference, part of the ECPR Research Methods Series, published in association with Palgrave Macmillan.

  @ingorohlfing


Course Dates and Times

Monday 6 to Friday 10 March 2017
Generally classes are either 09:00-12:30 or 14:00-17:30
15 hours over 5 days

Prerequisite Knowledge

The course does not discuss basics of regression analysis, QCA, case studies, process tracing, or any other method one might use in multi-method research. Participants are expected to have acquired skills on these methods when taking this course because it specifically focuses on how to combine them. Having sufficient knowledge of the methods you aim to use is important because it is important for understanding some of the principles of multi-method research.

Short Outline

This course deals with multi-method research (MMR) as it is currently developed in political science and sociology (e.g., Lieberman’s nested analysis). The course builds on this development and focuses on the combination of case studies and process tracing with a large-n method and Qualitative Comparative Analysis (QCA) and regression analysis in particular, as these are the most widely used large-n techniques in MMR. The relative emphasis we put on statistical methods and QCA depends on what methods the participants are applying in their own research. Participants combining case studies with another method such as social network analysis or experiments are also welcome. The goal of the course is to understand the different varieties in which MMR can be done. We discuss the unique advantages and methodological and practical challenges confronted in implementing multi-method designs. Topics include concepts in the small-n and the large-n analysis, case selection for process tracing, and the compatibility of theoretical expectations and inferences on causal effects and causal mechanisms. Method-centered discussions are illustrated with examples from different fields of political science and, if possible, the projects the participants are working on. At the end of the course, participants are able to realize their own MMR in a systematic manner and to critically evaluate published MMR studies.

Long Course Outline

Mixed-method research is an enduring topic in the social sciences (e.g., Creswell and Piano 2011), but multi-method research (MMR) more narrowly is a relatively new topic in the “US methods debate”. After long standing antagonistic discussions about the pros and cons of small-n and large-n methods, we now find a growing consensus that each method has its distinct advantages and that they work best in combination with each other. This course builds on the debate about MMR and focuses on its unique advantages and challenges for empirical researchers seeking to combine two (or more) methods.

On day 1, we lay the foundation and discuss different varieties of MMR. As regards the large-n method, the focus rests on regression analysis and Qualitative Comparative Analysis (QCA) as, arguably, the most widely applied cross-case methods in MMR (emphasis on both cross-case methods depends on the method the course participants are implementing). First, the process of making an informed choice between regression analysis and QCA is explained. In addition, we discuss the conditions under which it is better to begin with case studies and utilize the large-n method afterwards, and when it is better to apply the large-n technique first and process tracing second. Furthermore, we have a brief discussion of fundamental terms such as causal effects and causal mechanisms and levels of analysis.

On day 2, we begin with a reflection on concepts and concept formation in MMR as the cornerstone of all empirical research. The session is based on two interrelated claims one finds in the literature. First, it is argued that concepts are thin in large-n and thick in small-n research. Second, it is claimed that this discrepancy creates problems of conceptual stretching undermining causal inference in MMR. We elaborate on whether these assertions are warranted and, to the extent that they are accurate, how concept formation can be improved in MMR.

The topic of day 3 is case selection on the basis of results derived from the large-n analysis. First, it is shown that case selection strategies differ depending on whether one is running a regression analysis or QCA. Building on this insight, we expand on the designation and choice of different types of cases – e.g., typical and deviant cases – on the basis of large-n results. Following Fearon and Laitin’s (2008) plea for random case selection for process tracing, we further consider the pros and cons of random case selection versus different types of intentional case selection.

Because of the importance of case selection and the variety of arguments in the literature, the beginning of day 4 finishes the discussion of this topic. Afterwards, we turn to the question of generalization. Generalization is rarely considered for the large-n method – regression analysis and QCA alike – but we also spend some time considering the generalization of large-n results. The main focus lies on the generalization of the inferences generated via process tracing. According to a common line of reasoning, generalization of case study inferences is only possible under conditions that are difficult, if not impossible, to reconcile with MMR. Running counter to this, we discuss a statistical procedure for generalizing process tracing inferences and detail the conditions under which it is applicable.

On the last day, day 5, we have a wrap-up session. We take a closer look at published empirical studies with regard to the topics of day 1 to 4 and with a specific focus on what can be called causal consistency or causal coherence. This means that one’s theoretical expectations as regards the large-n results and process tracing insights should fit with each other. Similarly, the inferences that one derives from large-n and small-n analyses should be coherent. For example, a lack of fit occurs when process tracing leads to the conclusion that multiple factors work in conjunction, while the regression analysis models the effect of covariates as independent from each other. We discuss several sources and manifestations of inconsistency, strategies for achieving coherence in the specification of observable implications related to causal effects and causal mechanisms, and raise the awareness for making consistent causal inferences on effects and mechanisms.

On day 1, 2, 3, and 5, participants receive small assignments due the next day that will be discussed at the beginning of each section of the following day. The assignments deal with MMR studies published in journals or books.

Participants at a more advanced stage of their MMR are invited to bring their large-n data with them in order to discuss specific issues and to immediately attempt to implement the lessons learned in the course.

Day-to-Day Schedule

Day 
Topic 
Details 
1General introduction to the topic- Overview of course schedule - Course goals - Varieties of multi-method research (MMR) - Theories of causation and MMR
2Concepts and concept formation- Thin and thick concepts - Risks of conceptual stretching - Conceptual consistency in MMR
3Case selection- Identifying types of cases (e.g. typical case) on the basis of large-n method - Causal homogeneity and case selection - Intentional vs. random case selection
4Case selection & generalisation- Finishing discussion of case selection - Generalization of large-n inferences - Problems of generalizing small-n inferences - A Procedure for small-n generalization
5Wrap-up session and lessons learned- Applying insights to published MMR studies - Coherence of observable implications for large-n and small-n analysis - Consistency of large-n and small-n inferences in MMR
Day-to-Day Reading List

Day 
Readings 
1

Compulsory

Lieberman, Evan S. (2005): Nested Analysis as a Mixed-Method Strategy for Comparative Research. American Political Science Review 99 (3): 435-452.
Seawright, Jason (2016): Multi-Method Science: Combining Qualitative and Quantitative Tools. Cambridge: Cambridge University Press: chap. 3.

 

Voluntary

Creswell, John W. and Vicki L. Plano Clark (2011): Designing and Conducting Mixed Methods Research. Los Angeles: SAGE Publications: chap. 3.

Rohlfing, Ingo (2008): What You See and What You Get: Pitfalls and Principles of Nested Analysis in Comparative Research. Comparative Political Studies 41 (11): 1492-1514.

 

2

Compulsory

Coppedge, Michael (1999): Thickening Thin Concepts and Theories - Combining Large N and Small in Comparative Politics. Comparative Politics 31 (4): 465-476.
Ahram, Ariel I. (2013): Concepts and Measurement in Multimethod Research. Political Research Quarterly 66 (2): 280-291.

 

Voluntary

Sartori, Giovanni (1970): Concept Misformation in Comparative Politics. American Political Science Review 64 (4): 1033-1053.
Collier, David and James E. Mahon (1993): Conceptual Stretching Revisited - Adapting Categories in Comparative-Analysis. American Political Science Review 87 (4): 845-855.

3

Compulsory

Gerring, John and Lee Cojocaru (2016): Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods. Sociological Methods & Research 45 (3): 392-423.

Weller, Nicholas and Jeb Barnes (2016): Pathway Analysis and the Search for Causal Mechanisms. Sociological Methods & Research 45 (3): 424-457.

For QCA research (optional; depends on participants’ projects)
Schneider, Carsten Q. and Ingo Rohlfing (2013): Combining QCA and Process Tracing in Set-Theoretic Multi-Method Research. Sociological Methods & Research 42 (4): 559-597

 

Voluntary

Seawright, Jason and John Gerring (2008): Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options. Political Research Quarterly 61 (2): 294-308.

Rohlfing, Ingo and Peter Starke (2013): Building on Solid Ground: Robust Case Selection in Multi-Method Research. Swiss Political Science Review 19 (4): 492-512.

4

Compulsory

Kühn, David and Ingo Rohlfing (2016): Generalization of Process Tracing Inferences in Multimethod Research. Typescript.

 

Voluntary

Ruzzene, Attilia (2012): Drawing Lessons from Case Studies by Enhancing Comparability. Philosophy of the Social Sciences 42 (1): 99-120.

Lieberson, Stanley (1991): Small Ns and Big Conclusions: An Examination of the Reasoning in Comparative Studies Based on a Small Number of Cases. Social Forces 70 (2): 307-320.

5

Compulsory

Lange, Matthew (2009): Lineages of Despotism and Development: British Colonialism and State Power. Chicago: The University of Chicago Press: chap. 1, 2, 3, 4, 6.

(As a wrap-up, we will discuss one multi-method study in detail, which requires it to read the intro, theory, the quantitative analysis and one in-depth case study.)

Software Requirements

None

Hardware Requirements

None

Literature

A useful introduction to mixed-methods research in a broader sense is:

Creswell, John W. and Vicki L. Plano Clark (2011): Designing and Conducting Mixed Methods Research. Los Angeles: SAGE Publications.

The following other ECPR Methods School courses could be useful in combination with this one in a ‘training track .
Recommended Courses Before

Summer School

  • Multivariate regression QCA Case study research
  • Process tracing

Winter School

  • Process tracing
Recommended Courses After

Winter School

  • QCA
  • different advanced statistics courses

Additional Information

Disclaimer

The information contained in this course description form may be subject to subsequent adaptations (e.g. taking into account new developments in the field, specific participant demands, group size etc.). Registered participants will be informed in due time in case of adaptations.

Note from the Academic Convenors

By registering to this course, you certify that you possess the prerequisite knowledge that is requested to be able to follow this course. The instructor will not teach these prerequisite items. If you are not sure if you possess this knowledge to a sufficient level, we suggest you contact the instructor before you proceed with your registration.


Share this page
 

"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." - Benjamin Franklin


Back to top