Causal Inference in the Social Sciences II: Difference in Difference, Regression Discontinuity and Instruments

‘I used to think correlation implied causation. Then I took a statistics class. Now I don’t.’

‘Sounds like the class helped!’

‘Well, maybe.’

This course is a thorough introduction to the most widely used methods of causal inference. It is an applied methods course, with equal emphasis on methods and their applications.

Do hospitals make people healthier?

Is it a problem that more people die in hospitals than in bars?

Does an additional year of schooling increase future earnings?

Do parties that enter the parliament enjoy vote gains in subsequent elections?

The answers to these questions (and many others which affect our daily life) involve the identification and measurement of causal links: an old problem in philosophy and statistics. To address this problem we either use experiments or try to mimic them by collecting information on potential factors that may affect both treatment assignment and potential outcomes.

Customary ways of doing this in the past entailed the specification of sophisticated versions of multivariate regressions. However, it is by now well understood that causality can only be dealt with during the design, not during the estimation process.

The goal of this course is to familiarise you with the logic of casual inference, the underlying theory behind it, and to introduce research methods that help you approach experimental benchmarks with observational data. This will be a much-applied course, giving you ideas for strong research designs in your own work, and the knowledge to derive and interpret causal estimates based on these designs.

We start by discussing the fundamental problem of causal inference. After that, I introduce the potential outcomes framework (covered thoroughly in SB112A). I then illustrate how the selection problem creates bias in naïve estimators of such quantities using observational data and we will see how randomisation solves the problem of selection bias. The next steps will be dedicated into the methods through which we can approach the experimental benchmark using observational data.

The first method we will examine is matching. Although matching is not itself a design of causal inference but a family of techniques to ensure balance on a series of observables (and is thus based on the conditional-on-observables, aka. uncounfoundedness assumption), it is very useful as a first application to the potential outcomes language. We will discuss the logic behind matching, its identification assumptions and see how it differs from standard regression methods.

After matching we will switch to the three designs. We start with instrumental variables (IVs), motivating the discussion with causal diagrams. We then employ a running example to help us first unpack the identification assumptions upon which IVs can deliver unbiased causal estimates. We then focus on estimation issues and applications. We look at the Wald estimator and its covariate extension, i.e. the 2SLS estimator.

Next we examine the regression discontinuity design (RD), motivating the discussion with examples from various subfields in sociology, political science and economics to help you grasp the intuition behind the design. Then we move to the clarification of the assumptions upon which identification is based: under what assumptions does the RD generate unbiased causal estimators? Moreover, which causal quantity of interest is estimated? I then spend time explaining how exactly these effects can be estimated. I will cover parametric and non-parametric estimation. We will discuss inference, using also robust confidence intervals for the point estimates. We go through the procedure through which the bandwidth for the RD analysis is chosen. An important next step in this design is to discuss the plethora of robustness checks one needs to do when using the RD. Before moving to the lab applications, we will also look how the fuzzy RD operates. We will see the extra assumption needed for this design and look at examples to gauge the key intuition. Estimation with a fuzzy RD will be also discussed.

The last estimator we focus on is the difference-in-differences estimator (DiD). After explaining the logic of the method, we see the key assumption needed to identify causal effects through the DD estimator. We then look at how you can estimate these effects using a variety of designs, with two groups and with multiple groups. We then go back and discuss the parallel trends assumption in more detail, showing under what conditions one can examine whether it holds or not. We also look at an extension of this design, namely the difference-in-differences-in-differences estimator. Numerous hands-on applications will be covered and one of them used as the main example for our applied session in the lab.

Lab sessions will draw on the discussion related to each of the three designs. All exercises will be covered in STATA in class, but I will also provide codes for R. It is worth emphasising that this is not a software-intense course. I will not spend much time teaching you how STATA works in general (and especially not R). Full code will be provided so we can focus on the analysis. That said, if you already use either of the two programs, after this course you will be in position to implement your analyses using any of the techniques covered in the course.

Solid knowledge of statistics and regression analysis at undergraduate level.

Some knowledge of Stata (or R)

Familiarity with the OLS regression estimator.

We will be working mainly in Stata. Full code will be provided.

No knowledge of any specific software is required, but some knowledge of Stata is useful.