Linear Regression with R/Stata: Estimation, Interpretation and Presentation

It is frequently quipped that regression is the most used and abused method in the social sciences. My goal in this class is to expose participants to the abuse-free application of linear regression models to social science data. By the end of the sessions participants will have all the required theoretical and practical skills to responsibly run multivariate linear regressions to a variety of data configurations. This includes estimating multiple model specifications in R or Stata, presenting results in tables, in a graphical format or as informative quantities, and interpreting the coefficients for the reader. Furthermore, it also implies assessing the appropriateness of OLS regression for certain kinds of data distributions, and learning to make suitable corrections and adjustments when there is a mismatch between model requirements and data characteristics.

The course should appeal most to participants who have had statistical training at an introductory level as part of their undergraduate studies, and now wish to deepen it with a rigorous coverage of linear models. Although we will not be computing quantities by hand, there will be a few simple formulas as part of the lectures. In this sense, the class is also suitable for those of you who have briefly encountered OLS regression as part of a statistics class, but now wish to better understand how it works, where it breaks down, and how it can be applied in a thorough way. Due to the need to constantly focus on the application of linear models, the class is unsuitable for those who want an introductory course in general statistics. We briefly cover during one of the sessions some basic statistical concepts and tests, but this is only so that we can all delve into the topic of linear models from an equal footing. This cannot be considered a substitute for a good coverage of introductory statistics.

We start off the class with a condensed review of some fundamental concepts in basic statistics: the z and t distributions, hypothesis testing, confidence intervals and correlation. This short overview intends to provide a solid foundation from which to advance in the following days. In the lab session, we will go through a few of the basic data manipulation procedures that are commonly required before running any regression: data cleaning and recoding, transformations of data etc. This is also a good opportunity for participants to get familiar, if need be, with working with syntax files in R and Stata.

The second session delves fully into the fundamentals of Ordinary Least Squares (OLS) regression: how the estimation is carried out, how we interpret the coefficients and their associated uncertainty, and the best way in which to present results from these analyses. Some basic formulas will be presented, but the goal will be to gain an intuitive understanding of how the estimation process functions, and what the results mean. In the lab session we put this newly-gained knowledge to the test, by running a few examples of linear models in R or Stata. We will interpret the output and the model fit, generate predictions based on the model, and displaying them in a graphical way. For this, we will rely on the functionality provided by the Zelig package in R, and the Clarify add-on in Stata.

In the third session we advance in our coverage of OLS by tackling some model specifications that almost always appear in empirical research. Here I refer to dummy indicators and exploring effect heterogeneity through the use of interactions. We learn how to interpret results obtained from interaction terms, and how to present them in a graphical way. In the lab component we learn how to run these model specifications with the use of R/Stata. The fourth session is devoted to preventing abuse in the estimation of linear models. As with the vast majority of statistical procedures, a series of assumptions underpin OLS regression. If these are not met we have little reason to put our faith in the results we obtain. In this session we go over these assumptions, how they influence the results when they are not met, and what strategies we have available in the toolbox to overcome this situation. In the lab we turn to these issues from a practical perspective. We use R and Stata to run a test regression, assess whether the assumptions are met, and correct for the assumption violations that exist. Through a step-by-step process, participants have the opportunity to see how their estimates and model fit changes when engaging in such a process.

Finally, we devote the last session to extensions of the framework: non-linear and generalized linear models. While OLS is ubiquitous in the social sciences, a considerable number of phenomena are not linear in how their effect is manifested. Getting a taste of non-linear models opens up a new world of hypotheses and results for researchers. In the second half, we discuss generalized linear models (GLMs), which can be used to analyze non-continuous dependent variables: dichotomous, ordinal, or multinomial. Using the test case of a dichotomous outcome variable, we explore the way in which GLMs are a simple extension of the linear framework. In the lab we practice these models and focus on how to interpret their results.

Throughout the class, we will focus on graphical methods and intuitive quantities when presenting results from linear models. Graphs rather than tables, predicted values and uncertainty around them rather than coefficients and standard errors. While tables of coefficients are still the dominant way of presenting results in academic journals, graphs and predicted values tend to be preferred in reports and analyses done for larger, non-technical audiences. I strongly believe that participants should be exposed to both types, and should tailor the delivery of their results to the audience at a particular time.

In order to guarantee that we can progress at a steady and firm pace, participants will need to have a thorough understanding of basic statistical concepts such as mean, median, variance, standard deviation and standard error. Additionally, very basic statistical tests and analyses, such as t tests and ANOVA should be familiar to participants at least at a theoretical level. The class will be carried out in both R and Stata, so participants are expected to have basic knowledge of either one of these two software packages: reading in data, basic data recoding skills, and very basic plotting commands.