Linear Regression with R/Stata: Estimation, Interpretation and Presentation

It is frequently quipped that regression is the most used and abused method in the social sciences. My goal in this class is to expose participants to the abuse-free application of linear regression models to social science data. By the end of the sessions participants will have all the required theoretical and practical skills to responsibly run multivariate linear regressions to a variety of data configurations. This includes estimating multiple model specifications in R or Stata, presenting results in tables, in a graphical format or as informative quantities, and interpreting the coefficients for the reader. Furthermore, it also implies assessing the appropriateness of OLS regression for certain kinds of data distributions, and learning to make suitable corrections and adjustments when there is a mismatch between model requirements and data characteristics.

The course should appeal most to participants who have had statistical training at an introductory level as part of their undergraduate studies, and now wish to deepen it with a rigorous coverage of linear models. Although we will not be computing quantities by hand, there will be a few simple formulas as part of the lectures. In this sense, the class is also suitable for those of you who have briefly encountered OLS regression as part of a statistics class, but now wish to better understand how it works, where it breaks down, and how it can be applied in a thorough way. Due to the need to constantly focus on the application of linear models, the class is unsuitable for those who want an introductory course in general statistics. We briefly cover during one of the sessions some basic statistical concepts and tests, but this is only so that we can all delve into the topic of linear models from an equal footing. This cannot be considered a substitute for a good coverage of introductory statistics.

We start off the class with a condensed review of some fundamental concepts in basic statistics: the z and t distributions, hypothesis testing, confidence intervals and correlation. This short overview intends to provide a solid foundation from which to advance in the following days. We already begin to discuss now a few basics of regression, e.g. how it goes beyond correlation, and for what type of questions it is helpful. In the lab session, we will go through a few of the basic data manipulation procedures that are commonly required before running any regression: data cleaning and recoding, transformations of data etc. This is also a good opportunity for participants to get familiar, if need be, with working with syntax files in R and Stata, and with the Stata or Rstudio interfaces.

The second session delves fully into the fundamentals of Ordinary Least Squares (OLS) regression: how the estimation is carried out, and how we interpret the coefficients both for simple (one predictor) and multiple regression (two or more predictors).. Some basic formulas will be presented, but the goal will be to gain an intuitive understanding of how the estimation process functions, and what the results mean. In the lab session we put this newly-gained knowledge to the test, by running a few examples of linear models in R or Stata. We will interpret the output and the model fit, and generate predictions based on the model using the functionality provided by the Zelig package in R, and the Clarify add-on in Stata.

In the third session we advance in our coverage of OLS by tackling the issue of uncertainty of estimates, as well as some model specifications that almost always appear in empirical research. In the latter case I refer to dummy indicators (categorical predictors). We will discuss where estimated uncertainty comes from, how it impacts your results, what influences uncertainty, and how you ought to communicate it to your audience. In the lab component we learn how to run these model specifications with the use of R/Stata. The fourth session is devoted to preventing abuse in the estimation of linear models. As with the vast majority of statistical procedures, a series of assumptions underpin OLS regression. If these are not met we have little reason to put our faith in the results we obtain. In this session we go over these assumptions, how they influence the results when they are not met, and what strategies we have available in the toolbox to overcome this situation. In the lab we turn to these issues from a practical perspective. We use R to run a test regression, assess whether the assumptions are met, and correct for the assumption violations that exist. Through a step-by-step process, participants have the opportunity to see how their estimates and model fit changes when engaging in such a process.

Finally, we devote the last session  mainly to a recap of the most important elements in the regression framework, based on the participants’ needs and requests. We also delve into the various ways in which regression results can be presented, depending on the audience of your work: quantitative scholars, policymakers, or a general audience. As a slightly more advanced topic, I conclude with a presentation of how interactions can expand considerably the range of hypotheses you can test with regression. In the lab I intend to present code for producing tables and graphs with regression estimates, as well as demonstrate how an interaction effect between predictors can be set up, estimated, and evaluated.

Throughout the class, we will focus on graphical methods and intuitive quantities when presenting results from linear models. Graphs rather than tables, predicted values and uncertainty around them rather than coefficients and standard errors. While tables of coefficients are still the dominant way of presenting results in academic journals, graphs and predicted values tend to be preferred in reports and analyses done for larger, non-technical audiences. I strongly believe that participants should be exposed to both types, and should tailor the delivery of their results to the audience at a particular time.

In order to guarantee that we can progress at a steady and firm pace, participants will need to have a thorough understanding of basic statistical concepts such as mean, median, variance, standard deviation and standard error. Additionally, very basic statistical tests and analyses, such as t tests and ANOVA should be familiar to participants at least at a theoretical level. The class will be carried out primarily in R, but with Stata examples and scripts as well. Participants are expected to have basic knowledge of either one of these two software packages: reading in data, basic data recoding skills, and very basic plotting commands.