Logistic Regression and General Linear Models

Researchers, who are working in broadly-defined social sciences, have to deal, quite often, with analyses in which the dependent variable is not a continuous variable defined on the interval scale. These are the situations where the dependent variable is either (1) a binary variable, when respondents selected one out of two option (e.g., answer to the question whether a respondent voted in the last elections), (2) a nominal variable, when respondents selected one out of three or more options (e.g., answer to the question which party the respondent voted in the last elections),  (3) an ordinal variable (e.g., when a respondent chooses one of the answers on the Likert scale) or (4) a variable counting the number of occurrences of a phenomenon (e.g., answer to the question how many times a respondent participated in protest actions). For this type of dependent variables, it is not appropriate to use Ordinary Least Square (OLS) regression models. The most common approach is using General Linear Models (GLM). These models are estimated in a different way than linear regression models. Besides, interpretation of GLM models is also much more complex than OLS models. Although GL models are used very often in the field of social sciences, their use and correct interpretation still give difficulties for the researchers.

The aim of this course is to provide an introduction to the General Linear Models. The participants of the course will gain practical skills related to the use of GLM. However, the course will not be abstracted from the statistical theory. The theoretical aspects of GL modeling will be presented to the extent that is necessary to understanding and interpret GL models properly. Theoretical aspects will be introduced in an approachable way that is understandable for participants without rudiments in matrix algebra or calculus.

The course will be conducted according to the following scheme: On the first day, we will deal with the regression model with a binary dependent variable. We start with discovering why it is inappropriate to use OLS models in such cases, in particular, which OLS model assumptions are not met, and what might be the negative consequences of using OLS models for this type of data. Next, it will be shown how a linear model can be generalized so that it can be applied to models with a limited dependent variable. The participants will learn such concepts as the linear predictor and the link function. The first day of the course will end with a presentation of the Maximum Likelihood Estimation as a technique for estimating logistic regression model parameters. During the next two days, participants will develop practical skills related to the interpretation of the binary logistic regression model, i.e. the interpretation of regression coefficients and odds ratios. Moreover, measures of goodness of fit of the models and various versions of pseudo-R-squared measures will be presented. The next point will be the extension of additive logistic regression models by introducing interactions between independent variables. Participants will learn how to correctly interpret a logistics binary regression model that incorporates interaction terms. Participants will also learn the possibilities of reporting results using predicted probabilities, in particular through the techniques of statistical graphics. The fourth day will be devoted to the models with a nominal dependent variable, i.e. multinomial logistic regression and conditional logistic regression models. These techniques will be compared regarding their analytical capabilities and possible applications. On the fifth day, participants will learn a techniques designed for an ordinal dependent variable, i.e. ordinal logistic regression, and regression models for counts. In terms of counts, participants will learn such techniques as Poisson regression and negative binomial models.

The application of each method will be illustrated using analyses based on real-world data. General Linear Models will be presented together with its constraints and limitations.

For each of the techniques presented during the course, participants will acquire a similar set of practical skills. By the end of the course, participants will be able to run by themselves GLM models on their own datasets, interpret the GLM regression coefficients and odds ratios, assess the goodness of fit of the models, estimate the uncertainty of predicted effects using simulations, and present the results using statistical graphics techniques.

Each day, the course will consist of two parts. During the first 90 minutes, it will be a workshop with elements of the lecture. The second part, lasting for another 90 minutes, will take the form of a lab session. During lab sessions, participants will conduct practical exercises using the techniques introduced during the first part. It is recommended that the participants of the course bring their own laptops and be able to use them during both parts of the course.

During the course, participants will be given assignments. Students, who will need to get ECTS credits, will be meant to complete assignments after each class. Each assignment will consist of a set of practical exercises related to the techniques introduced on a given day. For assignments, participants will be able to use their own data (which is strongly recommended) or the data provided by the instructor.

The practical part of the course (the lab session and assignments) will be based on the statistical software R. This open-source software enables the efficient implementation of the GLM models. R also allows for the advanced interpretation of GLM models, and the graphical capabilities of the R allow for an effective and relatively simple presentation of GLM results using the statistical graphics techniques.

This is advanced course. In order to get the most out of the course, participants should met the following requirements:

1. Participants are expected to understand the logic of inferential statistics. In particular, participants should be familiar with hypothesis testing and such concepts as confidence intervals and significance level. Participants not familiar with these topics should have taken introductory course ‘Introduction to Statistics for Political and Social Scientists’ offered at the ECPR Winter School or ‘Introduction to Inferential Statistics: What you need to know before you take regression’ offered at the ECPR Summer School or have obtained equivalent prior knowledge through other means.

2. Participants should be familiar with rudiments of linear regression model estimated using Ordinary Least Squares method. In particular, participants should be familiar with the logic of the linear regression analysis, assumptions of the linear regression model and regression with dummy variables. Participants not familiar with the topics mentioned above should have taken the course ‘Linear Regression with R/Stata: Estimation, Interpretation and Presentation’ offered at the ECRP Winter School or ‘Multiple Regression Analysis: Estimation, Diagnostics, and Modelling’ offered at the ECPR Summer School or have obtained equivalent prior knowledge through other means.

3. The course relies on the R software. Participants should have at least a basic understanding of the R language. In particular, participants should be able to import into R a dataset written in the SPSS format and run linear regression model. Moreover, participants should be able to conduct a set of basic data manipulations in R, such as: selecting observations, selecting variables and computing new variables using existed variables. Participants new to R should have taken preparatory short course ‘Introduction to R’ offered at the ECPR Winter School or have obtained equivalent prior knowledge through other means.