Introduction to Machine Learning for the Social Sciences

Machine learning in the social sciences is still in its infancy. It has been recently used in areas such as policy evaluation, electoral fraud detection, election forecasting, and quantitative text and social media analysis, for example, but is still not widely known or understood – even by many skilled quantitative-oriented social scientists.

Given its rise in related disciplines (such as economics, as The Economist reports, however, we must expect machine learning to become a new trend in quantitative social science analysis in the coming years.

This course offers the basic toolkit to enable you to understand, evaluate, and build your own machine learning models for research and applied problems. I will introduce ML algorithms conceptually, and we will not go deeply into the math behind their workings. All are available as off-the-shelf packages in R, so you will not be required to code them from scratch, but only to use ready R functions.

Day 1
A general introduction into the logic of machine learning. You will learn how ML is related to more familiar statistical methods. We will discuss the ML emphasis on prediction (and how this is only another side of the explanation coin) and substantive effects over statistically significant ones; the problems and potential solutions to overfitting, and the bias/variance trade-off. We will also talk about the building blocks of machine learning models and algorithms: the role of hyperparameters, tuning, cross-validation, supervised vs. unsupervised learning, and the Bayes classifier as an ideal goal. Throughout our discussions we will refer to (and apply in R) the K-nearest-neighbour classifier to illustrate those conceptual points.

The following two days deal with supervised learning, the class of problems in which we know the value of the dependent variable for at least part of our data, meaning we can train models and evaluate the performance of their prediction against actually observed values.

Day 2
We discuss classification problems – those in which the dependent variable is categorical, and some of the most common problems for which ML is applied. It is especially useful, for example, as substitute to human coding when there is some raw data available – think of classifying political parties as left or right, or countries as democratic or not, etc.

We will cover some of the most common algorithms in use, such as support vector machines (SVM) and tree-based methods (random forests and boosting). Tree-based algorithms have often been used in diagnosis and effective decision making, but are also excellent for social scientists because they give interpretable explanatory models. They allow users to identify the most important variables for predicting the outcome (substantive importance, not statistical significance), which means they can be applied to any problem in which linear regression could be used.

Day 3
We move to regression problems – i.e. continuous outcomes. We look again at tree-based models and introduce variable-selection methods. Variable selection models are regressions in which independent variables with small coefficients (meaning, low predictive power), are penalised and effectively removed, retaining only those that substantively help predict/explain the outcome. These models are especially useful for cases of many more variables than observations (p > N), very common in text analysis.

Day 4
We cover unsupervised learning. These are cases where we know the characteristics of our observations in independent variables, but not on the dependent, so it is not possible to train the model on known outcome values. We will discuss and apply clustering methods, which identify groups of observations in the data given the independent variables, and outlier detection. Familiarity with Principal Component Analysis and Structural Equation Modelling is an advantage, but not essential.

Day 5
A brief introduction into more advanced issues in Machine Learning, which will depend to some extent on students’ interests and needs. Topics we will cover, initially, include neural networks and deep learning as extremely efficient predictors, the power of ensembles (learning models with several algorithms and then getting their joint prediction), and how to use machine learning for causal inference in social sciences.

By the end of this course, you will be able to identify what kind of machine learning method better suits the data and question you have at hand (whether it is a classification or regression problem, if supervised or unsupervised, etc), and to fit and interpret the appropriate models covered in class. Students who want full credits must submit daily take-home exercises.

This is an introductory course for social science students with no prior knowledge of machine learning or algorithms at large. Those who are familiar with and have used these methods already, or who have experience with computational sciences, programming, and software development, are likely to get frustrated with the basic level of teaching.

This is a beginner's course. I expect you to be familiar only with concepts and practices of traditional multivariate regression analysis.

All examples will be given in R, so you should also have a minimal working knowledge of this software (which can be replaced by an advanced knowledge of another programming language, such as Python, to adapt the codes and exercises on your own).

By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach these prerequisite items. If in doubt, contact the instructor before registering.