Gentle Introduction to Machine Learning for the Social Sciences

Machine learning in the social sciences is still in its infancy. It has been recently used in specific areas such as policy evaluation, electoral fraud detection, election forecasting, and quantitative text and social media analysis, for example, but is still not widely known or understood even by many skilled quantitative-oriented social scientists. Given its rise in related disciplines (such as economics, as The Economist reports here: http://econ.st/2g8g8Nk), however, it must be expected that machine learning will be a new trend in quantitative social science analysis in the coming years. This course offers the basic toolkit for students to be able to understand, evaluate, and build their own machine learning models for research and applied problems.  ML algorithms will be introduced conceptually, and we will not go over the math behind their workings. All are available as off-the-shelf packages in R, so students will not be required to code them from scratch, but only to use ready R functions.

The first day will be a general introduction into the logic of machine learning, and will situate students into how ML is related to more familiar statistical methods. We will discuss the ML emphasis on prediction (and how this is only another side of the explanation coin) and substantive effect over statistically significant ones; the problems and potential solutions to overfitting, and the bias/variance trade-off. We will also talk about the building blocks of machine learning models and algorithms: the role of hyperparameters, tuning, cross-validation, supervised vs. unsupervised learning, and the Bayes classifier as an ideal goal. Throughout our discussions we will refer to (and apply in R) the K-nearest-neighbor classifier to illustrate those conceptual points.

Days 2 and 3 deal with supervised learning, the class of problems in which we know the value of the dependent variable for at least part of our data, meaning we can train models and evaluate the performance of their prediction against actually observed values. Day 2 will discuss classification problems, some of the most common problems for which ML is applied. It is especially useful, for example, as substitute to human coding when there is some raw data available – think of classifying political parties as left or right, or countries as democratic or not, etc. We will cover some of the most common algorithms in use, such as linear discriminant analysis (LDA), support vector machines (SVM), and logistic regression with boosting.

In day 3 we move to classification and regression problems using two other very common and versatile classes of algorithms: tree-based ones, and variable-selection methods. Tree-based algorithms (decision trees, random forests, and bagging), have been often used in diagnosis and effective decision making, but are also excellent for social scientists because they give interpretable explanatory models. They allow users to identify what are the most important variables for predicting the outcome (substantive importance, not statistical significance), what means they can be applied to any problem in which linear regression could be used. Variable selection models are regression models in which independent variables with small coefficients (meaning, low predictive power), are penalized and effectively removed, retaining only those that substantively help predict/explain the outcome.

In the fourth day we cover unsupervised learning. These are cases where we know the characteristics of our observations in independent variables, but not on the dependent, so it is not possible to train the model on known outcome values. We will discuss and apply clustering methods, which identify groups of observations in the data given the independent variables, outlier detection, and latent variable models from a ML perspective – here, familiarity with Principal Component Analysis and Structural Equation Modeling is an advantage, but not a need.

The last day will be dedicated to a brief introduction into more advanced issues in Machine Learning, and will depend to some extent on students’ interests and needs from the course. Topics to be covered, initially, include how these models are applied to cases of many more variables than observations (p > N, common in text analysis), neural networks and deep learning as extremely efficient predictors, and the power of ensembles (learning models with several algorithms and then getting their joint prediction) for more accurate predictions.

At the end, students will be able to identify what kind of machine learning method better suits the data and question they have at hand (whether it is a classification or regression problem, if supervised or unsupervised, etc), and to fit and interpret the appropriate models covered in class. Students who want full credits will have to submit daily take-home exercises.

This is an introductory course for social science students with no prior knowledge of machine learning or algorithms at large. Those who are familiar with and have used these methods already, or who have experience with computational sciences, programming, and software development, are likely to get frustrated with the basic level of classes.

This course will be taught from a very beginner’s perspective to statistical learning. Students are only expected to be familiar with concepts and practices of traditional multivariate regression analysis. All examples will be given in R, so students should also have a minimal working knowledge of it (which can be replaced by an advanced knowledge of another programming language, such as Python, to adapt the codes and exercises on your own).