ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

SD301 - Advanced Bayesian Statistics for the Social Sciences

Instructor Details

Instructor Photo

Juraj Medzihorsky

Institution:
University of Gothenburg

Instructor Bio

Juraj Medzihorsky gained his PhD in Political Science from CEU and is currently a Postdoc at the Department of International Relations there.

His main research interests are models of election returns, modelling unobserved heterogeneity with mixtures, and text analysis.

His research has appeared in Political Analysis and PS:Political Science & Politics.

  @medzihorsky


Course Dates and Times

Monday 1 to Friday 5 August 2016
Generally classes are either 09:00-12:30 or 14:00-17:30
15 hours over 5 days

Prerequisite Knowledge
  • Understanding of the basics of Bayesian inference
  • Understanding of the differences between frequentist and Bayesian inference
  • Experience with the Generalized Linear Model (GLM), e.g. OLS, logit/probit
  • Experience with Multi-Level Models (MLM) is helpful, but not necessary
  • Comfortable with R
  • Some experience with at least one of the following WinBUGS/OpenBUGS/JAGS/Stan recommended, but not necessary
Short Outline

The course is intended for participants with a working understanding of Bayesian statistics who wish to learn about topics relevant for applied social research which are not standardly covered by introductions to Bayesian statistics for social scientists. These include robust regression, handling limited or zero-inflated dependent variables, the use of informative priors and regularization, mixture models, and latent variable models. The goal of the course is to allow the participants to competently use the methods in their own research. For that reason, the focus is on existing multi-purpose software as opposed to programming the Markov chain Monte Carlo algorithms from the ground, and the course does not include detailed technical expositions of the algorithms on which the methods rest. The primary software will be R and JAGS. Furthermore, the participants will be introduced also to  Stan, which allows to use Hamiltonian Monte Carlo with JAGS-like user comfort.

Long Course Outline

Day 1: Review

The first day reviews the material commonly included in introductory courses of Bayesian statistics for social sciences. These are the basics of Bayesian inference, the differences between frequentist and Bayesian statistics, basics of Bayesian computation using Markov chain Monte Carlo, Bayesian Normal linear regression, Bayesian Binomial logistic regression, Bayesian hierarchical linear model, and basics of Bayesian model evaluation. The topics will be reviewed with the emphasis on the use of the models with JAGS. The session is not a substitute for a first introduction to Bayesian statistics.

 

Day 2: GLM Special Topics
Normal linear regression and Binomial logistic regression are workhorses of social science research. However, in some contexts other members of the GLM family might be more appealing. Such contexts include count, limited, or zero-inflated left-hand-side (outcome/dependent) variables. The flexibility of Bayesian computation as implemented in JAGS or Stan makes easily accessible to applied researchers a wide variety of sampling distributions and link functions that allow to handle these issues in GLM/MLM. For example, in the linear-model context it might be desirable to reduce the influence of 'outlier' observations, i.e., a robust alternative to the Normal sampling distribution might be desirable. The t, Laplace, and Logistic distributions can be used to this aim. For another example, in some contexts the outcome of interest is a proportion limited to [0%, 100%], and the data contain considerable number of 0% and 100%. The Beta and Binomial distributions can be jointly used to handle it. For a final example, due to substantive or operational concerns a linear model that minimizes the largest absolute residual might be desirable, which can be achieved using the Uniform sampling distribution. 

 

Day 3: Informative Priors and Regularization

Introductions to Bayesian GLM often rely on priors that lead to results numerically close to those under MLE. Other priors are preferable in many contexts. Extra-data information such as expert judgment or qualitative case studies can be used to formulate priors. This is especially useful when the data is weak in the sense that it offers only limited information on the parameters of interest due to its small size or noisiness. Also, various deals on the bias-variance trade-off are available through priors. First, weakly informative priors can be used to handle linear separation in logistic models, or in other instances where MLE estimates are known to be biased. Second, through the use of priors are available Bayesian ridge and lasso, which offer to exchange an increase in bias for a substantial reduction of variance. The ridge can be used to handle multicollinearity. The lasso provides variable selection, which can appeal in settings with many right-hand-side (predictor/independent) variables. Furthermore, the 'horseshoe' prior offers an additional Bayesian alternative to the lasso.
 

 

Day 4: Mixture Models
In social science research, it is often an easy assumption that the observations differ in important but unobserved ways. In some contexts, it is appealing to assume that the population is composed of a limited number of groups that differ in a characterstic such as the association between two variables of interest. Mixture models can be used to handle such unobserved heterogeneity. The flexibility of MCMC as implemented in JAGS allows for an easy use of a variety of such models. Furthermore, from the Bayesian perspective it can be appealing to think of the groups in terms of a hierarchical model under which the memberships in upper-level units are not observed. Finally, mixture models can be used to evaluate competing theories using models in which each of the mixture components represents one of the theories.

 

Day 5: Latent Variable Models

In social science research, latent variable models are used to a variety of aims, including the measurement of concepts which are not directly observable. An example of such application is scaling  two-way contingency tables to obtain estimates of legislator ideal points or voter ideology. Bayesian Latent Variable Models allow for straightforward modeling of hierarchical structures in such data as well as for the use of extra-data information.

Day-to-Day Schedule

Day-to-Day Reading List

Software Requirements

R and JAGS, interfaced with the rjags R package. The rstan package is highly recommended by the instructor, but not necessary. It is not possible to indicate precise versions, as different versions might be available depending on the operating system. In any case, this  should make no difference with regards to the course content. The participants are strongly encouraged to set up the software in advance, and to contact the instructor regarding software issues if necessary.

 

Although it is not necessary, the participants might wish to use either RStudio or an editor with REPL. Examples of the latter are vim with vim-r-plugin (the instructor's personally preferred option), Emacs with Emacs Speaks Statistics, or Sublime Text with Sublime REPL and R-box. These options save time in reading and editing R scripts.

 

R: https://cran.r-project.org/

JAGS: http://mcmc-jags.sourceforge.net/

Stan and rstan: http://mc-stan.org/

vim-r-plugin: http://www.vim.org/scripts/script.php?script_id=2628

Emacs Speaks Statistics: http://ess.r-project.org/

on ESS see also: http://kieranhealy.org/resources/emacs-starter-kit/

RStudio: https://www.rstudio.com/products/rstudio/download/
 

Hardware Requirements

Participants to bring their own laptop. Disk space requirements vary depending on the operating system. In any case, 5 Gb of disk space should be enough under any conditions, and 2 Gb under most. For participants who wish to use their laptops, a minimum of 2 Gb of RAM is recommended.

The following other ECPR Methods School courses could be useful in combination with this one in a ‘training track .
Recommended Courses Before

Introduction to Bayesian Inference

Gentle Introduction to Bayesian Statistics

Additional Information

Disclaimer

This course description may be subject to subsequent adaptations (e.g. taking into account new developments in the field, participant demands, group size, etc). Registered participants will be informed in due time.

Note from the Academic Convenors

By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach these prerequisite items. If in doubt, contact the instructor before registering.


Share this page