Multilevel Models: Specification, Inference and Reporting

Key topics covered

Much of the data used in comparative political science and sociology is nested, with survey respondents grouped in countries, pupils in schools and employees in organisations. Multilevel models make it possible to analyse individuals and their contexts together, to separate variation within and between groups, and to test how context shapes relationships at the individual level. Specifying these models well involves a series of choices, such as whether a multilevel model is needed at all, how to centre predictors and which effects should vary across groups. Each choice shapes the conclusions that can be drawn.

This course builds the two-level model step by step, and each session is organised around one of these decisions. You will learn which model your question and your data require, how to fit it, and how to report it so that others can see what you estimated. The course covers two-level models for continuous outcomes, with an introduction to binary outcomes on Day 4. You will learn to estimate, read and check a multilevel logit, but the course does not cover the full range of generalised linear mixed models.

Day 1: Clustered data, variance decomposition and partial pooling

The course starts from the structure of the data rather than from the model. We decompose variance within and between countries, interpret the intraclass correlation, and compare complete pooling, no pooling and partial pooling as three answers to the same question. The session then asks whether a multilevel model is needed at all, or whether cluster-robust standard errors, fixed effects or aggregation would serve the research question better. The exercise asks you to compute the intraclass correlation for an outcome of your choice and to justify your modelling strategy.

Day 2: Random-intercept models, centring and model comparison

We introduce random-intercept models with predictors at both levels, together with what the random intercept assumes about predictors and country effects. Models are compared with likelihood-ratio tests, and we discuss when to use maximum likelihood (ML) or restricted maximum likelihood (REML). Centring is the core of the session. Grand-mean centring, group-mean centring and the within/between decomposition can turn the same predictor into different coefficients and different substantive claims. We also assess the variance explained at each level. You then estimate one model under alternative centrings and explain which one answers your research question.

Day 3: Random slopes and cross-level interactions

Relationships are now allowed to vary across countries, and variance components become as important as coefficients. Cross-level interactions require a random slope on the lower-level variable, without which inference is overconfident. The session also covers tests of variance components, convergence warnings and singular fits, and a step-by-step strategy for building models. You will build and document a sequence of nested models.

Day 4: Level-2 sample size and multilevel logistic regression

The first half asks how much a small number of countries can support. It looks at what happens to variance estimates, standard errors and cross-level interactions with 20 countries rather than 200. It also asks why the literature disagrees on how serious the problem is, and at how small-sample corrections (REML with Kenward–Roger or Satterthwaite degrees of freedom) change the results. A group discussion compares two opposing published positions. The second half extends the framework to binary outcomes, covering the latent scale, why the intraclass correlation is harder to define, and why coefficients are not comparable across nested models.

Day 5: Model diagnostics, influential units and reporting standards

Before reporting, models need checking. We examine the residuals and the influence of single countries, re-estimating the model while leaving out one country at a time. We then discuss what readers need in order to reconstruct what was estimated, and how to state the inferential scope of the results. The rest of the day is supervised work on your own data, or on a provided extension of the CSES example, with individual consultation. You will leave with a results table and an interpretation paragraph for your own model.

Post-course assignment

You will analyse your own nested data or, if you have none, from any publicly available comparative survey, for example the CSES, the European Social Survey (ESS) or the European Values Study and World Values Survey (EVS/WVS). You will write a report of about 3,000 words, with a reproducible R script. The report states the research question and explains why the data are nested. It assesses whether a multilevel model is warranted, using the intraclass correlation and the alternatives considered, and justifies the centring choices. It describes the random-effects structure, including random slopes for any cross-level interaction. It ends with a results table and an interpretation that follow the reporting standards discussed on Day 5. Assessment focuses on the justification of modelling decisions and the accuracy of interpretation rather than on the complexity of the model.

Prerequisite knowledge

You should be familiar with multiple linear regression, including how to specify a model and interpret coefficients, interaction terms and standard errors. You should also have basic R skills, such as loading a dataset, recoding variables, fitting a model with lm() and reading the output. Some knowledge of logistic regression is helpful but not required. No prior knowledge of multilevel models is assumed.

Install R and RStudio (both free) before the course. The list of required packages (lme4 and a few companion packages, all free), with the code to install them, will be shared in advance. The course uses Module 5 of the Comparative Study of Electoral Systems (CSES), which is free of charge. Downloading it may require a free registration. No paid software or accounts are needed.

Before the course

Allow about four hours in total.

1. Install R, RStudio and the course packages (30 minutes).
2. Read chapters 1 and 2 of Hox, Moerbeek and van de Schoot (2017), Multilevel Analysis: Techniques and Applications, 3rd edn (about two hours).
3. As an optional refresher, fit and interpret a multiple regression with an interaction term in R (about one hour).
4. Download the CSES Module 5 data. If you wish to work on your own data on Day 5, prepare a nested dataset and a research question. You will be asked to send a short description of them during the course week (30 minutes).

The course draws on Hox, Moerbeek and van de Schoot (2017) for the theory and on Finch, Bolin and Kelley (2024), Multilevel Modeling Using R, 3rd edn, for the code. A reading list for each day will be shared before the course.

Learning commitment

The course includes three hours of live teaching each day for five days, alongside readings and other learning activities.

Once you have registered and paid, you will receive access to our Learning Management System (LMS) approximately two weeks before the course start date. Course materials, including pre-course readings, will be available here. Allow approximately 20 hours per week before the course to review the materials and complete readings and any pre-course tasks.

During the course week, you should allow approximately two to three hours per day for preparation and assignments, in addition to the live teaching sessions.

The course offers the opportunity to earn three ECTS credits. To earn a fourth credit, you will need to complete a post-course assignment involving approximately 25 hours of work.

Disclaimer

This course description may be subject to subsequent adaptations (e.g. taking into account new developments in the field, participant demands, group size, etc.). Registered participants will be informed at the time of change.

By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach prerequisite items. If in doubt, please contact us before registering.