ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

Your subscription could not be saved. Please try again.
Your subscription to the ECPR Methods School offers and updates newsletter has been successful.

Discover ECPR's Latest Methods Course Offerings

We use Brevo as our email marketing platform. By clicking below to submit this form, you acknowledge that the information you provided will be transferred to Brevo for processing in accordance with their terms of use.

Survey Design

Course Dates and Times

Monday 17 – Friday 21 February 2019, 09:00–12:30
15 hours over five days

Kathrin Thomas, course instructor for Survey Design at ECPR's Research Methods and Techniques
Kathrin Thomas

kathrin.thomas@abdn.ac.uk

University of Aberdeen

Surveys ask a lot of people a lot of questions. They are the most popular method of data collection and are, arguably, very straightforward to design and run. Arguably? Yes, arguably.

In practice, designing a survey that provides us with high-quality data is a lot more complex than many people think.

This course provides in-depth guidelines to survey research design, embedding and discussing the so-called Total Survey Error (TSE) framework.

The course will discuss the assumption of the TSE and critically evaluate the full survey life cycle against this paradigm.

We will discuss the core of idea of survey sampling and trade-offs between probability and non-probability samples; mode selection and trade-offs; question and questionnaire design, fieldwork implementation; effects of response and non-response; as well as new challenges to survey design and extensions to the TSE.

The content will be applicable to surveys of individuals, households, and organisations.

Tasks for ECTS Credits

2 credits (pass/fail grade) Attend at least 90% of course hours, participate fully in in-class activities, and carry out the necessary reading and/or other work prior to, and after, class.

3 credits (to be graded) As above, plus complete 30–60 minute assignments throughout the week.

4 credits (to be graded) As above, plus complete a take-home report outlining or evaluating a survey design.

If you are working on a survey or a larger survey project, or if you are thinking about conducting a survey, you are encouraged to work on your own project.

Instructor Bio

Kathrin Thomas teaches at the University of Aberdeen. 

Previously, she was Senior Research Specialist in the Department of Politics at Princeton University, where she worked on the Arab Barometer.

Her main research interests are survey research methodology as well as social and political behaviour. Kathrin's current research includes measuring sensitive behaviour and attitudes using Randomised Response Techniques and other experimental designs in surveys, interviewer effects, and other methodological issues in survey practice.

Kathrin has extensive experience with survey data design, management, and analysis from her work on the Arab Barometer; the European Social Survey; the Austrian National Election Study; the Comparative Study of Electoral Systems; and applied survey practice at Kantarpublic (former TNS BMRB).

Why survey design?

  • You want to ask a lot of people (a lot of) questions?
  • You have never heard of the Total Survey Error?
  • You always wanted a tailored design of some kind, but you feel your sample is limited?
  • You fear CAPI, CASI, CATI, and CAWI put you in a bad 'mode'?
  • You frequently question questions and questionnaires?
  • You always run out of appropriate filters?
  • You think most things happen online anyway?
  • You cannot get the 'grids' on the radio buttons?
  • You like to mix it up?

Focus

  • Theoretical Framework
    • This course provides in-depth guidelines to survey research design following the so-called Total Survey Error (TSE) framework.
    • Critical discussion of all stages of the TSE
    • Critically evaluate the full survey life cycle against the TSE Framework.
  • Survey Design
    • Sampling designs and error including trade-offs between probability and non-probability samples
    • Modes of data collection including benefits and challenges of each mode
    • Question and questionnaire design
  • Survey Implementation
    • Fieldwork preparation, implementation, and monitoring
    • Pretesting
    • Response rates; response and non-response effects
    • Depending on student needs; session on the online implementation of smaller surveys using Lime Survey, Qualtrics, or Survey Monkey
  • Survey Documentation & Archiving
    • Preparing transparent documents for survey documentation
    • Dealing with archiving survey data

Structure

  • The course provides an applied and interactive learning approach
    • You will evaluate and learn from mistakes that existing surveys make; the Instructor will bring in examples from studies that she has worked on learned from
    • You will frequently be asked to improve designs on the basis of what you have learned about each stage of the survey lifecycle
    • You will be encouraged to work on your own survey projects during the week, receiving detailed feedback from the Instructor

What you will take home…

  • Knowledge & understanding
    • A comprehensive knowledge of the full survey life cycle
    • A systematic understanding of the Total Survey Error (TSE) framework
    • The ability to critically evaluate survey designs
  • Skills
    • The ability to design your own survey project
    • The skill to manage your own survey project
    • The capability to manage and transparently document surveys
  • Values & attitudes
    • A sound understanding of the relevance of surveys in the social sciences
    • Recognition of the importance of integrity when designing, implementing, and managing survey as well as and reporting survey research
    • The ability to learn independently in response to self-reflection on your understanding of the principles and application of survey research methodology

…and what you will not take home

  • This is a course focusing on survey research design. You will not get an in-depth introduction to survey data analysis. The Instructor will point you to relevant software and introductory literature focusing on the special requirements of survey data analysis.
  • This is not a course in statistics. Some technical background will be covered when we discuss aspects of the survey lifecycle such as sampling strategies, response rates. etc.
  • The course focuses on standardised quantitative interviews. It is not a course on conducting (semi-) structured qualitative interviews.

THIS IS THE WINTER SCHOOL 2019 OUTLINE

No previous experience in survey research is needed, but experience in data analysis using survey data is beneficial.

You don't need any statistical software skills. The course focuses on the design of surveys, not the analysis of survey data.

Day Topic Details
1 The survey life cycle; The Total Survey Error (TSE) framework
4 Survey implementation
3 The Psychology of the survey response; Question and questionnaire design
2 Survey sampling; Modes of data collection
5 Survey documentation and archiving; The future of surveys; Wrap up
Day Readings
Day 1

Lyberg, L. E. and Weisberg, H. F. (2016)
Total survey error: A paradigm for survey methodology
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE handbook of survey methodology. Sage, Chapter 2

Weisberg, H. F. (2005)
The total survey error approach: A guide to the new science of survey research, Chapters 1 and 2
University of Chicago Press

Paul P. B. (2016)
Total Survey Error Paradigm: Theory and Practice
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Chapter 10
Sage

Groves, Robert M., and Lyberg, L. (2010)
Total survey error: Past, present, and future
Public opinion quarterly, 74.5: pp: 849–879

McNabb, D. E. (2013)
Non sampling error in social surveys, Chapter 3
Sage publications

Groves, R.M. (2011)
Three Eras of Survey Research
Public Opinion Quarterly, Volume 75, Issue 5, pp: 861–871

Saris, W. E. and Gallhofer I. N. (2014)
Design, evaluation, and analysis of questionnaires for survey research, Chapter 1
John Wiley & Sons
Day 2

Tille, Y. and  Matei, A. (2016)
Basics of sampling for survey research
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Chapter 21
Sage

Vehovar, V., Toepoel V. and Steinmetz S. (2016)
Non-Probability Sampling
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology
Sage

Toepoel, V. (2016)
Doing Surveys Online, Chapters 4 and 5
Sage

Callegaro, M., Manfreda, K. L., and Vehovar V. (2015)
Web survey methodology, Chapter 2.2
Sage

Groves, R. M., Floyd, J., Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R (2011)
Survey methodology. Vol. 561, Chapter 2 and 3
John Wiley & Sons

De Leeuw E. D., Hox J. J. (2011)
Internet surveys as part of a mixed-mode design
In Social and Behavioral Research and the Internet, eds Das M., Ester P., Kaczmirek L., eds, Chapter 3
New York, NY: Routledge

Lynn, P. (2015)
Coping with nonresponse: Overview and introduction
In Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research, Chapter 26
Routledge

De Leeuw, D. (2005)
To mix or not to mix data collection modes in surveys
Journal of official statistics 21.2, pp: 233–255

Stoop, I. (2016)
Unit non-response
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y. eds. The SAGE handbook of survey methodology, Chapter 27
Sage

Edith de Leeuw and Jernal Berzelak (2016)
Survey mode or survey modes?
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y. eds. The Sage handbook of survey methodology, Chapter 11
Sage

Dillman, Don A., Jolene D. S, and Christian, L. M. (2014)
Internet, phone, mail, and mixed-mode surveys: the tailored design method, Chapter 8
John Wiley & Sons
Day 3

Smyth, J. D.. (2016)
Designing questions and questionnaires
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE Handbook of Survey methodology, Chapter 16
Sage

Groves, R. M. Floyd, J. Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R. (2011)
Survey methodology. Vol. 561, Chapter 7
John Wiley & Sons

Dillman, Don A., Smyth, J. D. and Christian, L. M. (2014)
Internet, phone, mail, and mixed-mode surveys: the tailored design method, Chapters 4, 5 and 6
John Wiley & Sons

Couper, M. P. (2008)
Technology and the Survey Interview/Questionnaire in Envisioning the Survey Interview of the Future eds Conrad, F. G.  and Schober, M. F., Chapter 5

Engel, U. and Koeste, B. (2015)
Response effects and the cognitive involvement in answering survey questions
In Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P. Improving survey methods: Lessons from recent research, Chapter 4
Routledge

Tourangeau, R. and Yan T. (2007)
Sensitive questions in surveys
Psychological bulletin, 133.5, pp: 859–883

Kreuter, F., Presser, S. and Tourangeau, R. (2008)
Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity
Public opinion quarterly 72.5, pp: 847–865

Krumpal, I. (2013)
Determinants of social desirability bias in sensitive surveys: a literature review
Quality & Quantity, 47.4: pp. 2025–2047

Jann, B. (2015)
Asking sensitive questions: Overview and introduction
In Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research, Chapter 9
Routledge
Day 4

Vardigan, M., Granda, P. and Hoetler, L. (2016)
Documenting Survey data across the life cycle
In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE Handbook of Survey methodology, Chapter 29
Sage

Dillman, Don A., Smyth, J. D. and Christian, L. M. (2014)
Internet, phone, mail, and mixed-mode surveys: the tailored design method, Chapter 11
John Wiley & Sons

Couper, M. (2008)
Designing Effective Web Surveys, Chapters 4 and 6
New York: Cambridge University Press

Toepoel, V. (2016)
Doing Surveys Online, Chapters 8, 9,10, 11, 13 and 14
Sage

Callegaro, M., Manfreda, K.L. and Vehovar, V. (2015)
Web survey methodology, Chapters 2.4, 5.3, 3, 4
Sage

Lugtig, P. J.  and Toepoel, V,. (2016)
The Use of PCs, Smartphones, and Tablets in a Probability-Based Panel Survey: Effects on Survey Measurement Error
Social Science Computer Review, 34, 1, pp: 78–94

Kaczmirek, L. (2015)
Conducting web surveys: Overview and introduction
In Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research, Chapter 13
Routledge

Bethlehem, J. (2015)
Web Surveys in official statistics
In Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P., eds. Improving survey methods: Lessons from recent research, Chapter 14
Routledge

Engel, U., Scherpenzee A. (2015)
Conducting access panels: Overview and introduction
In Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P., eds. Improving survey methods: Lessons from recent research
Routledge

Knapp, H., Kirk, S. A., (2003)
Using pencil and paper, internet and touch-tone phones for self-administered surveys: Does methodology matter?
Computers in Human Behavior, 19 (1), pp: 117–134
Day 5

Groves, R. M. Floyd, J. Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R. (2011)
Survey methodology. Vol. 561, Chapter 10
John Wiley & Sons

Toepoel, V. (2016)
Doing Surveys Online, Chapters 6 and 15
Sage

Callegaro, M., Manfreda, K. L., and Vehovar, V. (2015)
Web survey methodology, Chapters 5, 6 and 7
Sage

Software Requirements

None

Hardware Requirements

Please bring your own laptop with an updated browser (flash player etc.) for online survey programming – Mac or Windows is fine.

Literature

As indicated in the day-to-day schedule.

Recommended Courses to Cover After this One

Winter and Summer School

Course in (survey) data analysis in SPSS, Stata or R