ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

Your subscription could not be saved. Please try again.
Your subscription to the ECPR Methods School offers and updates newsletter has been successful.

Discover ECPR's Latest Methods Course Offerings

We use Brevo as our email marketing platform. By clicking below to submit this form, you acknowledge that the information you provided will be transferred to Brevo for processing in accordance with their terms of use.

Survey Design

Course Dates and Times

Monday 25 February – Friday 1 March, 09:00–12:30
15 hours over 5 days

Kathrin Thomas, course instructor for Survey Design at ECPR's Research Methods and Techniques
Kathrin Thomas

kathrin.thomas@abdn.ac.uk

University of Aberdeen

Surveys ask a lot of people a lot of questions. They are the most popular method of data collection and are, arguably, very straightforward to design and run. Arguably? Yes, arguably.

In practice, designing a survey that provides us with high-quality data is a lot more complex than many people think.

This course provides in-depth guidelines to survey research design, embedding and discussing the so-called Total Survey Error (TSE) framework.

The course will discuss the assumption of the TSE and critically evaluate the full survey life cycle against this paradigm.

We will discuss the core of idea of survey sampling and trade-offs between probability and non-probability samples; mode selection and trade-offs; question and questionnaire design, fieldwork implementation; effects of response and non-response; as well as new challenges to survey design and extensions to the TSE.

The content will be applicable to surveys of individuals, households, and organisations.

Tasks for ECTS Credits

2 credits (pass/fail grade) Attend at least 90% of course hours, participate fully in in-class activities, and carry out the necessary reading and/or other work prior to, and after, class.

3 credits (to be graded) As above, plus complete 30–60 minute assignments throughout the week.

4 credits (to be graded) As above, plus complete a take-home report outlining or evaluating a survey design.

If you are working on a survey or a larger survey project, or if you are thinking about conducting a survey, you are encouraged to work on your own project.

Instructor Bio

Kathrin Thomas teaches at the University of Aberdeen. 

Previously, she was Senior Research Specialist in the Department of Politics at Princeton University, where she worked on the Arab Barometer.

Her main research interests are survey research methodology as well as social and political behaviour. Kathrin's current research includes measuring sensitive behaviour and attitudes using Randomised Response Techniques and other experimental designs in surveys, interviewer effects, and other methodological issues in survey practice.

Kathrin has extensive experience with survey data design, management, and analysis from her work on the Arab Barometer; the European Social Survey; the Austrian National Election Study; the Comparative Study of Electoral Systems; and applied survey practice at Kantarpublic (former TNS BMRB).

Why survey design?

  • You want to ask a lot of people (a lot of) questions?
  • You have never heard of the Total Survey Error?
  • You always wanted a tailored design of some kind, but you feel your sample is limited?
  • You fear CAPI, CASI, CATI, and CAWI put you in a bad 'mode'?
  • You frequently question questions and questionnaires?
  • You always run out of appropriate filters?
  • You think most things happen online anyway?
  • You cannot get the 'grids' on the radio buttons?
  • You like to mix it up?

Focus

  • Theoretical Framework
    • This course provides in-depth guidelines to survey research design following the so-called Total Survey Error (TSE) framework.
    • Critical discussion of all stages of the TSE
    • Critically evaluate the full survey life cycle against the TSE Framework.
  • Survey Design
    • Sampling designs and error including trade-offs between probability and non-probability samples
    • Modes of data collection including benefits and challenges of each mode
    • Question and questionnaire design
  • Survey Implementation
    • Fieldwork preparation, implementation, and monitoring
    • Pretesting
    • Response rates; response and non-response effects
    • Depending on student needs; session on the online implementation of smaller surveys using Lime Survey, Qualtrics, or Survey Monkey
  • Survey Documentation & Archiving
    • Preparing transparent documents for survey documentation
    • Dealing with archiving survey data

Structure

  • The course provides an applied and interactive learning approach
    • You will evaluate and learn from mistakes that existing surveys make; the Instructor will bring in examples from studies that she has worked on learned from
    • You will frequently be asked to improve designs on the basis of what you have learned about each stage of the survey lifecycle
    • You will be encouraged to work on your own survey projects during the week, receiving detailed feedback from the Instructor

What you will take home…

  • Knowledge & understanding
    • A comprehensive knowledge of the full survey life cycle
    • A systematic understanding of the Total Survey Error (TSE) framework
    • The ability to critically evaluate survey designs
  • Skills
    • The ability to design your own survey project
    • The skill to manage your own survey project
    • The capability to manage and transparently document surveys
  • Values & attitudes
    • A sound understanding of the relevance of surveys in the social sciences
    • Recognition of the importance of integrity when designing, implementing, and managing survey as well as and reporting survey research
    • The ability to learn independently in response to self-reflection on your understanding of the principles and application of survey research methodology

…and what you will not take home

  • This is a course focusing on survey research design. You will not get an in-depth introduction to survey data analysis. The Instructor will point you to relevant software and introductory literature focusing on the special requirements of survey data analysis.
  • This is not a course in statistics. Some technical background will be covered when we discuss aspects of the survey lifecycle such as sampling strategies, response rates. etc.
  • The course focuses on standardised quantitative interviews. It is not a course on conducting (semi-) structured qualitative interviews.

No previous experience in survey research is needed, but experience in data analysis using survey data is beneficial.

You don't need any statistical software skills. The course focuses on the design of surveys, not the analysis of survey data.

Day Topic Details
1 The survey life cycle; The Total Survey Error (TSE) framework
4 Survey implementation
3 The Psychology of the survey response; Question and questionnaire design
2 Survey sampling; Modes of data collection
5 Survey documentation and archiving; The future of surveys; Wrap up
Day Readings
Day 1

Lyberg, L. E. and Weisberg, H. F. (2016). Total survey error: A paradigm for survey methodology in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE handbook of survey methodology. Sage. (Chapter 2)

Weisberg, H. F. (2005) The total survey error approach: A guide to the new science of survey research. University of Chicago Press. (Chapter 1 and 2)

Paul P. B. (2016). Total Survey Error Paradigm: Theory and practice in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Sage.  (Chapter 10) 

Groves, Robert M., and Lyberg, L. (2010). Total survey error: Past, present, and future. Public opinion quarterly, 74.5: pp: 849-879.

McNabb, D. E. (2013). Non sampling error in social surveys. Sage publications. (Chapter 3)

Groves, R.M. (2011). Three Eras of Survey Research, Public Opinion Quarterly, Volume 75, Issue 5, pp: 861–871.

Saris, W. E. and Gallhofer I. N. (2014) Design, evaluation, and analysis of questionnaires for survey research. John Wiley & Sons.  (Chapter 1)
Day 2

Tille, Y. and  Matei, A. (2016). Basics of sampling for survey research n Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Sage. (Chapter 21)

Vehovar, V., Toepoel V. and Steinmetz S. (2016). Non-Probability Sampling in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Sage. (Chapter 22)

Toepoel, V. (2016). Doing Surveys Online. SAGE. (Chapters 4 and 5) 

Callegaro, M., Manfreda, K. L., and Vehovar V. (2015). Web survey methodology. Sage. (Chapter 2.2)

Groves, R. M., Floyd, J., Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R (2011). Survey methodology. Vol. 561. John Wiley & Sons. (Chapter 2 and 3)

De Leeuw E. D., Hox J. J. (2011). Internet surveys as part of a mixed-mode design, in Social and Behavioral Research and the Internet, eds Das M., Ester P., Kaczmirek L., eds. New York, NY: Routledge.  (Chapter 3)

Lynn, P. (2015). Coping with nonresponse: Overview and introduction in Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research. Routledge. (Chapter 26)

De Leeuw, D. (2005). 'To mix or not to mix data collection modes in surveys'. Journal of official statistics 21.2, pp: 233–255.

Stoop, I. (2016). Unit non-response in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y. eds. The SAGE handbook of survey methodology, Sage. (Chapter 27)

Edith de Leeuw and Jernal Berzelak (2016). Survey mode or survey modes? In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y. eds. The Sage handbook of survey methodology. Sage (Chapter 11)

Dillman, Don A., Jolene D. S, and Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons. (Chapter 8)
Day 3

Smyth, J. D.. (2016). Designing questions and questionnaire in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE Handbook of Survey methodology, Sage.  (Chapter 16)

Groves, R. M. Floyd, J. Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R. (2011). Survey methodology. Vol. 561. John Wiley & Sons. (Chapter 7)

Dillman, Don A., Smyth, J. D. and Christian, L. M. (2014) Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons. (Chapters 4, 5 and 6)

Couper, M. P.   (2008). Technology and the Survey Interview/Questionnaire in Envisioning the Survey Interview of the Future eds Conrad, F. G.  and Schober, M. F. (Chapter 5)

Engel, U. and Koeste, B.  (2015). Response effects and the cognitive involvement in answering survey questions in Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P. Improving survey methods: Lessons from recent research. Routledge.  (Chapter 4)

Tourangeau, R. and Yan T. (2007). Sensitive questions in surveys. Psychological bulletin. 133.5, pp: 859-883.

Kreuter, F., Presser, S. and Tourangeau, R. (2008). Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity. Public opinion quarterly 72.5, pp: 847-865.

Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity ,47.4: pp: 2025-2047.

Jann, B. .(2015). Asking sensitive questions: Overview and introduction in Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research. Routledge. (Chapter 9)
Day 4

Vardigan, M., Granda, P. and Hoetler, L.(2016). Documenting Survey data across the life cycle in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE Handbook of Survey methodology, Sage. (Chapter 29)

Dillman, Don A., Smyth, J. D. and Christian, L.  M. (2014). Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons. (Chapter 11)

Couper, M. (2008). Designing Effective Web Surveys. New York: Cambridge University Press. (Chapters 4 and 6)

Toepoel, V. (2016). Doing Surveys Online. SAGE.  (Chapters 8, 9,10, 11, 13 and 14)

Callegaro, M., Manfreda, K.L. and Vehovar, V. (2015). Web survey methodology. Sage. (Chapter 2.4, 5.3, 3, 4)

Lugtig, P. J.  and Toepoel, V,. (2016). 'The Use of PCs, Smartphones, and Tablets in a Probability-Based Panel Survey: Effects on Survey Measurement Error.' Social Science Computer Review, 34, 1, pp: 78–94.

Kaczmirek, L. (2015) Conducting web surveys: Overview and introduction in Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research. Routledge.  (Chapter 13)

Bethlehem, J. (2015) 'Web Surveys in official statistics' in Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P., eds. Improving survey methods: Lessons from recent research. Routledge (Chapter 14)

Engel, U., Scherpenzee A. (2015) 'Conducting access panels: Overview and introduction' in Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P., eds. Improving survey methods: Lessons from recent research. Routledge.

Knapp, H., Kirk, S. A., (2003). 'Using pencil and paper, internet and touch-tone phones for self-administered surveys: Does methodology matter?' Computers in Human Behavior, 19 (1), pp: 117–134
Day 5

Groves, R. M. Floyd, J. Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R. (2011). Survey methodology. Vol. 561. John Wiley & Sons. (Chapter 10)

Toepoel, V. (2016). Doing Surveys Online. SAGE. (Chapters 6 and 15) 

Callegaro, M., Manfreda, K. L., and Vehovar, V. (2015). Web survey methodology. Sage. (Chapters 5, 6 and 7)

Software Requirements

None

Hardware Requirements

Please bring your own laptop with an updated browser (flash player etc.) for online survey programming – Mac or Windows is fine.

Literature

As indicated in the day-to-day schedule.

Recommended Courses to Cover After this One

Winter and Summer School

Course in (survey) data analysis in SPSS, Stata or R