ECPR Winter School
University of Bamberg, Bamberg
2 - 9 March 2018




WC104 - Survey Design

Instructor Details

Instructor Photo

Roula Nezi

Institution:
University of Surrey

Instructor Bio

Roula Nezi is a Senior Researcher at the GESIS-Leibniz Institute for the Social Sciences. Previously, she was a postdoctoral fellow at the University of Konstanz.

Her research interests lie in public opinion and political attitudes, political representation, and political accountability. She is also interested in political methodology, including comparative analysis and survey design.

In recent years, Roula has conducted research in a wide range of comparative European projects on political representation, political behaviour, and political elites. She is currently leading a project on the support of authoritarian beliefs and the vote for rightwing populist parties among young cohorts in Europe.

Twitter @roula_nezi


Course Dates and Times

Monday 5 to Friday 9 March 2018
09:00-12:30
15 hours over 5 days

Prerequisite Knowledge
  • This course focuses on survey research design, it is not a course in survey data analysis
  • No previous experience in survey research is required. A general background about survey research and/or experience in conducting surveys will be beneficial, however.
  • An introduction to programming an online survey in Lime Survey, Qualtrics, or Survey Monkey will be provided
Short Outline

Surveys ask a lot of people a lot of questions. They are arguably the most popular method of data collection as it is supposedly straight forward to run a survey. However, in practice, designing survey collecting good quality data is a lot more complex. This course provides in-depth guidelines to survey research design embedding and discussing the so-called Total Survey Error (TSE) Framework. The course will provide the theoretical background of the TSE and critically evaluate the full survey life cycle against this theory. We will discuss the core of idea of survey sampling and trade-offs between probability and non-probability samples; mode selection; question and questionnaire design, fieldwork implementation; and effects of response and non-response. Throughout the course’s focus especially lays on online survey designs. However, we will discuss online surveys against the benefits and challenges of other modes of data collection as well as mixed mode designs. The content will be applicable to surveys of individuals, households, and organisation

 

Tasks for ECTS Credits

  • Participants attending the course: 2 credits (pass/fail grade) The workload for the calculation of ECTS credits is based on the assumption that students attend classes and carry out the necessary reading and/or other work prior to, and after, classes.
  • Participants attending the course and completing one task (see below): 3 credits (to be graded)
  • Participants attending the course, and completing two tasks (see below): 4 credits (to be graded)
  1. Take-home or seated exam
  2. Take-home paper
Long Course Outline

Why survey design and the online lifecycle?

  • You want to ask a lot of people (a lot of) questions?
  • You have never heard of the Total Survey Error?
  • You always wanted a tailored design of some kind, but you feel your sample is limited?
  • You fear CAPI, CASI, CATI, and CAWI put you in a bad “mode”?
  • You frequently question questions and questionnaires?
  • You always run out of appropriate filters?
  • You think most things happen online anyway?
  • You cannot get the “grids” on the radio buttons?
  • You like to mix it up?

Focus

  • Theoretical Framework
    • This course provides in-depth guidelines to survey research design embedding and discussing the so-called Total Survey Error (TSE) Framework.
    • Critical discussion of all stages of the TSE
    • Critically evaluate the full survey life cycle against the TSE Framework.
  • Survey Design
    • Sampling designs and error including trade-offs between probability and non-probability samples
    • Modes of data collection including benefits and challenges of each mode
    • Question and questionnaire design
  • (Online) Survey Implementation
    • Fieldwork preparation, implementation, and monitoring
    • Pretesting
    • Response and non-response
    • Special focus on online implementation using Lime Survey, Qualtrics, Survey Monkey
  • Survey Documentation & Archiving
    • Response rates reporting
    • Preparing transparent documents for survey documentation
    • Dealing with archiving survey data

Structure

  • The course provides an applied and interactive learning approach
    • Participants will learn from some of the mistakes that existing surveys projects made
    • Participants are asked to make suggestions for improvement of these survey designs on the basis of what they have learned about each stage of the survey lifecycle
    • All participants are encouraged to work on their own survey projects during the week; they will be able to obtain detailed feedback on their work from the instructor

What you will take home…

  • Knowledge & understanding
    • A comprehensive knowledge of the full (online) survey life cycle
    • A systematic understanding of the Total Survey Error (TSE) framework
    • The ability to critically evaluate (online) survey designs
  • Skills
    • The ability to design a (online) survey
    • The technical skills to implement a (online) survey
    • The capability to manage and transparently document a (online) survey
  • Values & attitudes
    • A sound understanding of the relevance of (online) surveys in the social sciences
    • Recognition of the importance of integrity when designing, implementing, and managing survey as well as and reporting survey research
    • The ability to learn independently in response to self-reflection on your understanding of the principles and application of (online) survey research methodology

…and what you will not take home

  • As this is a course on survey research design, you will not get an in-depth introduction to survey data analysis. The instructor will be able to point you relevant software and introductory literature focusing on the special requirements of survey data analysis.
  • It is also not a course in statistics. Some more technical background will be covered when we discuss sampling procedure, however.
  • The course focuses on standardised quantitative interviews. It is not a course on conducting (semi-) structured qualitative interviews.
Day-to-Day Schedule

Day 
Topic 
Details 
1The survey life cycle; The Total Survey Error (TSE) framework
 
2Survey sampling; Modes of data collection
 
3The Psychology of the survey response; Question and questionnaire design
 
4Survey implementation 
5Survey documentation and archiving; The future of surveys; Wrap up
 
Day-to-Day Reading List

Day 
Readings 
1

Lyberg, L. E. and Weisberg, H. F. (2016). Total survey error: A paradigm for survey methodology in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE handbook of survey methodology. Sage. (Chapter 2)

Weisberg, H. F. (2005) The total survey error approach: A guide to the new science of survey research. University of Chicago Press. (Chapter 1 and 2)

Paul P. B. (2016). Total Survey Error Paradigm: Theory and practice in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Sage.  (Chapter 10) 

Groves, Robert M., and Lyberg, L. (2010). Total survey error: Past, present, and future. Public opinion quarterly, 74.5: pp: 849-879.

McNabb, D. E. (2013). Non sampling error in social surveys. Sage publications. (Chapter 3)

Groves, R.M. (2011). Three Eras of Survey Research, Public Opinion Quarterly, Volume 75, Issue 5, pp: 861–871.

Saris, W. E. and Gallhofer I. N. (2014) Design, evaluation, and analysis of questionnaires for survey research. John Wiley & Sons.  (Chapter 1)

2

Tille, Y. and  Matei, A. (2016). Basics of sampling for survey research n Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Sage. (Chapter 21)

Vehovar, V., Toepoel V. and Steinmetz S. (2016). Non-Probability Sampling in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Yang., eds. The SAGE handbook of survey methodology, Sage. (Chapter 22)

Toepoel, V. (2016). Doing Surveys Online. SAGE. (Chapters 4 and 5) 

Callegaro, M., Manfreda, K. L., and Vehovar V. (2015). Web survey methodology. Sage. (Chapter 2.2)

Groves, R. M., Floyd, J., Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R (2011). Survey methodology. Vol. 561. John Wiley & Sons. (Chapter 2 and 3)

De Leeuw E. D., Hox J. J. (2011). Internet surveys as part of a mixed-mode design, in Social and Behavioral Research and the Internet, eds Das M., Ester P., Kaczmirek L., eds. New York, NY: Routledge.  (Chapter 3)

Lynn, P. (2015). Coping with nonresponse: Overview and introduction in Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research. Routledge. (Chapter 26)

De Leeuw, D. (2005). "To mix or not to mix data collection modes in surveys. Journal of official statistics 21.2, pp: 233-255.

Stoop, I. (2016). Unit non-response in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y. eds. The SAGE handbook of survey methodology, Sage. (Chapter 27)

Edith de Leeuw and Jernal Berzelak (2016). Survey mode or survey modes? In Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y. eds. The Sage handbook of survey methodology. Sage (Chapter 11)

Dillman, Don A., Jolene D. S, and Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons. (Chapter 8)

3

Smyth, J. D.. (2016). Designing questions and questionnaire in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE Handbook of Survey methodology, Sage.  (Chapter 16)

Groves, R. M. Floyd, J. Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R. (2011). Survey methodology. Vol. 561. John Wiley & Sons. (Chapter 7)

Dillman, Don A., Smyth, J. D. and Christian, L. M. (2014) Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons. (Chapters 4, 5 and 6)

Couper, M. P.   (2008). Technology and the Survey Interview/Questionnaire in Envisioning the Survey Interview of the Future eds Conrad, F. G.  and Schober, M. F. (Chapter 5)

Engel, U. and Koeste, B.  (2015). Response effects and the cognitive involvement in answering survey questions in Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P. Improving survey methods: Lessons from recent research. Routledge.  (Chapter 4)

Tourangeau, R. and Yan T. (2007). Sensitive questions in surveys. Psychological bulletin. 133.5, pp: 859-883.

Kreuter, F., Presser, S. and Tourangeau, R. (2008). Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity. Public opinion quarterly 72.5, pp: 847-865.

Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity ,47.4: pp: 2025-2047.

Jann, B. .(2015). Asking sensitive questions: Overview and introduction in Engel, Uwe, et al., eds. Improving survey methods: Lessons from recent research. Routledge. (Chapter 9)

4

Vardigan, M., Granda, P. and Hoetler, L.(2016). Documenting Survey data across the life cycle in Wolf, C., Joye, D.,Tom W Smith, T. W. and Fu, Y., eds. The SAGE Handbook of Survey methodology, Sage. (Chapter 29)

Dillman, Don A., Smyth, J. D. and Christian, L.  M. (2014). Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons. (Chapter 11)

Couper, M. (2008). Designing Effective Web Surveys. New York: Cambridge University Press. (Chapters 4 and 6)

Toepoel, V. (2016). Doing Surveys Online. SAGE.  (Chapters 8, 9,10, 11, 13 and 14)

Callegaro, M., Manfreda, K.L. and Vehovar, V. (2015). Web survey methodology. Sage. (Chapter 2.4, 5.3, 3, 4)

Lugtig, P. J.  and Toepoel, V,. (2016). 'The Use of PCs, Smartphones, and Tablets in a Probability-Based Panel Survey: Effects on Survey Measurement Error.' Social Science Computer Review, 34, 1, pp: 78 – 94.

Kaczmirek ,L. (2015) Conducting web surveys: Overview and introduction in Engel, Uwe, et al., eds.Improving survey methods: Lessons from recent research. Routledge.  (Chapter 13)

Bethlehem ,J.  (2015) Web Surveys in official statistics in Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P., eds. Improving survey methods: Lessons from recent research. Routledge. (Chapter 14)

Engel, U., Scherpenzee A. (2015) Conducting access panels: Overview and introduction in Engel, U., Lynn, P., Scherpenzeel, A. and Sturgis, P., eds. Improving survey methods: Lessons from recent research. Routledge.

Knapp, H., Kirk, S. A., (2003). Using pencil and paper, internet and touch-tone phones for self-administered surveys: Does methodology matter? . Computers in Human Behavior, 19 (1), pp: 117-134

5

Groves, R. M. Floyd, J. Fowler, Jr., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R. (2011).Survey methodology. Vol. 561. John Wiley & Sons. (Chapter 10)

Toepoel, V. (2016). Doing Surveys Online. SAGE. (Chapters 6 and 15) 

Callegaro, M.,  Manfreda, K. L., and Vehovar, V. (2015). Web survey methodology. Sage. (Chapters 5, 6 and 7)

Software Requirements

None

Hardware Requirements

Participants should bring their own laptops with an updated browser (flashplayer etc.) for online survey programming. Mac and Windows laptops are fine.

Literature

See Readings as indicated.

The following other ECPR Methods School courses could be useful in combination with this one in a ‘training track .
Recommended Courses After

Winter and Summer School

Course in (survey) data analysis in SPSS, Stata or R

Additional Information

Disclaimer

This course description may be subject to subsequent adaptations (e.g. taking into account new developments in the field, participant demands, group size, etc). Registered participants will be informed in due time.

Note from the Academic Convenors

By registering for this course, you confirm that you possess the knowledge required to follow it. The instructor will not teach these prerequisite items. If in doubt, contact the instructor before registering.


Share this page
 

"History is past politics, and politics is past history" - E.A. Freeman


Back to top