Content Analysis

Content analysis is typically defined as a method whose goal is to summarize a body of information, often in the form of text, and to make inferences about the actor behind this body of information. This implies that content analysis can be seen as a data reduction method since its goal is to reduce the text material in to more manageable bits of information. As these manageable bits are often in the form of quantitative data (i.e. numbers), most often than not, researchers refer to content analysis as a ‘quantitative’ method. Content analysis can be also seen as a method for descriptive inference. Weber (1990, p. 9) for instance, defines content analysis as ‘a method that uses a set of procedures to make valid inferences from text’. The idea is that, by analysing the textual output of an actor, we can infer something about this actor. This conceptualization of content analysis implies that we can use it as a tool for measurement in the social sciences.

Since many social science concepts are not directly observable, content analysis can provide a useful method in which we can measure quantities of interest that are otherwise difficult to estimate. For instance, by content analysing the speeches of legislators, we can classify them as charismatic, populist, authoritarian, liberal, and so on. Similarly, by content analysing the content of newspaper editorials, we can infer whether the media in question were biased in favour of a particular candidate during an election campaign. This view of content analysis, however, assumes that we are employing the scientific method and therefore any content analysis application should be concerned with replicability, objectivity, reliability, validity and so on (Neuendorf 2002, pp. 10-15). As such, content analysis should not be confused with other approaches/methods in the ‘qualitative’ research tradition such as discourse analysis, rhetorical analysis, constructivism, ethnography and so on.

Using the chapters in Krippenforff (2004) and Neuendorf (2002), the course will introduce participants to the basic concepts and building blocks in content analysis designs. The course will focus on both  manual and computer-assisted content analysis and compare the respective approaches extensively. Specifically, for manual content analysis, the course will also look at the, often overlooked, distinction between the analysis of manifest content and judgemental coding, whether computer-assisted content analysis will cover a variety of methods (dictionaries, wordscores, wordfish scaling methods and so on). The course will look at relationship between reliability and validity and outline methods for estimating inter-coder reliability and validating the results produced by computer-assisted content analysis. In this respect, the course will use many examples to illustrate the promises as well as the pitfalls of content analysis in various applications across the social sciences (e.g. sentiment analysis of the press, frames analysis of social movements, estimating the positions of political actors, agenda-setting in the EU).

To give a more extensive indication of the the issues that will be discussed in the course, you can consider the following questions:

• Coding scheme (What are the theoretical underpinnings of the coding scheme? How are the categories selected and operationalized? What are the coding units? How is coding performed? Is our coding scheme valid?)
• Selection of documents (What guides the selection of texts? Are texts sufficiently comparable? Are our documents valid and reliable indicators of the quantities of interest? How can we acquire and process text for computer-assisted content analysis?)
• Aggregation (Are texts coded by different coders? If so, how are their results aggregated? If not, how can we ensure inter-coder reliability? What statistical measures can be used to estimate inter-coder reliability?)
• Scaling (Are we estimating the quantities of interest directly? If not, how do we scale data in order to estimate the quantities of interest? Is our scaling valid and reliable?)

The format of the course will be a mixture of lectures, seminars, participant assignments and presentations. The lectures will outline the building blocks, challenges and trade-offs in content analysis. Participants will need to complete three assignments that will be discussed and extended during seminars. Finally, participants will have the opportunity to present their own content analysis project and receive feedback from the instructor and other participants.

Participants are expected to be familiar with basic statistical concepts such as measures of central tendency (mean, median), dispersion (standard deviation), tests of association (Pearson’s r) and inference (χ2, t-test). These material are covered in the first few chapters of introductory statistics or data analysis textbooks. A useful example is Pollock P.H. III, The Essentials of Political Analysis, fourth edition (Washington, DC: CQ Press, 2012), Chapters 2, 3, 6, and 7. Some familiarity with SPSS, and Stata or R statistical software is also desirable.