The classical content analysis and operationalization

A classical content analysis is a coding procedure in which extensively trained human coders analyze texts by means of a codebook that was designed by the researcher. Usually this goes as follows: Based on theoretical reasoning the researcher operationalizes the concept(s) he intends to measure and constructs a codebook (a scheme of coding rules). This step is sometimes facilitated by documents of which the researcher knows that they contain relevant information. These documents can provide inspiration on which arguments or words are often used. Eventually the codebook is applied to the texts from the actual coding sample. The researcher does not, however, analyze these texts himself. Extensively trained coders do that. It is of the upmost importance that they are well-trained, because for the measurement to be reliable, they all have to code the texts in the same way.

In most classical content analyses the unit of measurement (i.e. the smallest unit to which the codebook is applied) is not the whole text. The text is split up in different parts because the whole text contains too much information to code at once. Parts can be chapters, paragraphs, statements, sentences or words. The unit of measurement in our classical content analysis is the paragraph. The reason is that we focus on particular political arguments or ideas (we have, after all, defined populism as a thin ideology). Complete texts and chapters contain too many different arguments to code meaningfully, while a word usually does not provide us with enough information to infer an argument from it. The problem with the sentence as the unit of measurement is that one argument can be spread out over different sentences. A problem with statements is that it is very difficult for coders to extricate them from the text. Paragraphs, however, are objectively traceable distinctions between arguments. Moreover, paragraphs can relatively easily be separated from other paragraphs.

Earlier it was argued that the two dimensions of populism - people-centrism and anti-elitism – are inseparable because they are inextricably related to each other. For every single paragraph we have therefore determined whether both people-centrism and anti-elitism were present. If that was the case, the paragraph was selected as being populist. Eventually the percentage of populist paragraphs in every election manifesto was computed. We have measured people-centrism with the following question: “Do the authors of the manifesto refer to the people?” The people can be referred to in many different ways and often it is very unclear what populists exactly mean when they refer to the people (Canovan 2004). Coders have therefore looked at every possible reference to the people and their heartland. It did not matter whether this reference concerned, for instance, “citizens”, “our country”, “society” or “we” (as in “we the people”). Coders were explicitly instructed to interpret the broader context in deciding whether to code people-centrism or not. To help the coders, we have added an extensive list of words and combinations of words that could refer to it. As we have argued before, a paragraph was coded populist only when a reference to the people coincided with anti-elitism. Anti-elitism was measured by means of the question: “Do the authors of the manifesto criticize elites?” Anti-elitism is about criticism on political elites in general. Criticisms on a specific party or a particular politician are not general enough, and were therefore not coded. Because anti-elitism can be expressed in many different ways, coders were again instructed to interpret the context while coding. See Appendix A for the exact wordings of both questions.

We have to add one important comment about the indicators of populism. That people- centrism and anti-elitism are inextricably related in the thin ideology of populism does not mean that there should be a strong empirical correlation between them. After all, many political parties will centralize the people without being negative towards elites (Canovan labels this politician’s populism). On the other hand, however, most criticisms towards elites can be expected to go together with people-centrism. The reason is that negativity towards elites is almost always motivated by the argument that elites betray ordinary people. The classical content analysis confirmed this expectation: there is only a weak correlation between people-centrism and anti-elitism (r =.04, not significant at p<.05), whereas almost every anti­elitist paragraph also contains a reference to the people. This means that although conceptually people-centrism and anti-elitism are equally important, and both need to be present for populism to exist, empirically anti-elitism by itself is a pretty good indicator of populism. As a result of this, the computerized content analysis, which allows for little interpretation, will use mainly anti-elitist words to measure populism.

Дата добавления: 2015-07-10