Computer-based content analysis and operationalization

The second technique with which we measured the degree of populism of political parties was a computer-based content analysis. The main difference with a classical content analysis is that texts are not treated as discourses to be understood and interpreted, but rather as data in the form of words (Laver et al. 2003; Laver and Garry 2000). The researcher designs a dictionary containing different categories, depending on the information he or she wants to extract from the texts. In a next step, words are allocated to the different dictionary categories. Just as in the case of the classical content analysis, the construction of the dictionary is based on theoretical reasoning and can be facilitated by exploring documents of which the researcher knows that they contain relevant information. In a second stage, the dictionary is applied to the actual documents from the sample. The dictionary is the equivalent of the codebook in the classical content analysis. The training of the coders can be skipped: the analysis is, after all, conducted by the computer.

One of the major differences with classical content analysis is that words, and not entire documents, chapters, paragraphs, statements or sentences are the units of measurement. Although skeptics might argue that the same word can have different meanings depending on the context, in many cases it is possible to code words unambiguously (Laver and Garry 2000). The word “taxes”, for instance, might be associated with cutting taxes but it can equally be used to indicate that a party wants to increase taxes. In practice, however, this latter meaning will hardly be found in party programs, and the word taxes is hence a good indicator for the category “reduce state involvement in the economy”, identifying socio-economic rightist parties.

In our dictionary we had to be very selective with our choice of words. In the case of anti­elitism, we have only selected words of which it was to be expected that they refer to anti­elitism only. Sometimes this turned out to be very difficult. Not every word that could possibly refer to anti-elitism does indeed always refer to it. At the same time, many instances of anti-elitism can easily be missed because it is impossible to formulate every word that could refer to anti-elitism beforehand. This problem is perhaps even more acute for people- centrism. The computer cannot distinguish between different references to the people. It can for instance not decide whether the word “people” is meant in a populist way or not. An additional, and maybe even bigger problem with people-centrism is that every political party centralizes the people to a certain extent. The computer cannot detect when people-centrism is related to anti-elitism and when it is not. We have found an empirical solution for this problem. In the previous section we have given theoretical and empirical arguments why anti­elitism by itself is a pretty good indicator of populism. For this reason we have chosen to include almost only anti-elitist dictionary words in the computer-based analysis.

It must be kept in mind that the dictionary for the party manifestos in the different countries were not entirely similar. Leaving aside that a translation can never be perfect, the populist language also differs from one context to another. For instance, while populists in the Netherlands sometimes talk about “regenten” to express anti-elitism, British populists do not talk about “regents” in this context. We used the open software program Yoshikoder to measure the percentage of our dictionary words that were identified in the different election manifestos. For a complete overview of our dictionary, see Appendix B.

Our expectation is that due to coders’ ability to take into account contextual matters, the classical content analysis will yield more valid results than the computer-based method which is not sensitive to context. Yet, for the computer analysis this disadvantage in terms of validity is an advantage in terms of reliability; the absence of human coders will lead to less interpretation and therefore to more reliable measurements.

Дата добавления: 2015-07-10