New publication by Prof. Klaus Krippendorff

Klaus Krippendorff, Ph.D.

Klaus Krippendorff, Ph.D., the Gregory Bateson Professor for Cybernetics, Language, and Culture just published an article titled “Agreement and Information in the Reliability of Coding” in Communication Measures and Methods (5,2: 93-112, 2011).

 

 

In this article, he resolved what reliability statisticians have referred to as “the paradox of high agreement measuring low in reliability.” He identified the paradox as arising out of privileging percent measures over chance corrected measures and argued that all data need to possess two qualities. (1) They need to represent phenomena that are inter-subjectively reproducible, and (2) they need to exhibit variation in order to provide enough information to answer a research question, for example, to support desirable inferences from texts, establish a correlation or select one hypothesis from several. Researchers are misled believing that measuring inter-observer agreement could assess both qualities. By shifting the attention from measuring agreement to measuring the required qualities of data, he proposed a measure of information as a companion of measures of reliability.