Klaus Krippendorff, Ph.D., the Gregory Bateson Professor of Communication, has co-authored the article (with Gabriela Trindade Perry from the Universidade Federal do Rio Grande do Sul), titled “On the reliability of identifying design moves in protocol analysis” in the journal Design Studies (35, 5: 612 – 635, August 2013).
Abstract: This paper discusses issues and ways of measuring the reliability of segmenting verbal protocols of design activity, a central focus of design research. Reliability is an important issue in distinguishing ‘design moves’. In the present study, seven students working for a master in design degree, one graduated designer and two professors segment a 30 min protocol of a product design process into design moves. The intra and inter reliability was calculated for these observers using alpha coefficients. Neither the students', designer's nor professors' segmentation reached the desired cut-off value of 0.8. This negative finding questions the clarity of existing conceptions and urges more concise definitions, better training of analysts, and formulating more decisive instructions.