In the previous chapter we introduced Cohen’s (1960) classic kappa and discussed a number of issues associated with it and its use. In this chapter we describe how kappa can be used with each of the data types described in Chapter 4. The basic kappa computation remains basically unchanged, but some data types require matching and tallying procedures that differ from the classic Cohen’s kappa. Where procedures differ, we regard the result as a variant of Cohen’s kappa and, to make the distinction clear, provide a name for that variant. To provide orientation (and to let you select which sections of this chapter may best fit your circumstances), Figure 6.1 links data types with their appropriate kappa variant. The remaining sections of this chapter detail appropriate kappa procedures for each data type.

Event-Based Agreement: the Alignment Problem

When events are demarcated before coding, as turns in a transcript, the classic Cohen’s kappa described in the previous chapter is appropriate. However, when observes first segment the stream of behavior into events – that is, decide where the seams between events occur – and only then code the events, the situation is more complicated.