INTRODUCTION

The quotation from Messick (1994) that opens this chapter eloquently lays out the essential narrative of educational assessment. Specifically, it captures how the design of tasks should be guided by the structure of the argument about learner competencies that one seeks to develop and eventually support with data from behaviors that activate these competencies. Moreover, the quote alludes to the fact that reasoning about learner competencies is a complex communicative act, which, like all communicative acts, requires the thoughtful integration of distinct pieces of information to construct a coherent, concise, and rhetorically effective argument.

Reasoning about learner competencies in educational assessment is both probabilistic and evidence based. It is probabilistic because it is concerned with developing an argument about one or more unobservable latent characteristics of learners. It is evidence based because the development of the argument must rely on data that are derived from aspects of examinees' observable behavior. However, such data provide only indirect information about latent characteristics, so the process of reasoning about learner competencies requires an agreed-upon framework about which data patterns constitute relevant evidence in what manner (Schum, 1994).