Chapter Overview and Goals
In 2001, a report was issued by the National Research Council (NRC) entitled “Knowing What Students Know: The Science and Design of Educational Assessment” (Pellegrino, Chudowsky, & Glaser, 2001). The goal was to evaluate the state of research and theory on educational assessment and establish the scientific foundations for their design and use. As argued in that volume, many of the debates that surround educational assessment emanate from a failure to understand its fundamental nature, including the ways in which theories and models (1) of learning and knowing and (2) of measurement and statistical inference interact with and influence processes of assessment design, use, and interpretation. In this chapter we review some of the key issues regarding educational assessment raised in that report as well as examples from science, technology, engineering, and mathematics (STEM) education and educational research contexts. Our goal in explicating current understanding of the science and design of educational assessment and its applications to STEM education is to provide background knowledge that supports the effective design and use of assessment in engineering education and sharpens the focus of engineering education R&D.
In the first section we briefly introduce some key ideas critical to understanding educational assessment. This includes consideration of formal and informal uses of assessment and some conceptual issues associated with assessment design, interpretation, and use. In the second section we discuss three related conceptual frameworks about assessment that should be considered by anyone using assessment for instructional or research purposes. These include (1) assessment as a process of reasoning from evidence, (2) the use of an evidence-centered design process to develop and interpret assessments, and (3) centrality of the concept of validity in the design, use, and interpretation of any assessment. In the third section we turn to a discussion of concepts of measurement as applied to assessment and the role of statistical inference. This includes the assumptions underlying different types of psychometric models used to estimate student proficiency. The fourth section then presents applications of key ideas from the preceding sections in the form of illustrative examples of assessments used in engineering education research. In the final section we close by briefly considering the significance of a careful and thoughtful approach to assessment design, use, and interpretation in engineering education research.