Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-8448b6f56d-tj2md Total loading time: 0 Render date: 2024-04-25T01:36:48.413Z Has data issue: false hasContentIssue false

6 - Putting It All Together: Cognitive Models to Inform the Design and Development of Large-Scale Educational Assessment

Published online by Cambridge University Press:  05 June 2012

Jacqueline P. Leighton  and
Mark J. Gierl
Jacqueline P. Leighton
Affiliation:
University of Alberta
Mark J. Gierl
Affiliation:
University of Alberta
Get access

Summary

In this penultimate chapter, we summarize our impetus for identifying and evaluating diagrammatic cognitive models in reading, science, and mathematics and offer some conclusions about where we go from here. Borrowing the definition from Leighton and Gierl (2007a), a cognitive model was defined as a “simplified description of human problem solving on standardized educational tasks, which helps to characterize the knowledge and skills students at different levels of learning have acquired and to facilitate the explanation and prediction of students' performance” (p. 6). In Chapter 1, we indicated that large-scale educational tests, redesigned and redeveloped from cognitive models in the learning sciences, may offer enhanced information (test-based inferences) about student problem solving and thinking. This enhanced information may help remediate the relatively low test performance of many students, including U.S. students, who are struggling to learn and demonstrate knowledge in core domains. In Chapter 1, we also presented accepted knowledge and principles from the learning sciences about the nature of thinking, learning, and performance to set the stage for what may be required for redesigning and redeveloping large-scale assessments. Illustrative empirical studies in the field of educational measurement were described to demonstrate attempts at redesigning and redeveloping educational assessments based on the learning sciences. Chapters 2, 3, 4, and 5 presented our criteria for evaluating cognitive models and also offered examples of diagrammatic cognitive models in reading, science, and mathematics that have garnered substantial empirical support.

Type
Chapter
Information
The Learning Sciences in Educational Assessment
The Role of Cognitive Models
 , pp. 197 - 233
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adds, J., Larkcom, E., Miller, R., & Sutton, R. (1999). Tools, techniques and assessment in biology: A course guide for students and teachers. Chelteham, UK: Nelson Thornes Publishing.Google Scholar
Ainsworth, S. & Loizou, A. (2003). The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27, 669−681.CrossRefGoogle Scholar
Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebière, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111, 1036–1060.CrossRefGoogle ScholarPubMed
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., & Wittrock, C. (Eds.). (2001). A Taxonomy for learning, teaching, and assessing – A revision of Bloom's taxonomy of educational objectives. Addison Wesley Longman.
,Alberta Education (2009). Information bulletin: Biology 30, 2009–2010 diploma examinations program. Author. Retrieved May 21, 2010 from http://education. alberta.ca/media/1116889/10_b30_inforbulle_2009–2010_print.pdf.
Bandura, A. (2006). Toward a psychology of human agency. Perspectives on Psychological Science, 1, 164–180.CrossRefGoogle Scholar
Bennett, R.E. & Gitomer, D.H. (2008). Transforming K-12 assessment: Integrating accountability testing, formative assessment, and professional support. ETS Research Memorandum-08–13, 1–30. Princeton, NJ: Educational Testing Service.Google Scholar
Bloom, B., Englehart, M. Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: Longmans, Green.Google Scholar
Britton, B.K. & Gülgöz, S. (1991). Using Kinstch's computational model to improve instructional text: effects of repairing inference calls on recall and cognitive structures, Journal of Educational Psychology, 83, 329–345.CrossRefGoogle Scholar
Chi, M.T.H., Glaser, R., & Farr, M. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: Erlbaum.
Embretson, S.E. (2002). Generating abstract reasoning items with cognitive theory. In Irvine, S.H. & Kyllonen, P.C. (Eds.), Item generation for test development (pp. 219–250). Mahwah, NJ: Erlbaum.Google Scholar
Ericsson, K.A. (2009). (Ed.). Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments. New York, New York: Cambridge University Press.CrossRef
Everson, H. (2008). The diagnostic challenge in education: A review of Cognitive Diagnostic Assessment: Theory and applications. PsycCRITIQUES, 53(29), no pagination specified.
Ferrara, S. & DeMauro, G.E. (2006). Standardized assessment of individual achievement in K-12. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 579–621). Westport, CT: National Council on Measurement in Education and American Council on Education.Google Scholar
Ferstl, E.C. & Kintsch, W. (1998). Learning from text: Structural knowledge assessment in the study of discourse comprehension. In Goldman, S.R. & Oostendorp, H. (Eds.), The constructions of mental representations during reading, (pp. 247–277). Mahwah, NJ: Erlbaum.Google Scholar
Gierl, M.J. & Leighton, J.P. (2007). Directions for future research in cognitive diagnostic assessment (pp. 341–351). In Leighton, J.P. & Gierl, M.J. (Eds.), Cognitive diagnostic assessment for education: Theory and applications. Cambridge, UK: Cambridge University Press.Google Scholar
Graesser, A.C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101, 371–395.CrossRefGoogle ScholarPubMed
Greeno, J.G. (1983). Forms of understanding in mathematical problem solving. In Paris, S.G., Olson, G.M., Stevenson, H.W. (Eds.), Learning and motivation in the classroom (pp. 83–111). Hillsdale, NJ: Erlbaum.Google Scholar
Hammond, D. (2003). The science of synthesis. Boulder, CO: University of Colorado Press.Google Scholar
Jackson, N.E. (2005). Are university students' component reading skills related to their text comprehension and academic achievement?Learning and Individual Differences, 15, 113–139.CrossRefGoogle Scholar
Kilpatrick, J., Swafford, J. & Findell, B. (Ed.). (2001). Adding It Up: Helping Children Learn Mathematics. Washington, DC, USA: National Academies Press.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge, UK: Cambridge University Press.Google Scholar
Kintsch, W. & Kintsch, E. (2005). Comprehension. In Paris, S.G. & Stahl, S.A. (Eds.), Children's reading comprehension and assessment (pp. 71–92). Mahwah, NJ: Erlbaum.Google Scholar
Klahr, D. & Dunbar, K. (1988). Dual search space during scientific reasoning. Cognitive Science, 12, 1–48.CrossRefGoogle Scholar
Koedinger, K.R. & Anderson, J.R. (1990). Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science, 14, 511–550.CrossRefGoogle Scholar
Kuhn, D. (2001). How do people know?Psychological Science, 12, 1–8.CrossRefGoogle ScholarPubMed
Kuhn, D. (2005). Education for thinking. Cambridge, MA: Harvard University Press.Google ScholarPubMed
Larkin, J. & Simon, H. (1987). Why a diagram is (sometimes) worth 10,000 words. Cognitive Science, 11, 65–99.CrossRefGoogle Scholar
Leighton, J.P., Cui, Y., & Cor, M.K. (2009). Testing expert-based and student-based cognitive models: An application of the attribute hierarchy method and hierarchical consistency index. Applied Measurement in Education, 22, 229–254.CrossRefGoogle Scholar
Leighton, J.P. & Gierl, M.J. (2007a). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees' thinking processes. Educational Measurement: Issues and Practice, 26, 3–16.CrossRefGoogle Scholar
Leighton, J.P. & Gierl, M.J. (2007b). (Eds.). Cognitive diagnostic assessment for education: Theories and applications. New York: Cambridge University Press.CrossRefGoogle Scholar
Mislevy, R.J. (2006). Cognitive psychology and educational assessment. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 257–305). Westport, CT: National Council on Measurement in Education and American Council on Education.Google Scholar
Mislevy, R.J., Behrens, J.T., Bennett, R.E., Demark, S.F., Frezzo, D.C., Levy, R., Robinson, D.H., Rutstein, D.W., Shute, V.J., Stanley, K., & Winters, F.I. (2010). On the roles of external knowledge representations in assessment design. Journal of Technology, Learning, and Assessment, 8(2). http://escholarship.bc.edu/jtla/vol8/2.Google Scholar
,National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.Google Scholar
Newell, A. (1973). You can't play 20 questions with nature and win: Projective comments on the papers of this symposium. In Chase, W. G. (Ed.), Visual information processing (pp. 283–310). New York: Academic Press.Google Scholar
Ryle, G. (1949). The concept of mind. Chicago, IL: University of Chicago Press.Google Scholar
Schmeiser, C.B. & Welch, C.J. (2006). Test development. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 307–353). Westport, CT: National Council on Measurement in Education and American Council on Education.Google Scholar
Stanovich, K.E. (2009). What intelligence tests miss: The psychology of rational thought. New Haven, CT: Yale University Press.Google Scholar
Tubau, E. (2008). Enhancing probabilistic reasoning: The role of causal graphs, statistical format and numerical skills. Learning and Individual Differences, 18, 187–196.CrossRefGoogle Scholar
Tufte, E.R. (1983). The visual display of quantitative information. Cheshire, Connecticut: Graphics Press.Google Scholar
Tversky, B. (2002). Some ways that graphics communicate. In Allen, N. (Editor), Words and images: New steps in an old dance (pp. 57–74). Westport, CT: Ablex.Google Scholar
Bertalanffy, L. (1955). The quest for a general system theory. Main Currents in Modern Thought, 11 (4), 75–83.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×