Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-848d4c4894-8kt4b Total loading time: 0 Render date: 2024-06-23T09:33:27.213Z Has data issue: false hasContentIssue false

2 - Evaluating Cognitive Models in Large-Scale Educational Assessments

Published online by Cambridge University Press:  05 June 2012

Jacqueline P. Leighton  and
Mark J. Gierl
Jacqueline P. Leighton
Affiliation:
University of Alberta
Mark J. Gierl
Affiliation:
University of Alberta
Get access

Summary

In educational assessment, an idea whose time has come is that tests can be designed according to advances in the learning sciences. Currently, researchers and practitioners are studying how learning science outcomes can inform the design, development, and use of large-scale educational assessments. To date, however, there are few examples of large-scale educational tests designed from a learning science perspective and, hence, few empirical studies exist to evaluate the strengths and weaknesses of this approach. Moreover, developing the first generation of assessments using outcomes from the learning sciences is hampered by the limited information available on the types of cognitive models that can facilitate the design, validation, and administration of educational tests. Yet the role of cognitive models is paramount to the success of this endeavor. The National Research Council (NRC, 2001) asserted in their seminal publication, Knowing What Students Know: The Science and Design of Educational Assessments, that cognitive models have a foundational role in educational assessment that is currently poorly developed and under-valued:

A model of cognition and learning, or a description of how people represent knowledge and develop competence in a subject domain, is a cornerstone of the assessment development enterprise. Unfortunately, the model of learning is not made explicit in most assessment development efforts, is not empirically derived, and/or is impoverished relative to what it could be. (p. 176)

Type
Chapter
Information
The Learning Sciences in Educational Assessment
The Role of Cognitive Models
 , pp. 45 - 70
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

,Alberta Education (2007). The Alberta K–9 mathematics program of studies with achievement indicators. Edmonton, AB: Alberta Education.Google Scholar
,Alberta Education (2008). Alberta provincial achievement testing, subject bulletin 2008–2009: Grade 3 mathematics. Edmonton, AB: Alberta Education.Google Scholar
,American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington DC: Author.Google Scholar
Bloom, B., Englehart, M. Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: Longmans, Green.Google Scholar
Brown, J.S. & Burton, R.R. (1978). Diagnostic models for procedural bugs in basic mathematics skills. Cognitive Science, 2, 155–192.CrossRefGoogle Scholar
Busemeyer, J.R. & Diederich, A. (2010). Cognitive modeling. Thousand Oaks, CA: Sage.Google Scholar
Case, R. (1992). The mind's staircase: Exploring the conceptual underpinnings of children's thought and knowledge. Hillsdale, NJ: Erlbaum.Google Scholar
Case, R. (1996). Reconceptualizing the nature of children's conceptual structures and their development in middle childhood. Monographs of the Society for Research in Child Development, 61 (1–26), Serial 246.CrossRefGoogle ScholarPubMed
Case, R. & Griffin, S., (1990). Child cognitive development: The role of central conceptual structures in the development of scientific and social thought. In Hauert, E. A. (Ed.), Developmental psychology: Cognitive, perceptuo-motor, and neurological perspectives (pp. 193–230). North-Holland: Elsevier.Google Scholar
Chi, M.T.H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6, 271–315.CrossRefGoogle Scholar
Dawson, M.R.W. (1998). Understanding cognitive science. Blackwell.Google Scholar
Dawson, M.R.W. (2004). Minds and machines: Connectionism and psychological modeling. Malden, MA: Blackwell.CrossRefGoogle Scholar
Downing, S.M. & Haladyna, T.M. (2006). Handbook of test development. Mahwah, NJ: Erlbaum.Google Scholar
Ericsson, K.A. & Simon, H.A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: The MIT Press.Google Scholar
Gierl, M.J. (1997). Comparing the cognitive representations of test developers and students on a mathematics achievement test using Bloom's taxonomy. Journal of Educational Research, 91, 26–32.CrossRefGoogle Scholar
Gierl, M.J., Alves, C., Roberts, M., & Gotzmann, A. (2009, April). Using judgments from content specialists to develop cognitive models for diagnostic assessments. In Gorin, J. (Chair), How to build a cognitive model for educational assessments. Symposium conducted at the meeting of the National Council on Measurement in Education, San Diego, CA.Google Scholar
Huff, K. & Goodman, D. (2007). The demand for cognitive diagnostic assessment. In Leighton, J. P. & Gierl, M. J. (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 19–60). New York: NY: Cambridge University Press.Google Scholar
Griffin, S. (2002). The development of math competence in the preschool and early school years: Cognitive foundations and instructional strategies. In Royer, J. (Ed.), Mathematical cognition (pp. 1–32). Greenwich, CT: Information Age Publishing.Google Scholar
Griffin, S. (2004). Building number sense with number worlds: A mathematics program for young children. Early Childhood Research Quarterly, 19, 173–180.CrossRefGoogle Scholar
Griffin, S. & Case, R. (1997). Re-thinking the primary school math curriculum: An approach based on cognitive science. Issues in Education, 3, 1–49.Google Scholar
Griffin, S., Case, R., & Sandieson, R. (1992). Synchrony and asynchrony in the acquisition of children's everyday mathematical knowledge. In Case, R. (Ed.), The mind's staircase: Exploring the conceptual underpinnings of children's thought and knowledge (pp. 75–97). Hillsdale, NJ: Erlbaum.Google Scholar
Griffin, S., Case, R., & Siegler, R. (1994). Rightstart: Providing the central conceptual prerequisites for first formal learning of arithmetic to students at-risk for school failure. In McGilly, K. (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp.24–49). Cambridge, MA: Bradford Books MIT Press.Google Scholar
Leighton, J.P. (2004). Avoiding misconceptions, misuse, and missed opportunities: The collection of verbal reports in educational achievement testing. Educational Measurement: Issues and Practice, 4, 1–10.Google Scholar
Leighton, J.P. & Gierl, M.J. (2007a). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees' thinking processes. Educational Measurement: Issues and Practice, 26, 3–16.CrossRefGoogle Scholar
Leighton, J.P. & Gierl, M.J. (2007b). Verbal reports as data for cognitive diagnostic assessment. In Leighton, J. P. & Gierl, M. J. (Eds.), Cognitive diagnostic assessment for education: Theory and applications. (pp. 146–172). Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
Leighton, J.P. & Gokiert, R.J. (2008). Identifying test item misalignment using verbal reports of item misinterpretation and uncertainty. Educational Assessment, 13, 215–242.CrossRefGoogle Scholar
Lohman, D. & Nichols, P. (1990). Training spatial abilities: Effects of practice on rotation and synthesis tasks. Learning and Individual Differences, 2, 67–93.CrossRefGoogle Scholar
Marr, D. (1982). Vision. San Francisco, CA: W. H. Freeman.Google Scholar
,National Research Council (2001). Knowing what students know: The science and design of educational assessment. Committee on the Foundations of Assessment. Pellegrino, J., Chudowsky, N., and Glaser, R. (Eds.). Board on Testing and Assessment, Center for Education. Washington, DC: National Academy Press.Google Scholar
Newell, A. & Simon, H.A. (1972). Human problem solving. NJ: Prentice Hall.Google Scholar
Norris, S.P., Leighton, J.P., & Phillips, L.M. (2004). What is at stake in knowing the content and capabilities of children's minds? A case for basing high stakes tests on cognitive models. Theory and Research in Education, 2, 283–308.Google Scholar
Okamoto, Y. & Case, R. (1996). Exploring the microstructure of children's central conceptual structures in the domain of number. Monographs of the Society for Research in Child Development, 61 (27–58), Serial 246.CrossRefGoogle Scholar
Poggio, A., Clayton, D.B., Glasnapp, D., Poggio, J., Haack, P., & Thomas, J. (April, 2005). Revisiting the item format question: Can the multiple choice format meet the demand for monitoring higher-order skills? Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
Schmeiser, C.B. & Welch, C.J. (2006). Test development. In Brennan, R. L. (Ed.), Educational measurement (4th edition, pp. 307–353). Westport, CT: Praeger.Google Scholar
Snow, R.E. & Lohman, D.F. (1989). Implications of cognitive psychology for educational measurement. In Linn, R. L. (Ed.), Educational measurement (3rd ed., 263–331). New York: American Council on Education, Macmillan.Google Scholar
Taylor, K.L. & Dionne, J-P. (2000). Accessing problem-solving strategy knowledge: The complementary use of concurrent verbal protocols and retrospective debriefing. Journal of Educational Psychology, 92, 413–425.CrossRefGoogle Scholar
Wellman, H.M. & Lagattuta, K.H. (2004). Theory of mind for learning and teaching: The nature and role of explanation. Cognitive Development, 19, 479–497.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×