Book contents
- The Cambridge Handbook of Motivation and Learning
- The Cambridge Handbook of Motivation and Learning
- Copyright page
- Epigraph
- Contents
- Figures
- Tables
- Contributors
- Foreword
- Acknowledgments
- Introduction Motivation and Its Relation to Learning
- Part I The Self and Its Impact
- Part II Rewards, Incentives, and Choice
- Part III Interest and Internal Motivation
- Part IV Curiosity and Boredom
- Part V Goals and Values
- Part VI Methods, Measures, and Perspective
- 26 Motivation and Learning
- 27 Addressing the Challenge of Measuring Student Engagement
- 28 Measuring Motivation in Educational Settings
- 29 An Integrative Perspective for Studying Motivation in Relation to Engagement and Learning
- 30 Affordances and Attention
- Index
- References
28 - Measuring Motivation in Educational Settings
A Case for Pragmatic Measurement
from Part VI - Methods, Measures, and Perspective
Published online by Cambridge University Press: 15 February 2019
Book contents
- The Cambridge Handbook of Motivation and Learning
- The Cambridge Handbook of Motivation and Learning
- Copyright page
- Epigraph
- Contents
- Figures
- Tables
- Contributors
- Foreword
- Acknowledgments
- Introduction Motivation and Its Relation to Learning
- Part I The Self and Its Impact
- Part II Rewards, Incentives, and Choice
- Part III Interest and Internal Motivation
- Part IV Curiosity and Boredom
- Part V Goals and Values
- Part VI Methods, Measures, and Perspective
- 26 Motivation and Learning
- 27 Addressing the Challenge of Measuring Student Engagement
- 28 Measuring Motivation in Educational Settings
- 29 An Integrative Perspective for Studying Motivation in Relation to Engagement and Learning
- 30 Affordances and Attention
- Index
- References
Summary
A reality of conducting motivation research in educational settings is that there are tensions between technical standards of research and practical constraints of a given situation. Although adherence to standards for high-quality measurement is critical for good-quality data to be collected, measurement also requires substantial resources to ensure quality. In the current chapter, we discuss several examples of real data collected in different educational settings using a pragmatic measurement framework. Based on contemporary measurement perspectives, the pragmatic measurement framework emphasizes building evidence-based arguments to support the use and interpretation of a measure. Example 1 explores college students’ attitudes toward general education classes. Example 2 tracks students’ classroom motivation over several time points. Example 3 assesses experimental differences from an online motivation intervention. Together the three examples cover a range of possible research questions that researchers may encounter. As a whole, this chapter demonstrates that important and meaningful insights can be gained using pragmatic approaches to measurement. Importantly, we discuss the trade-offs that researchers or other measure users must consider when adopting a pragmatic approach to measurement.
- Type
- Chapter
- Information
- The Cambridge Handbook of Motivation and Learning , pp. 713 - 738Publisher: Cambridge University PressPrint publication year: 2019
References
AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.Google Scholar
Atkinson, J. W. (1964). An introduction to motivation. Princeton, NJ: D. Van Nostrand Company.Google Scholar
Bandura, A. & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. Journal of Personality and Social Psychology, 41(3), 586–98. doi: 10.1037/0022-3514.41.3.586.Google Scholar
Barron, K. E., Grays, M., & Hulleman, C. S. (2014). Assessing motivation in general education. Poster presented at the Annual Meeting of the American Educational Research Association, Philadelphia, PA.Google Scholar
Barron, K. E. & Hulleman, C. S. (2015). Expectancy-value-cost model of motivation. In J. D. Wright (Ed.) International encyclopedia of the social & behavioral sciences (Vol. 8, pp. 503–9). Elsevier. doi: 10.1016/B978-0-08-097086-8.26099-6.Google Scholar
Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17(2–3), 62–87. doi: 10.1080/10627197.2012.715014.Google Scholar
Bill and Melinda Gates Foundation. (2013). Ensuring fair and reliable measures of effective teaching. Retrieved from www.edweek.org/media/17teach-met1.pdf.Google Scholar
Bowman, N. A. (2010). Can 1st-year college students accurately report their learning and development? American Educational Research Journal, 47(2), 466–96. doi: 10.3102/0002831209353595.Google Scholar
Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Multivariate applications series. New York, NY: Routledge.Google Scholar
Cronbach, L. J. & Meehl, P. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.Google Scholar
Duckworth, A. L. & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–51. doi: 10.3102/0013189X15584327.Google Scholar
Eccles, J. S. (1983). Expectancies, values, and academic behaviors. In Spence, J. T. (Ed.), Achievement-related motives and behaviors (pp. 119–46). San Francisco, CA: W. H. Freeman.Google Scholar
Eccles, J. S., Adler, T., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1983). Expectancies, values, and academic behaviors. In Spence, J. T. (Ed.), Achievement and achievement motivation (pp. 75–146). San Francisco, CA: W. H. Freeman.Google Scholar
Embretson, S. E. & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Fredricks, J. A. & McColsky, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Christenson, S. L., Reschly, A. L., & Wylie, C. (Eds.), Handbook of research on student engagement (pp. 763–82). New York, NY: Springer. doi: 10.1007/978-1-4614-2018-7_37.Google Scholar
Gaspard, H., Dicke, A., Flunger, B., Brisson, B. M., Häfner, I., Nagengast, B., & Trautwein, U. (2015). Fostering adolescents’ value beliefs for mathematics with a relevance intervention in the classroom. Developmental Psychology, 51(9), 1226–40. doi: 10.1037/dev0000028.Google Scholar
Goldstein, J. & Flake, J. K. (2016). Towards a framework for the validation of early childhood assessment systems. Educational Assessment, Evaluation and Accountability, 28(3), 273–93. doi: 10.1007/s11092-015-9231-8.Google Scholar
Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106(2), 375–89. doi: 10.1037/a0034679.Google Scholar
Hektner, J. M. & Csikszentmihalyi, M. (1996). A longitudinal exploration of flow and intrinsic motivation in adolescents. In Annual Meeting of the American Educational Research Association, NYC, New York.Google Scholar
Hidi, S. & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–27. doi: 10.1207/s15326985ep4102_4.Google Scholar
Holstermann, N., Ainley, M., Grube, D., Roick, T., & Bögeholz, S. (2012). The specific relationship between disgust and interest: Relevance during biology class dissections and gender differences. Learning and Instruction, 22(3), 185–92. doi: 10.1016/j.learninstruc.2011.10.005.Google Scholar
Hulleman, C. S. (2007). The role of utility value in the development of interest and achievement. University of Wisconsin-Madison. Retrieved from http://files.eric.ed.gov/fulltext/ED498264.pdf.Google Scholar
Hulleman, C. S., Barron, K. E., Kosovich, J. J., & Lazowski, R. A. (2016). Expectancy-value models of achievement motivation in education. In Lipnevich, A. A., Preckel, F., & Robers, R. D. (Eds.), Psychosocial skills and school systems in the twenty-first century: Theory, research, and applications. (pp. 241–78). Basel, Switzerland: Springer. doi: 10.1007/978-3-319-28606-8.Google Scholar
Hulleman, C. S., Godes, O., Hendricks, B. L., & Harackiewicz, J. M. (2010). Enhancing interest and performance with a utility value intervention. Journal of Educational Psychology 102, 880–95. doi: 10.1037/a0019506.Google Scholar
Hulleman, C. S. & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326(5958), 1410–12. doi: 10.1126/science.1177067.Google Scholar
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2016). Making connections: Replicating and extending the utility value intervention in the classroom. Journal of Educational Psychology. 109(3), 387–404. doi: 10.1037/edu0000146.Google Scholar
Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-analytic review of achievement goal measures: Different labels for the same constructs or different constructs with similar labels? Psychological Bulletin, 136(3), 422–49. doi: 10.1037/a0018947.Google Scholar
Jacobs, J. E., Lanza, S., Osgood, D. W., Eccles, J. S., & Wigfield, A. (2002). Changes in children's self-competence and values: Gender and domain differences across grades one through twelve. Child Development, 73(2), 509–27. doi: 10.1111/1467-8624.00421.Google Scholar
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–35.Google Scholar
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. doi: 10.1111/jedm.12000.Google Scholar
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: The Guilford Press.Google Scholar
Kosovich, J. J. (2017). Pragmatic measurement for education science: A method-substance synergy of validation and motivation. Charolottesville, VA: University of Virginia.Google Scholar
Kosovich, J. J., Flake, J. K., & Hulleman, C. S. (2017). Short-term motivation trajectories: A parallel process model of expectancy-value. Contemporary Educational Psychology. 49(3), 130–139. doi: 10.1016/j.cedpsych.2017.01.004.Google Scholar
Kosovich, J. J., Hulleman, C. S., Barron, K. E., & Getty, S. (2015). A practical measure of student motivation: Establishing validity evidence for the expectancy-value-cost scale in middle school. The Journal of Early Adolescence, 35(5–6), 790–816. doi: 10.1177/0272431614556890.Google Scholar
Kosovich, J. J., Hulleman, C. S., & Flake, J. K. (2017). Practical measurement: An argument-based approach to exploring alternative psychometric validity evidence. Poster presented at the annual Society for Research on Educational Effectiveness, Washington, DC.Google Scholar
Krumm, A. E., Beattie, R., Takahashi, S., D'Angelo, C., Feng, M., & Cheng, B. (2016). Practical measurement and productive persistence: Strategies for using digital learning system data to drive improvement. Journal of Learning Analytics, 3(2), 116–38. doi: 10.18608/jla.2016.32.6.Google Scholar
Lazowski, R. A. & Hulleman, C. S. (2016). Motivation interventions in education: A meta-analytic review. Review of Educational Research, 86(2), 602–40. doi: 10.3102/0034654315617832.Google Scholar
Lewin, K., Dembo, T., Festinger, L., & Sears, P. S. (1944). Level of aspiration. In Hunt, J. M. (Ed.), Personality and the behavior disorders (pp. 333–78). New York, NY: Ronal Press. doi: 10.1037/10319-006.Google Scholar
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11.Google Scholar
Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46(3), 598–609. doi: 10.1037//0022-3514.46.3.598.Google Scholar
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95(4), 667–86. doi: 10.1037/0022-0663.95.4.667.Google Scholar
Raynor, J. O. (1982). Future orientation, self-evaluation, and achievement motivation: Use of an expectancy × value theory of personality functioning and change. In Feather, N. T. (Ed.), Expectations and actions: Expectancy-value models in psychology (pp. 97–124). Hillsdale, NJ: Erlbaum.Google Scholar
Renninger, K. A. & Bachrach, J. E. (2015). Studying triggers for interest and engagement using observational methods. Educational Psychologist, 50(March), 58–69. doi: 10.1080/00461520.2014.999920.Google Scholar
Renninger, K. A. & Hidi, S. (2016). The power of interest for motivation and engagement. Abingdon: Routledge. Retrieved from http://works.swarthmore.edu/fac-education/91.Google Scholar
Rosenzweig, E. Q., Hulleman, C. H., Barron, K. E., et al. (in press). Promises and pit- falls of adapting utility value interventions for online mathematics courses. Journal of Experimental Education. doi: 10.1080/00220973.2018.1496059.Google Scholar
Rosenzweig, E. Q. & Wigfield, A. (2016). STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist, 51(2), 146–63. doi: 10.1080/00461520.2016.1154792.Google Scholar
Rotgans, J. I. & Schmidt, H. G. (2011). The role of teachers in facilitating situational interest in an active-learning classroom. Teaching and Teacher Education, 27(1), 37–42. doi: 10.1016/j.tate.2010.06.025.Google Scholar
Schmeiser, C. B. & Welch, C. J. (2006). Test development. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 307–53). Westport, CT: Praeger Publishers.Google Scholar
Schwarz, N. & Oyserman, D. (2001). Asking questions about behavior: Cognition, communication, and questionnaire construction. American Journal of Evaluation, 22(2), 127–60. doi: 10.1177/109821400102200202.Google Scholar
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1), 107–20. doi: 10.1007/s11336-008-9101-0.Google Scholar
Tibbetts, Y., Harackiewicz, J. M., Canning, E. A., Boston, J. S., Priniski, S. J., & Hyde, J. S. (2016). Affirming independence: Exploring mechanisms underlying a values affirmation intervention for first-generation students. Journal of Personality and Social Psychology, 110, 635–59. doi: 10.1037/pspa0000049.Google Scholar
Traub, R. E. & Rowley, G. L. (1991). Understanding reliability. Educational Measurement: Issues and Practice, 10(1), 37–45. doi: 10.1111/j.1745-3992.1991.tb00183.x.Google Scholar
Vroom, V. H. (1964). Work and motivation: Classic readings in organizational behavior. New York, NY: John Wiley & Sons.Google Scholar
Wise, S. L. & DeMars, C. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 1–17.Google Scholar
Yeager, D. S., Bryk, A., Muhich, J., Hausman, H., & Morales, L. (2013). Practical measurement. Palo Alto, CA: Carnegie Foundation for the Advancement of Teaching. www.carnegiefoundation.org/resources/publications/practical-measurement/.Google Scholar
Yeager, D. S. & Walton, G. M. (2011). Social-psychological interventions in education: They're not magic. Review of Educational Research, 81(2), 267–301. doi: org/10.3102/0034654311405999.CrossRefGoogle Scholar
- 3
- Cited by