Skip to main content Accessibility help
×
Home
Hostname: page-component-59b7f5684b-569ts Total loading time: 1.49 Render date: 2022-09-29T22:21:32.245Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "displayNetworkTab": true, "displayNetworkMapGraph": false, "useSa": true } hasContentIssue true

28 - Measuring Motivation in Educational Settings

A Case for Pragmatic Measurement

from Part VI - Methods, Measures, and Perspective

Published online by Cambridge University Press:  15 February 2019

K. Ann Renninger
Affiliation:
Swarthmore College, Pennsylvania
Suzanne E. Hidi
Affiliation:
University of Toronto
Get access

Summary

A reality of conducting motivation research in educational settings is that there are tensions between technical standards of research and practical constraints of a given situation. Although adherence to standards for high-quality measurement is critical for good-quality data to be collected, measurement also requires substantial resources to ensure quality. In the current chapter, we discuss several examples of real data collected in different educational settings using a pragmatic measurement framework. Based on contemporary measurement perspectives, the pragmatic measurement framework emphasizes building evidence-based arguments to support the use and interpretation of a measure. Example 1 explores college students’ attitudes toward general education classes. Example 2 tracks students’ classroom motivation over several time points. Example 3 assesses experimental differences from an online motivation intervention. Together the three examples cover a range of possible research questions that researchers may encounter. As a whole, this chapter demonstrates that important and meaningful insights can be gained using pragmatic approaches to measurement. Importantly, we discuss the trade-offs that researchers or other measure users must consider when adopting a pragmatic approach to measurement.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Atkinson, J. W. (1964). An introduction to motivation. Princeton, NJ: D. Van Nostrand Company.Google Scholar
Bandura, A. & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. Journal of Personality and Social Psychology, 41(3), 586–98. doi: 10.1037/0022-3514.41.3.586.CrossRefGoogle Scholar
Barron, K. E., Grays, M., & Hulleman, C. S. (2014). Assessing motivation in general education. Poster presented at the Annual Meeting of the American Educational Research Association, Philadelphia, PA.Google Scholar
Barron, K. E. & Hulleman, C. S. (2015). Expectancy-value-cost model of motivation. In J. D. Wright (Ed.) International encyclopedia of the social & behavioral sciences (Vol. 8, pp. 503–9). Elsevier. doi: 10.1016/B978-0-08-097086-8.26099-6.Google Scholar
Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17(2–3), 6287. doi: 10.1080/10627197.2012.715014.CrossRefGoogle Scholar
Bill and Melinda Gates Foundation. (2013). Ensuring fair and reliable measures of effective teaching. Retrieved from www.edweek.org/media/17teach-met1.pdf.
Bowman, N. A. (2010). Can 1st-year college students accurately report their learning and development? American Educational Research Journal, 47(2), 466–96. doi: 10.3102/0002831209353595.CrossRefGoogle Scholar
Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Multivariate applications series. New York, NY: Routledge.Google Scholar
Cronbach, L. J. & Meehl, P. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281302.CrossRefGoogle ScholarPubMed
Duckworth, A. L. & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–51. doi: 10.3102/0013189X15584327.CrossRefGoogle ScholarPubMed
Eccles, J. S. (1983). Expectancies, values, and academic behaviors. In Spence, J. T. (Ed.), Achievement-related motives and behaviors (pp. 119–46). San Francisco, CA: W. H. Freeman.Google Scholar
Eccles, J. S., Adler, T., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1983). Expectancies, values, and academic behaviors. In Spence, J. T. (Ed.), Achievement and achievement motivation (pp. 75146). San Francisco, CA: W. H. Freeman.Google Scholar
Embretson, S. E. & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Fredricks, J. A. & McColsky, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Christenson, S. L., Reschly, A. L., & Wylie, C. (Eds.), Handbook of research on student engagement (pp. 763–82). New York, NY: Springer. doi: 10.1007/978-1-4614-2018-7_37.Google Scholar
Gaspard, H., Dicke, A., Flunger, B., Brisson, B. M., Häfner, I., Nagengast, B., & Trautwein, U. (2015). Fostering adolescents’ value beliefs for mathematics with a relevance intervention in the classroom. Developmental Psychology, 51(9), 1226–40. doi: 10.1037/dev0000028.CrossRefGoogle ScholarPubMed
Goldstein, J. & Flake, J. K. (2016). Towards a framework for the validation of early childhood assessment systems. Educational Assessment, Evaluation and Accountability, 28(3), 273–93. doi: 10.1007/s11092-015-9231-8.CrossRefGoogle Scholar
Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106(2), 375–89. doi: 10.1037/a0034679.CrossRefGoogle ScholarPubMed
Hektner, J. M. & Csikszentmihalyi, M. (1996). A longitudinal exploration of flow and intrinsic motivation in adolescents. In Annual Meeting of the American Educational Research Association, NYC, New York.Google Scholar
Hidi, S. & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–27. doi: 10.1207/s15326985ep4102_4.CrossRefGoogle Scholar
Holstermann, N., Ainley, M., Grube, D., Roick, T., & Bögeholz, S. (2012). The specific relationship between disgust and interest: Relevance during biology class dissections and gender differences. Learning and Instruction, 22(3), 185–92. doi: 10.1016/j.learninstruc.2011.10.005.CrossRefGoogle Scholar
Hulleman, C. S. (2007). The role of utility value in the development of interest and achievement. University of Wisconsin-Madison. Retrieved from http://files.eric.ed.gov/fulltext/ED498264.pdf.
Hulleman, C. S., Barron, K. E., Kosovich, J. J., & Lazowski, R. A. (2016). Expectancy-value models of achievement motivation in education. In Lipnevich, A. A., Preckel, F., & Robers, R. D. (Eds.), Psychosocial skills and school systems in the twenty-first century: Theory, research, and applications. (pp. 241–78). Basel, Switzerland: Springer. doi: 10.1007/978-3-319-28606-8.Google Scholar
Hulleman, C. S., Godes, O., Hendricks, B. L., & Harackiewicz, J. M. (2010). Enhancing interest and performance with a utility value intervention. Journal of Educational Psychology 102, 880–95. doi: 10.1037/a0019506.CrossRefGoogle Scholar
Hulleman, C. S. & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326(5958), 1410–12. doi: 10.1126/science.1177067.CrossRefGoogle ScholarPubMed
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2016). Making connections: Replicating and extending the utility value intervention in the classroom. Journal of Educational Psychology. 109(3), 387404. doi: 10.1037/edu0000146.CrossRefGoogle Scholar
Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-analytic review of achievement goal measures: Different labels for the same constructs or different constructs with similar labels? Psychological Bulletin, 136(3), 422–49. doi: 10.1037/a0018947.CrossRefGoogle ScholarPubMed
Jacobs, J. E., Lanza, S., Osgood, D. W., Eccles, J. S., & Wigfield, A. (2002). Changes in children's self-competence and values: Gender and domain differences across grades one through twelve. Child Development, 73(2), 509–27. doi: 10.1111/1467-8624.00421.CrossRefGoogle ScholarPubMed
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–35.CrossRefGoogle Scholar
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 173. doi: 10.1111/jedm.12000.CrossRefGoogle Scholar
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: The Guilford Press.Google Scholar
Kosovich, J. J. (2017). Pragmatic measurement for education science: A method-substance synergy of validation and motivation. Charolottesville, VA: University of Virginia.Google Scholar
Kosovich, J. J., Flake, J. K., & Hulleman, C. S. (2017). Short-term motivation trajectories: A parallel process model of expectancy-value. Contemporary Educational Psychology. 49(3), 130139. doi: 10.1016/j.cedpsych.2017.01.004.CrossRefGoogle Scholar
Kosovich, J. J., Hulleman, C. S., Barron, K. E., & Getty, S. (2015). A practical measure of student motivation: Establishing validity evidence for the expectancy-value-cost scale in middle school. The Journal of Early Adolescence, 35(5–6), 790816. doi: 10.1177/0272431614556890.CrossRefGoogle Scholar
Kosovich, J. J., Hulleman, C. S., & Flake, J. K. (2017). Practical measurement: An argument-based approach to exploring alternative psychometric validity evidence. Poster presented at the annual Society for Research on Educational Effectiveness, Washington, DC.
Krumm, A. E., Beattie, R., Takahashi, S., D'Angelo, C., Feng, M., & Cheng, B. (2016). Practical measurement and productive persistence: Strategies for using digital learning system data to drive improvement. Journal of Learning Analytics, 3(2), 116–38. doi: 10.18608/jla.2016.32.6.CrossRefGoogle Scholar
Lazowski, R. A. & Hulleman, C. S. (2016). Motivation interventions in education: A meta-analytic review. Review of Educational Research, 86(2), 602–40. doi: 10.3102/0034654315617832.CrossRefGoogle Scholar
Lewin, K., Dembo, T., Festinger, L., & Sears, P. S. (1944). Level of aspiration. In Hunt, J. M. (Ed.), Personality and the behavior disorders (pp. 333–78). New York, NY: Ronal Press. doi: 10.1037/10319-006.Google Scholar
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 511.CrossRefGoogle Scholar
Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46(3), 598609. doi: 10.1037//0022-3514.46.3.598.CrossRefGoogle Scholar
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95(4), 667–86. doi: 10.1037/0022-0663.95.4.667.CrossRefGoogle Scholar
Raynor, J. O. (1982). Future orientation, self-evaluation, and achievement motivation: Use of an expectancy × value theory of personality functioning and change. In Feather, N. T. (Ed.), Expectations and actions: Expectancy-value models in psychology (pp. 97124). Hillsdale, NJ: Erlbaum.Google Scholar
Renninger, K. A. & Bachrach, J. E. (2015). Studying triggers for interest and engagement using observational methods. Educational Psychologist, 50(March), 5869. doi: 10.1080/00461520.2014.999920.CrossRefGoogle Scholar
Renninger, K. A. & Hidi, S. (2016). The power of interest for motivation and engagement. Abingdon: Routledge. Retrieved from http://works.swarthmore.edu/fac-education/91.Google Scholar
Rosenzweig, E. Q., Hulleman, C. H., Barron, K. E., et al. (in press). Promises and pit- falls of adapting utility value interventions for online mathematics courses. Journal of Experimental Education. doi: 10.1080/00220973.2018.1496059.
Rosenzweig, E. Q. & Wigfield, A. (2016). STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist, 51(2), 146–63. doi: 10.1080/00461520.2016.1154792.CrossRefGoogle Scholar
Rotgans, J. I. & Schmidt, H. G. (2011). The role of teachers in facilitating situational interest in an active-learning classroom. Teaching and Teacher Education, 27(1), 3742. doi: 10.1016/j.tate.2010.06.025.CrossRefGoogle Scholar
Schmeiser, C. B. & Welch, C. J. (2006). Test development. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 307–53). Westport, CT: Praeger Publishers.Google Scholar
Schwarz, N. & Oyserman, D. (2001). Asking questions about behavior: Cognition, communication, and questionnaire construction. American Journal of Evaluation, 22(2), 127–60. doi: 10.1177/109821400102200202.CrossRefGoogle Scholar
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1), 107–20. doi: 10.1007/s11336-008-9101-0.CrossRefGoogle ScholarPubMed
Tibbetts, Y., Harackiewicz, J. M., Canning, E. A., Boston, J. S., Priniski, S. J., & Hyde, J. S. (2016). Affirming independence: Exploring mechanisms underlying a values affirmation intervention for first-generation students. Journal of Personality and Social Psychology, 110, 635–59. doi: 10.1037/pspa0000049.CrossRefGoogle ScholarPubMed
Traub, R. E. & Rowley, G. L. (1991). Understanding reliability. Educational Measurement: Issues and Practice, 10(1), 3745. doi: 10.1111/j.1745-3992.1991.tb00183.x.CrossRefGoogle Scholar
Vroom, V. H. (1964). Work and motivation: Classic readings in organizational behavior. New York, NY: John Wiley & Sons.Google Scholar
Wise, S. L. & DeMars, C. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 117.CrossRefGoogle Scholar
Yeager, D. S., Bryk, A., Muhich, J., Hausman, H., & Morales, L. (2013). Practical measurement. Palo Alto, CA: Carnegie Foundation for the Advancement of Teaching. www.carnegiefoundation.org/resources/publications/practical-measurement/.Google Scholar
Yeager, D. S. & Walton, G. M. (2011). Social-psychological interventions in education: They're not magic. Review of Educational Research, 81(2), 267301. doi: org/10.3102/0034654311405999.CrossRefGoogle Scholar
1
Cited by

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×