No CrossRef data available.
Article contents
Proxy failure and poor measurement practices in psychological science
Published online by Cambridge University Press: 13 May 2024
Abstract
We argue that proxy failure contributes to poor measurement practices in psychological science and that a tradeoff exists between the legibility and fidelity of proxies whereby increasing legibility can result in decreased fidelity.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2024. Published by Cambridge University Press
References
Alexandrova, A., & Haybron, D. M. (2016). Is construct validation valid? Philosophy of Science, 83(5), 1098–1109. https://doi.org/10.1086/687941CrossRefGoogle Scholar
Alexandrova, A., & Singh, R. (2022). When well-being becomes a number. In Newfield, C., Alexandrova, A., & John, S. (Eds.), Limits of the numerical: The abuses and uses of quantification (pp. 181–199). University of Chicago Press.Google Scholar
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.Google Scholar
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191CrossRefGoogle ScholarPubMed
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The “Reading the Mind in the Eyes” test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, 42(2), 241–251. https://doi.org/10.1017/S0021963001006643CrossRefGoogle ScholarPubMed
Bringmann, L. F., Elmer, T., & Eronen, M. I. (2022). Back to basics: The importance of conceptual clarification in psychological science. Current Directions in Psychological Science: A Journal of the American Psychological Society, 31(4), 340–346. https://doi.org/10.1177/09637214221096485CrossRefGoogle Scholar
Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626CrossRefGoogle ScholarPubMed
Dordevic, J., Zivanovic, M., Pavlovic, A., Mihajlovic, G., Karlicic, I. S., & Pavlovic, D. (2017). Psychometric evaluation and validation of the Serbian version of “Reading the Mind in the Eyes” test. Psihologija, 50(4), 483–502. https://doi.org/10.2298/PSI170504010DGoogle Scholar
Eronen, M. I., & Bringmann, L. F. (2021). The theory crisis in psychology: How to move forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/10.1177/1745691620970586CrossRefGoogle ScholarPubMed
Feest, U. (2020). Construct validity in psychological tests – The case of implicit social cognition. European Journal for Philosophy of Science, 10(1), 4. https://doi.org/10.1007/s13194-019-0270-8CrossRefGoogle Scholar
Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393CrossRefGoogle Scholar
Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological & Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063CrossRefGoogle Scholar
Fried, E. I., Flake, J. K., & Robinaugh, D. J. (2022). Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology, 1(6), 358–368. https://doi.org/10.1038/s44159-022-00050-2CrossRefGoogle ScholarPubMed
Higgins, W. C., Ross, R. M., Langdon, R., & Polito, V. (2023a). The “Reading the Mind in the Eyes” test shows poor psychometric properties in a large, demographically representative U.S. sample. Assessment, 30(6), 1777–1789. https://doi.org/10.1177/10731911221124342CrossRefGoogle Scholar
Higgins, W. C., Ross, R. M., Polito, V., & Kaplan, D. M. (2023b). Three threats to the validity of the Reading the Mind in the Eyes Test: A commentary on Pavlova and Sokolov (2022). Neuroscience and Biobehavioral Reviews, 147, 105088. https://doi.org/10.1016/j.neubiorev.2023.105088CrossRefGoogle Scholar
Hussey, I., & Hughes, S. (2020). Hidden invalidity among 15 commonly used measures in social and personality psychology. Advances in Methods and Practices in Psychological Science, 3(2), 166–184. https://doi.org/10.1177/2515245919882903CrossRefGoogle Scholar
Kittel, A. F. D., Olderbak, S., & Wilhelm, O. (2022). Sty in the mind's eye: A meta-analytic investigation of the nomological network and internal consistency of the “Reading the Mind in the Eyes” test. Assessment, 29(5), 872–895. https://doi.org/10.1177/1073191121996469CrossRefGoogle Scholar
Olderbak, S., Wilhelm, O., Olaru, G., Geiger, M., Brenneman, M. W., & Roberts, R. D. (2015). A psychometric analysis of the reading the mind in the eyes test: Toward a brief form for research and applied settings. Frontiers in Psychology, 6, 1503. https://doi.org/10.3389/fpsyg.2015.01503CrossRefGoogle Scholar
Pavlova, M. A., & Sokolov, A. A. (2022). Reading language of the eyes. Neuroscience and Biobehavioral Reviews, 140, 104755–104755. https://doi.org/10.1016/j.neubiorev.2022.104755CrossRefGoogle ScholarPubMed
Redondo, I., & Herrero-Fernández, D. (2018). Validation of the Reading the Mind in the Eyes Test in a healthy Spanish sample and women with anorexia nervosa. Cognitive Neuropsychiatry, 23(4), 201–217. https://doi.org/10.1080/13546805.2018.1461618CrossRefGoogle Scholar
Shaw, M., Cloos, L. J. R., Luong, R., Elbaz, S., & Flake, J. K. (2020). Measurement practices in large-scale replications: Insights from many labs 2. Canadian Psychology=Psychologie Canadienne, 61(4), 289–298. https://doi.org/10.1037/cap0000220CrossRefGoogle Scholar
Slaney, K. (2017). Validating psychological constructs historical, philosophical, and practical dimensions. Palgrave Macmillan.Google Scholar
Topić, M. K., & Kovačević, M. P. (2019). Croatian adaptation of the revised Reading the Mind in the Eyes Test (RMET). Psihologijske Teme, 28(2), 377–395. https://doi.org/10.31820/pt.28.2.8CrossRefGoogle Scholar
Van Staden, J. G., & Callaghan, C. W. (2021). An evaluation of the reading the mind in the eyes test's psychometric properties and scores in South Africa-cultural implications. Psychological Research, 86(7), 2289–2300. https://doi.org/10.1007/s00426-021-01539-wCrossRefGoogle ScholarPubMed
Target article
Dead rats, dopamine, performance metrics, and peacock tails: Proxy failure is an inherent risk in goal-oriented systems
Related commentaries (20)
An updated perspective on teleonomy
Animal welfare science, performance metrics, and proxy failure
Behavioral proxies compete by the time courses of their rewards, including endogenous rewards
Changing the incentive structure of social media may reduce online proxy failure and proliferation of negativity
Dynamic diversity is the answer to proxy failure
Genies, lawyers, and smart-asses: Extending proxy failures to intentional misunderstandings
It's the biology, stupid! Proxy failures in economic decision making
Navigating proxy failures in education: Learning from human and animal play
On abstract goals’ perverse effects on proxies: The dynamics of unattainability
Proxies, heuristics, and goal alignment
Proxy failure and poor measurement practices in psychological science
Proxy failure as a feature of adaptive control systems
Proxy failure in academia: More than just another example
Proxy failure in social policies as one of the main causes of persistent sexism and racism
Proxy failures in practice: Examples from the sociology of science
Reductionism and proxy failure: From neuroscience to target-based drug discovery
Regulator and agent sophistication as an explanation-generating engine for proxy failure dynamics
Subjective and objective corruption of intuition and rational choice
The cost of success or failure for proxy signals in ecological problems
The determinants of proxy treadmilling in evolutionary models of reliable signals
Author response
Teleonomy, legibility, and diversity: Do we need more “proxynomics”?