Skip to main content Accessibility help
Hostname: page-component-99c86f546-qdp55 Total loading time: 0.235 Render date: 2021-12-04T04:53:01.736Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Measuring Simultaneous Emotions: Existing Problems and a New Way Forward

Published online by Cambridge University Press:  17 February 2020

Matthew Rhodes-Purdy
Department of Political Science, Clemson University, Clemson, SC 29634, USA, e-mail:
Rachel Navarre
Department of Political Science, Bridgewater State University, Bridgewater, MA 02325, USA, e-mail:
Stephen M. Utych
Department of Political Science, Boise State University, Boise, ID 83725, USA, e-mail:


Studies on discrete emotions typically work to evoke one emotion at a time. Yet many political phenomena cause multiple emotions. Threats, for example, cause, anger, and fear, have diametrically opposing behavioral consequences. As a result, the effect of experimental treatments can be masked by the countervailing influence of emotions with similar affect. This issue is exacerbated by existing measures of negative emotions, such as the Positive and Negative Affect Schedule (PANAS). We show that the PANAS is contaminated by systematic measurement error, as negative affect produced by one emotion influences responses on the other. To overcome this, we develop an alternative version of the PANAS that allows respondents to select which emotions they are feeling, then rate the severity. This technique accurately captures respondent’s emotional reactions, reducing measurement error and thus decreasing the correlation between fear and anger. The tactics we developed have broad relevance for experimental researchers analyzing emotional responses to politics.

Research Article
© The Experimental Research Section of the American Political Science Association 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


This research was funded through the Boise State School of Public Service. A previous version of this manuscript was presented at the 2019 Annual Meeting of the Midwest Political Science Association. The authors declare no conflicts of interest. The data, code, and any additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: doi: 10.7910/DVN/MDMVZ5.


Albertson, B. and Gadarian, S. K.. 2015. Anxious Politics: Democratic Citizenship in a Threatening World. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Anderson, K. G., Deschenes, S. S. and Dugas, M. J.. 2016. Experimental Manipulation of Avoidable Feelings of Uncertainty: Effects on Anger and Anxiety. Journal of Anxiety Disorders 41: 5058.CrossRefGoogle ScholarPubMed
Barker, D. C. and Hansen, S. B.. 2005. All Things Considered: Systematic Cognitive Processing and Electoral Decision-Making. Journal of Politics 67(2): 319344.CrossRefGoogle Scholar
Brader, T. 2005. Striking a Responsive Chord: How Political Ads Motivate and Persuade Voters by Appealing to Emotions. American Journal of Political Science 49(2): 388405.CrossRefGoogle Scholar
Brader, T. and Marcus, G. E.. 2014. Emotion and Political Psychology. In Oxford Handbook of Political Psychology (2nd ed.), eds. Huddy, L., Sears, D. O. and Levy, J. S. Oxford: Oxford University Press.Google Scholar
Chandler, J., Mueller, P. and Paolacci, G.. 2014. Nonnaïveté among Amazon Mechanical Turk Workers: Consequences and Solutions for Behavioral Researchers. Behavior Research Methods 46: 112130.CrossRefGoogle ScholarPubMed
Flake, J. K., Pek, J. and Hehman, E.. 2017. Construct Validation in Social and Personality Research: Current Practice and Recommendations. Social Psychological and Personality Science 8(4): 370378.CrossRefGoogle Scholar
Forgas, J. P. 1995. Mood and Judgment: The Affect Infusion Model (AIM). Psychological Bulletin 117(1): 3966.CrossRefGoogle Scholar
Gross, J. 2009. Handbook of Emotion Regulation. New York, NY: Guilford Press.Google Scholar
Gross, J. J. 1998. The Emerging Field of Emotion Regulation: An Integrative Review. Review of General Psychology 2(3): 271299.CrossRefGoogle Scholar
Harmon-Jones, C., Bastian, B. and Harmon-Jones, E.. 2016. The Discrete Emotions Questionnaire: A New Tool for Measuring State Self-Reported Emotions. PLoS ONE 11(8): e0159915.CrossRefGoogle ScholarPubMed
Hauser, D. J. and Schwarz, N.. 2016. Attentive Turkers: MTurk Participants Perform Better on Online Attention Checks Than Do Subject Pool Participants. Behavior Research Methods 48(1): 400407.CrossRefGoogle ScholarPubMed
Herzog, A. R. and Bachman, J. G.. 1981. Effects of Questionnaire Length on Response Quality. Public Opinion Quarterly 45(4): 549559.CrossRefGoogle Scholar
Huddy, L. and Feldman, S.. 2011. Americans Respond Politically to 9/11: Understanding the Impact of the Terrorist Attacks and Their Aftermath. American Psychologist 66(6): 455467.CrossRefGoogle ScholarPubMed
Kahneman, D. 2011. Thinking, Fast and Slow. New York, NY: Farrar Straus and Giroux.Google Scholar
Krosnick, J. A. 1991. Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys. Special Issue: Cognition and Survey Measurement. Applied Cognitive Psychology 5: 213236.CrossRefGoogle Scholar
Lerner, J. S. and Keltner, D.. 2000. Beyond Valence: Toward a Model of Emotion-Specific Influences on Judgment and Choice. Cognition and Emotion 14: 473493.CrossRefGoogle Scholar
Mackuen, M. 1993. Anxiety, Enthusiasm and the Vote: The Emotional Underpinnings of Learning and Involvement during Presidential Campaigns. American Political Science Review 87(3): 688701.Google Scholar
Marcus, G. E., Neuman, W. R. and MacKuen, M. B.. 2000. Affective Intelligence and Political Judgment. Chicago, IL: University of Chicago Press.Google Scholar
Marcus, G. E., Neuman, W. R. and MacKuen, M. B.. 2017. Measuring Emotional Response: Comparing Alternative Approaches to Measurement. Political Science Research and Methods 5(4): 733754.CrossRefGoogle Scholar
Mauss, I. B. and Robinson, M. B.. 2009. Measures of Emotion: A Review. Cognition and Emotion 23(2): 209237.CrossRefGoogle ScholarPubMed
Montgomery, J. M., Nyhan, B. and Torres, M.. 2018. How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It. American Journal of Political Science 62(3): 760775.CrossRefGoogle Scholar
Petty, R. E. and Cacioppo, J. T.. 1981. Attitudes and Persuasion: Classic and Contemporary Approaches. Dubuque, IA: William C. Brown.Google Scholar
Rhodes-Purdy, M., Navarre, R. and Utych, S.. 2019. Replication Data For: Measuring Simultaneous Emotions: Existing Challenges and a New Way Forward. Harvard Dataverse, doi: 10.7910/DVN/MDMVZ5.CrossRefGoogle Scholar
Robinson, M. D. and Clore, G. L.. 2002. Episodic and Semantic Knowledge in Emotional Self-Report: Evidence for Two Judgment Processes. Journal of Personality and Social Psychology 83(1): 198215.CrossRefGoogle ScholarPubMed
Scheer, K. R. 2005. What Are Emotions? And How Can They Be Measured?. Social Science Information 44(4): 695729.CrossRefGoogle Scholar
Silva, B. C., Andreadis, I., Adnduiza, E., Blanusa, N., Corti, Y. M., Delfino, G., Rico, G., Ruth-Lovell, S. P., Spruyt, B., Steenbergen, M. and Littvay, L.. 2018. Public Opinion Surveys: A New Scale. In The Ideational Approach to Populism: Concept, Theory and Analysis, eds. Hawkins, K. A., Carlin, R. E. and Kaltwasser, C. R. New York, NY: Routledge, 150178.CrossRefGoogle Scholar
Stine, R.A. 1995. Graphical interpretation of variance inflation factors. The American Statistician 49(1): 5356.Google Scholar
Watson, D. & Clark, L. A.. 1994. The PANAS-X: Manual for the Positive and Negative Affect Schedule-Expanded Form. Unpublished manuscript. University of Iowa, Iowa City.CrossRefGoogle Scholar
Watson, D., Clark, L. A. and Tellegen, A.. 1988. Development and Validation of Brief Measures of Positive and Negative Affect: The PANAS Scales. Journal of Personality and Social Psychology 54(6): 10631070.CrossRefGoogle ScholarPubMed
Watson, D. and Tellegen, A.. 1985. Toward a Consensual Structure of Mood. Psychological Bulletin 98(2): 219235.CrossRefGoogle Scholar
Yan, T. 2008. Nondifferentiation. In Encyclopedia of Survey Research Methodology, eds. Lavrakas, P. J. Newbury Park, CA: Sage, 520521.Google Scholar
Zaller, J. and Feldman, S.. 1992. A Simple Theory of the Survey Response: Answering Questions versus Revealing Preferences. American Journal of Political Science 36(3): 576616.CrossRefGoogle Scholar
Supplementary material: Link

Rhodes-Purdy et al. Dataset

Supplementary material: File

Rhodes-Purdy et al. supplementary material


Download Rhodes-Purdy et al. supplementary material(File)
File 80 KB
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Measuring Simultaneous Emotions: Existing Problems and a New Way Forward
Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Measuring Simultaneous Emotions: Existing Problems and a New Way Forward
Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Measuring Simultaneous Emotions: Existing Problems and a New Way Forward
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *