Hostname: page-component-8448b6f56d-42gr6 Total loading time: 0 Render date: 2024-04-23T18:52:26.927Z Has data issue: false hasContentIssue false
  • English
  • Français

Survey Nonresponse and Mass Polarization: The Consequences of Declining Contact and Cooperation Rates

Published online by Cambridge University Press:  02 May 2022

AMNON CAVARI Open the ORCID record for AMNON CAVARI [Opens in a new window]  and
GUY FREEDMAN Open the ORCID record for GUY FREEDMAN [Opens in a new window]
AMNON CAVARI*
Affiliation:
Reichman University, Israel
GUY FREEDMAN*
Affiliation:
University of Texas at Austin, United States
*
Amnon Cavari, Assistant Professor, Lauder School of Government, Diplomacy and Strategy, Reichman University (IDC), Israel, cavari@idc.ac.il.
Guy Freedman, PhD Candidate, Department of Government, University of Texas at Austin, United States, freedmanguy@utexas.edu.
Get access Rights & Permissions [Opens in a new window]

Abstract

Recent studies question whether declining response rates in survey data overstate the level of polarization of Americans. At issue are the sources of declining response rates—declining contact rates, associated mostly with random polling mechanisms, or declining cooperation rates, associated with personal preferences, knowledge, and interest in politics—and their differing effects on measures of polarization. Assessing 158 surveys (2004–2018), we show that declining cooperation is the primary source of declining response rates and that it leads to survey overrepresentation of people who are more engaged in politics. Analyzing individual responses to 1,223 policy questions in those surveys, we further show that, conditional on the policy area, this survey bias overestimates or underestimates the partisan divide among Americans. Our findings question the perceived strength of mass polarization and move forward the discussion about the effect of declining survey response on generalizations from survey data.

Type
Letter
Information
American Political Science Review , Volume 117 , Issue 1 , February 2023 , pp. 332 - 339
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abramowitz, Alan I. 2018. The Great Alignment: Race, Party Transformation, and the Rise of Donald Trump. New Haven, CT: Yale University Press.CrossRefGoogle Scholar
Abramowitz, Alan I., and Saunders, Kyle L.. 2008. “Is Polarization a Myth?” Journal of Politics 70 (2): 542–55.CrossRefGoogle Scholar
Berinsky, Adam J. 2004. Silent Voices: Public Opinion and Political Participation in America. Princeton, NJ: Princeton University Press.Google Scholar
Brehm, John. 1993. The Phantom Respondents: Opinion Surveys and Political Representation. Ann Arbor: University of Michigan Press.Google Scholar
Burns, Nancy, Schlozman, Kay Lehman, and Verba, Sidney. 2001. The Private Roots of Public Action: Gender, Equality, and Political Participation. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Campbell, James E. 2016. Polarized: Making Sense of a Divided America. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar
Cavari, Amnon. 2017. The Party Politics of Presidential Rhetoric. New York: Cambridge University Press.CrossRefGoogle Scholar
Cavari, Amnon, and Freedman, Guy. 2018. “Polarized Mass or Polarized Few? Assessing the Parallel Rise of Survey Nonresponse and Measures of Polarization.” Journal of Politics 80 (2): 719–25.CrossRefGoogle Scholar
Cavari, Amnon, and Freedman, Guy. 2021. American Public Opinion toward Israel: From Consensus to Divide. New York: Routledge.Google Scholar
Cavari, Amnon, and Freedman, Guy. 2022. “Replication Data for: Survey Nonresponse and Mass Polarization: The Consequences of Declining Contact and Cooperation Rates.” Harvard Dataverse. Dataset. https://doi.org/10.7910/DVN/UECUBY.CrossRefGoogle Scholar
Claassen, Ryan L., and Highton, Benjamin. 2009. “Policy Polarization among Party Elites and the Significance of Political Awareness in the Mass Public.” Political Research Quarterly 62 (3): 538–51.CrossRefGoogle Scholar
Clinton, Josh, Agiesta, Jennifer, Brenan, Megan, Burge, Camille, Connelly, Marjorie, Edwards-Levy, Ariel, and Fraga, Bernard. 2021. “Task Force on 2020 Pre-Election Polling: An Evaluation of the 2020 General Election Polls.” New York: American Association for Public Opinion Research.Google Scholar
Cranmer, Skyler J., and Desmarais, Bruce A.. 2017. “What Can We Learn from Predictive Modeling?” Political Analysis 25 (2):145–66.CrossRefGoogle Scholar
Dutwin, David, and Buskirk, Trent D.. 2017. “Apples to Oranges or Gala versus Golden Delicious? Comparing Data Quality of Nonprobability Internet Samples to Low Response Rate Probability Samples.” Public Opinion Quarterly 81 (S1): 213–39.CrossRefGoogle Scholar
Egan, Patrick J. 2013. Partisan Priorities: How Issue Ownership Drives and Distorts American Politics. New York: Cambridge University Press.CrossRefGoogle Scholar
Gelpi, Christopher. 2010. “Performing on Cue? The Formation of Public Opinion toward War.” Journal of Conflict Resolution 54 (1): 88–116.CrossRefGoogle Scholar
Groves, Robert M., Presser, Stanley, and Dipko, Sarah. 2004. “The Role of Topic Interest in Survey Participation Decisions.” Public Opinion Quarterly 68 (1): 2–31.CrossRefGoogle Scholar
Guisinger, Alexandra, and Saunders, Elizabeth N.. 2017. “Mapping the Boundaries of Elite Cues: How Elites Shape Mass Opinion across International Issues.” International Studies Quarterly 61 (2): 425–41.CrossRefGoogle Scholar
Hillygus, Sunshine D. 2005. “The Missing Link: Exploring the Relationship between Higher Education and Political Engagement.” Political Behavior 27 (1): 25–47.CrossRefGoogle Scholar
Hoerl, Arthur E., and Kennard, Robert W.. 1970. “Ridge Regression: Biased Estimation for Nonorthogonal Problems.” Technometrics 12 (1): 55–67.CrossRefGoogle Scholar
Imdad, Muhammad Ullah, and Aslam, Muhammad. 2018. “{lmridge}: Linear Ridge Regression with Ridge Penalty and Ridge Statistics.” (version 1.2). R package. https://CRAN.R-project.org/package=lmridge.Google Scholar
Jennings, Will, and Wlezien, Christopher. 2018. “Election Polling Errors across Time and Space.” Nature Human Behaviour 2 (4): 276–83.CrossRefGoogle ScholarPubMed
Keeter, Scott. 2018. “Are Public Opinion Polls Doomed?” Nature Human Behaviour 2 (4): 246–47.CrossRefGoogle ScholarPubMed
Keeter, Scott, Kennedy, Courtney, and Deane, Claudia. 2020. “Understanding How 2020 Election Polls Performed and What It Might Mean for Other Kinds of Survey Work.” Pew Research Center. November 13. https://www.pewresearch.org/fact-tank/2020/11/13/understanding-how-2020s-election-polls-performed-and-what-it-might-mean-for-other-kinds-of-survey-work/.Google Scholar
Keeter, Scott, Kennedy, Courtney, Dimock, Michael, Best, Jonathan, and Craighill, Peyton. 2006. “Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey.” International Journal of Public Opinion Quarterly 70 (5): 759–79.CrossRefGoogle Scholar
Kertzer, Joshua D., Brooks, Deborah Jordan, and Brooks, Stephen G.. 2021. “Do Partisan Types Stop at the Water’s Edge?” Journal of Politics 83 (4): 1764–82.CrossRefGoogle Scholar
Mellon, Jonathan, and Prosser, Christopher. 2017. “Missing Nonvoters and Misweighted Samples: Explaining the 2015 Great British Polling Miss.” Public Opinion Quarterly 81 (3): 661–87.CrossRefGoogle Scholar
Mellon, Jonathan, and Prosser, Christopher. 2021. “Correlation with Time Explains the Relationship between Survey Nonresponse and Mass Polarization.” Journal of Politics 83 (1): 390–95.CrossRefGoogle Scholar
Miller, William L. 1972. “Measures of Electoral Change Using Aggregate Data.” Journal of the Royal Statistical Society: Series A (General) 135 (1): 122–42.CrossRefGoogle Scholar
Mintz, Alex, and Huang, Chi. 1990. “Defense Expenditures, Economic Growth, and the ‘Peace Dividend.’” American Political Science Review 84 (4): 1283–93.CrossRefGoogle Scholar
Muniz, Gisela, and Kibria, B. M. Golam. 2009. “On Some Ridge Regression Estimators: An Empirical Comparison.” Communications in Statistics—Simulation and Computation 38 (3): 621–30.CrossRefGoogle Scholar
Page, Benjamin I., and Bouton, Marshall M.. 2006. The Foreign Policy Disconnect: What Americans Want from Our Leaders but Don’t Get. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Panagopoulos, Costas. 2021. “Accuracy and Bias in the 2020 U.S. General Election Polls.” Presidential Studies Quarterly 51 (1): 214–27.CrossRefGoogle Scholar
Perrin, Andrew J., and Gillis, Alanna. 2019. “How College Makes Citizens: Higher Education Experiences and Political Engagement.” Socius 5: 1–16.CrossRefGoogle Scholar
Prosser, Christopher, and Mellon, Jonathan. 2018. “The Twilight of the Polls? A Review of Trends in Polling Accuracy and the Causes of Polling Misses.” Government and Opposition 53 (4): 757–90.CrossRefGoogle Scholar
Rawlings, John O., Pantula, Sastry G., and Dickey, David A.. 1998. Applied Regression Analysis. New York: Springer Verlag.CrossRefGoogle Scholar
Seber, George A. F., and Lee, Alan J.. 2003. Linear Regression Analysis. Hoboken, NJ: John Wiley & Sons.CrossRefGoogle Scholar
Sulfaro, Valerie A. 1996. “The Role of Ideology and Political Sophistication in the Structure of Foreign Policy.” American Politics Quarterly 24 (3): 303–37.CrossRefGoogle Scholar
Tourangeau, Roger, Groves, Robert M., and Redline, Cleo D.. 2010. “Sensitive Topics and Reluctant Respondents: Demonstrating a Link between Nonresponse Bias and Measurement Error.” Public Opinion Quarterly 74 (3): 413–32.CrossRefGoogle Scholar
Tripp, Robert E. 1983. “Non-Stochastic Ridge Regression and Effective Rank of the Regressors Matrix.” PhD diss. Department of Statistics, Virginia Polytechnic Institute and State University.Google Scholar
Webster, Steven W., and Abramowitz, Alan I.. 2017. “The Ideological Foundations of Affective Polarization in the US Electorate.” American Politics Research 45 (4): 621–47.CrossRefGoogle Scholar
Supplementary material: Link

Cavari and Freedman Dataset

Dataset

https://doi.org/10.7910/DVN/UECUBY
Link
Supplementary material: PDF

Cavari and Freedman supplementary material

Cavari and Freedman supplementary material

Download Cavari and Freedman supplementary material(PDF)
PDF 109.6 KB