Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-26T10:08:01.434Z Has data issue: false hasContentIssue false
  • English
  • Français

The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments

Published online by Cambridge University Press:  28 September 2017

JOSHUA L. KALLA  and
DAVID E. BROOCKMAN
JOSHUA L. KALLA*
Affiliation:
University of California, Berkeley
DAVID E. BROOCKMAN*
Affiliation:
Stanford Graduate School of Business
*
Joshua L. Kalla is a Graduate Student, Department of Political Science, University of California, Berkeley. kalla@berkeley.edu, http://polisci.berkeley.edu/people/person/joshua-kalla.
David E. Broockman is an Assistant Professor, Stanford Graduate School of Business. dbroockman@stanford.edu, https://people.stanford.edu/dbroock/.
Get access Rights & Permissions [Opens in a new window]

Abstract

Significant theories of democratic accountability hinge on how political campaigns affect Americans’ candidate choices. We argue that the best estimate of the effects of campaign contact and advertising on Americans’ candidates choices in general elections is zero. First, a systematic meta-analysis of 40 field experiments estimates an average effect of zero in general elections. Second, we present nine original field experiments that increase the statistical evidence in the literature about the persuasive effects of personal contact tenfold. These experiments’ average effect is also zero. In both existing and our original experiments, persuasive effects only appear to emerge in two rare circumstances. First, when candidates take unusually unpopular positions and campaigns invest unusually heavily in identifying persuadable voters. Second, when campaigns contact voters long before election day and measure effects immediately—although this early persuasion decays. These findings contribute to ongoing debates about how political elites influence citizens’ judgments.

Type
Research Article
Information
American Political Science Review , Volume 112 , Issue 1 , February 2018 , pp. 148 - 166
Copyright
Copyright © American Political Science Association 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We acknowledge seminar participants at the Midwest Political Science Association conference, the Northeast Political Methodology Meeting at NYU, Berkeley, Stanford, and the University of British Columbia, as well as Adam Berinsky, Donald Green, Avi Feller, Shanto Iyengar, Jon Krosnick, Gabriel Lenz, Joel Middleton, Daron Shaw, Jas Sekhon, Eric Schickler, Laura Stoker, and Lynn Vavreck for helpful feedback. All remaining errors are our own. The original studies reported herein were conducted by Working America. The authors served as unpaid consultants to Working America in their personal capacity.

References

REFERENCES

Abramowitz, Alan I. 2010. The Disappearing Center: Engaged Citizens, Polarization, and American Democracy. New Haven, CT: Yale University Press.Google Scholar
Adams, William C., and Smith, Dennis J.. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44 (3): 389–95.Google Scholar
Ansolabehere, Stephen. 2006. “The Paradox of Minimal Effects.” In Capturing Campaign Effects, ed. Brady, Henry E. and Johnston, Richard. Ann Arbor: University of Michigan Press, 29–44.Google Scholar
Anzia, Sarah F. 2011. “Election Timing and the Electoral Influence of Interest Groups.” The Journal of Politics 73 (2): 412–27.CrossRefGoogle Scholar
Arceneaux, Kevin. 2005. “Using Cluster Randomized Field Experiments to Study Voting Behavior.” Annals of the American Academy of Political and Social Science 601 (1): 169–79.CrossRefGoogle Scholar
Arceneaux, Kevin. 2007. “I’m Asking for Your Support: The Effects of Personally Delivered Campaign Messages on Voting Decisions and Opinion Formation.” Quarterly Journal of Political Science 2 (1): 43–65.Google Scholar
Arceneaux, Kevin, and Nickerson, David W.. 2010. “Comparing Negative and Positive Campaign Messages: Evidence From Two Field Experiments.” American Politics Research 38 (1): 54–83.Google Scholar
Arceneaux, Kevin, and Kolodny, Robin. 2009. “Educating the Least Informed: Group Endorsements in a Grassroots Campaign.” American Journal of Political Science 53 (4): 755–70.CrossRefGoogle Scholar
Bailey, Michael A., Hopkins, Daniel J., and Rogers, Todd. 2016. “Unresponsive, Unpersuaded: The Unintended Consequences of Voter Persuasion Efforts.” Political Behavior 38 (3): 713–46.Google Scholar
Barabas, Jason, and Jerit, Jennifer. 2010. “Are Survey Experiments Externally Valid? ”American Political Science Review 104 (2): 226–42.CrossRefGoogle Scholar
Barton, Jared, Castillo, Marco, and Petrie, Ragan. 2014. “What Persuades Voters? A Field Experiment on Political Campaigning.” The Economic Journal 124 (574): F293–F326.CrossRefGoogle Scholar
Bawn, Kathleen, Cohen, Martin, Karol, David, Masket, Seth, Noel, Hans, and Zaller, John. 2012. “A Theory of Political Parties: Groups, Policy Demands and Nominations in American Politics.” Perspectives on Politics 10 (3): 571–97.CrossRefGoogle Scholar
Berelson, Bernard R., Lazarsfeld, Paul F., and McPhee, William N.. 1954. Voting: A Study of Opinion Formation in a Presidential Campaign. Chicago, IL: University of Chicago Press.Google Scholar
Berinsky, Adam J. 2009. In Time of War: Understanding American Public Opinion From World War II to Iraq. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Berinsky, Adam J. 2017. “Measuring Public Opinion with Surveys.” Annual Review of Political Science 20: 309–29.CrossRefGoogle Scholar
Brady, Henry E., Johnston, Richard, and Sides, John. 2006. “The Study of Political Campaigns.” In Capturing Campaign Effects, ed. Brady, Henry E. and Johnston, Richard. Ann Arbor, MI: University of Michigan Press, 1–26.Google Scholar
Broockman, David, and Green, Donald. 2014. “Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments.” Political Behavior 36 (2): 263–89.Google Scholar
Broockman, David E., and Kalla, Joshua L.. 2016. “Durably Reducing Transphobia: A Field Experiment on Door-to-door Canvassing.” Science 352 (6282): 220–4.Google Scholar
Broockman, David, Kalla, Joshua L., and Sekhon, Jasjeet S.. 2017. “The Design of Field Experiments With Survey Outcomes: A Framework for Selecting More Efficient, Robust, and Ethical Designs.” Political Analysis.CrossRefGoogle Scholar
Brox, Brian J., and Shaw, Daron R.. 2009. “Political Parties, American Campaigns, and Effects on Outcomes.” In Handbook of Party Politics, ed Katz, Richard S. & Crotty, William. London: Sage, 146–59.Google Scholar
Campbell, Angus, Converse, Philip E., Miller, Warren E., and Stokes, Donald E.. 1960. The American Voter. Chicago: University of Chicago Press.Google Scholar
Cantoni, Enrico and Pons, Vincent. 2017. “Do Interactions with Candidates Increase Voter Support and Participation? Experimental Evidence from Italy.” MPSA Conference Paper.Google Scholar
Cardy, Emily Arthur. 2005. “An Experimental Field Study of the GOTV and Persuasion Effects of Partisan Direct Mail and Phone Calls.” The Annals of the American Academy of Political and Social Science 601 (1): 28–40.CrossRefGoogle Scholar
Carpenter, Daniel. 2016. “Recruitment by Petition: American Antislavery, French Protestantism, English Suppression.” Perspectives on Politics 14 (3): 700–23.Google Scholar
Chong, Dennis, and Druckman, James N. 2007. “A Theory of Framing and Opinion Formation in Competitive Elite Environments.” Journal of Communication 57 (1): 99–118.Google Scholar
Cubbison, William. 2015. “The Marginal Effects of Direct Mail on Vote Choice.” Paper presented at the Annual Meeting of the Midwest Political Science Association. URL: http://media.wix.com/ugd/3a8c0a_47330c730f56431f8f982a3d842f434a.pdf.Google Scholar
Cunow, Saul, and Schwenzfeier, Meg. 2015. Working America 2014 Persuasion Experiments Results Memo. Technical Report Analyst Institute.Google Scholar
Deaton, Angus. 2010. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.Google Scholar
DellaVigna, Stefano, and Gentzkow, Matthew. 2010. “Persuasion: Empirical Evidence.” Annual Review of Economics 2: 643–69.CrossRefGoogle Scholar
Doherty, David, and Adler, E. Scott. 2014. “The Persuasive Effects of Partisan Campaign Mailers.” Political Research Quarterly 67 (3): 562–573.Google Scholar
Druckman, James N. 2001. “The Implications of Framing Effects for Citizen Competence.” Political Behavior 23 (3): 225–56.Google Scholar
Druckman, James N. 2004a. “Political Preference Formation: Competition, Deliberation, and the (Ir) Relevance of Framing Effects.” American Political Science Review 98 (4): 671–86.CrossRefGoogle Scholar
Druckman, James N. 2004b. “Priming the Vote: Campaign Effects in a U.S. Senate Election.” Political Psychology 25 (04): 577–94.Google Scholar
Druckman, James N., Peterson, Erik, and Slothuus, Rune. 2013. “How Elite Partisan Polarization Affects Public Opinion Formation.” American Political Science Review 107 (01): 57–79.Google Scholar
Druckman, James N., and Leeper, Thomas J.. 2012a. “Is Public Opinion Stable? Resolving the Micro/Macro Disconnect in Studies of Public Opinion.” Daedalus 141 (4): 50–68.Google Scholar
Druckman, James N., and Leeper, Thomas J.. 2012b. “Learning More from Political Communication Experiments: Pretreatment and Its Effects.” American Journal of Political Science 56 (4): 875–96.Google Scholar
Endres, Kyle. 2016. “The Accuracy of Microtargeted Policy Positions.” PS: Political Science & Politics 49 (4): 771–4.Google Scholar
Enos, Ryan D. and Fowler, Anthony. 2016. “Aggregate Effects of Large-Scale Campaigns on Voter Turnout.” Political Science Research and Methods: 1–19. https://doi.org/10.1017/psrm.2016.21 Google Scholar
Enos, Ryan D., and Hersh, Eitan D.. 2015. “Party Activists as Campaign Advertisers: The Ground Campaign as a Principal-Agent Problem.” American Political Science Review 109 (2): 252–78.CrossRefGoogle Scholar
Follman, Dean A., and Proschan, Michael A.. 1999. “Valid Inference in Random Effects Meta-analysis.” Biometrics 55 (3): 732–7.Google Scholar
Foos, Florian. 2017. “First Impressions - Lasting Impressions? The Short- and Long-term Effects of Candidate Contact on Voting Intentions During a High-salience Campaign.” MPSA Conference Paper.Google Scholar
Fowler, Erika Franklin, Ridout, Travis N., and Franz, Michael M.. 2016. “Political Advertising in 2016: The Presidential Election as Outlier?” The Forum: A Journal of Applied Research in Contemporary Politics 4 (14): 445–69.Google Scholar
Franco, Annie, Malhotra, Neil, and Simonovits, Gabor. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–05.Google Scholar
Freeder, Sean, Lenz, Gabriel S., and Turney, Shad. 2017. “The Importance of Knowing ‘What Goes With What’: Reinterpreting the Evidence on Policy Attitude Stability.” Working paper, available at https://www.ocf.berkeley.edu/~glenz/wgww/WGWW.pdf.Google Scholar
Gelman, Andrew, and King, Gary. 1993. “Why Are American Presidential Election Campaign Polls so Variable When Votes Are so Predictable?” British Journal of Political Science 23 (4): 409–51.CrossRefGoogle Scholar
Gelman, Andrew, Goel, Sharad, Rivers, Douglas, Rothschild, David et al. 2016. “The Mythical Swing Voter.” Quarterly Journal of Political Science 11 (1): 103–30.Google Scholar
Gerber, Alan S. 2004. “Does Campaign Spending Work? Field Experiments Provide Evidence and Suggest New Theory.” American Behavioral Scientist 47 (5): 541–74.Google Scholar
Gerber, Alan S., Kessler, Daniel P., and Meredith, Marc. 2011. “The Persuasive Effects of Direct Mail: A Regression Discontinuity Based Approach.” The Journal of Politics 73 (1): 140–55.CrossRefGoogle Scholar
Gerber, Alan S., Karlan, Dean, and Bergan, Daniel. 2009. “Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions.” American Economic Journal: Applied Economics 1 (2): 35–52.Google Scholar
Gerber, Alan S., and Green, Donald P.. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94 (3): 653–63.CrossRefGoogle Scholar
Gerber, Alan S., Green, Donald P., and Larimer, Christopher W.. 2008. “Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment.” American Political Science Review 102 (1): 33–48.Google Scholar
Gerber, Alan S., Gimpel, James, Green, Donald, and Shaw, Daron. 2011. “How Large and Long-lasting Are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Experiment.” American Political Science Review 105 (1): 135–50.Google Scholar
Green, Donald, Palmquist, Bradley, and Schickler, Eric. 2002. Partisan Hearts & Minds: Political Parties and the Social Identities of Voters. New Haven, CT: Yale University Press.Google Scholar
Green, Donald P., and Gerber, Alan S.. 2015. Get Out the Vote: How to Increase Voter Turnout. 3rd ed. Brookings.Google Scholar
Green, Donald P., Krasno, Jonathan S., Coppock, Alexander, Farrer, Benjamin D., Lenoir, Brandon, and Zingher, Joshua N.. 2016. “The Effects of Lawn Signs on Vote Outcomes: Results from Four Randomized Field Experiments.” Electoral Studies 41: 143–50.Google Scholar
Green, Donald P., McGrath, Mary C., and Aronow, Peter M.. 2013. “Field Experiments and the Study of Voter Turnout.” Journal of Elections, Public Opinion and Parties 23 (1): 27–48.Google Scholar
Hersh, Eitan D. 2015. Hacking the Electorate: How Campaigns Perceive Voters. New York: Cambridge University Press.Google Scholar
Hersh, Eitan D., and Schaffner, Brian F.. 2013. “Targeted Campaign Appeals and the Value of Ambiguity.” The Journal of Politics 75 (2): 520–34.Google Scholar
Hill, Seth J., Lo, James, Vavreck, Lynn, and Zaller, John. 2013. “How Quickly We Forget: The Duration of Persuasion Effects from Mass Communication.” Political Communication 30 (4): 521–47.Google Scholar
Hillygus, D. Sunshine, and Shields, Todd G.. 2008. The Persuadable Voter: Wedge Issues in Presidential Campaigns. Princeton, NJ: Princeton University Press.Google Scholar
Holbrook, Thomas M. 1996. Do Campaigns Matter? Thousand Oaks, CA: Sage.Google Scholar
Issenberg, Sasha. 2012. The Victory Lab: The Secret Science of Winning Campaigns. New York: Crown.Google Scholar
Iyengar, Shanto, and Simon, Adam F.. 2000. “New Perspectives and Evidence on Political Communication and Campaign Effects.” Annual Review of Psychology 51 (1): 149–69.CrossRefGoogle ScholarPubMed
Iyengar, Shanto, Sood, Gaurav, and Lelkes, Yphtach. 2012. “Affect, Not Ideology: A Social Identity Perspective on Polarization.” Public Opinion Quarterly 76 (3): 405–31.CrossRefGoogle Scholar
Jacobson, Gary C. 2015. “How do Campaigns Matter?” Annual Review of Political Science 18: 31–47.Google Scholar
Johnston, Richard, Hagen, Michael G., and Jamieson, Kathleen Hall. 2004. The 2000 Presidential Election and the Foundation of Party Politics. New York, NY: Cambridge University Press.Google Scholar
Kalla, Joshua L., and Sekhon, Jasjeet S.. 2017. “Do TV Ads Persuade? A Randomized Experiment in the 2016 Presidential Election.” Working Paper.Google Scholar
Keane, Laura, and Nickerson, David W.. 2013. “A Field Experiment on Nonpartisan Mobilization and Persuasion Down-Ballot.” Working Paper, URL: https://www.researchgate.net/publication/254065918_A_Field_Experiment_on_Nonpartisan_Mobilization_and_Persuasion_Down-Ballot.Google Scholar
Klapper, Joseph T. 1960. The Effects of Mass Communication. New York, NY: Free Press.Google Scholar
Klar, Samara, and Krupnikov, Yanna. 2016. Independent Politics. Cambridge University Press.Google Scholar
Lau, Richard R., Sigelman, Lee, and Rovner, Ivy Brown. 2007. “The Effects of Negative Political Campaigns: A Meta-Analytic Reassessment.” Journal of Politics 69 (4): 1176–209.Google Scholar
Lauderdale, Benjamin E. 2016. “Partisan Disagreements Arising from Rationalization of Common Information.” Political Science Research and Methods 4 (3): 477–92.Google Scholar
Lazarsfeld, Paul F., Berelson, Bernard R., and Gaudet, Hazel. 1948. The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York, NY: Columbia University Press.Google Scholar
Leeper, Thomas J., and Slothuus, Rune. 2015. “Can Citizens Be Framed? How Information, Not Emphasis, Changes Opinions.” Unpublished paper, Aarhus University.Google Scholar
Lenz, Gabriel S. 2012. Follow the Leader? How Voters Respond to Politicians’ Performance and Policies. Chicago, IL: University of Chicago Press.Google Scholar
Miller, Roy E., and Robyn, Dorothy L.. 1975. “A Field Experimental Study of Direct Mail in a Congressional Primary Campaign: What Effects Last Until Election Day.” Experimental Study of Politics 4 (3): 1–36.Google Scholar
Nickerson, David W. 2005. “Partisan Mobilization Using Volunteer Phone Banks and Door Hangers.” Annals of the American Academy of Political and Social Science 601 (1): 10–27.Google Scholar
Nickerson, David W. 2007 a. “Don’t Talk to Strangers: Experimental Evidence of the Need for Targeting.” Presented at the 2007 Annual Meeting of the Midwest Political Science Assocation. Available at https://www.scribd.com/document/98714549/Nickerson-independents.Google Scholar
Nickerson, David W. 2007 b. “Quality Is Job One: Professional and Volunteer Voter Mobilization Calls.” American Journal of Political Science 51 (2): 269–82.CrossRefGoogle Scholar
Nickerson, David W., and Rogers, Todd. 2010. “Do You Have a Voting Plan? Implementation Intentions, Voter Turnout, and Organic Plan Making.” Psychological Science 21 (2): 194–9.Google Scholar
Niven, David. 2013. “Hiding Racism from Ourselves: A Field Experiment on Race Effects in a Statewide Election.” Paper presented at the Annual Meeting of the Midwest Political Science Association, Chicago, April 11–14, 2013.Google Scholar
Page, Benjamin I., and Shapiro, Robert Y.. 1992. The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences. Chicago, IL: University of Chicago Press.Google Scholar
Paluck, Elizabeth Levy, and Green, Donald P.. 2009. “Deference, Dissent, and Dispute Resolution: An Experimental Intervention using Mass Media to Change Norms and Behavior in Rwanda.” American Political Science Review 103 (4): 622–44.CrossRefGoogle Scholar
Panagopoulos, Costas. 2016. “All about that base: Changing campaign strategies in US presidential elections.” Party Politics 22 (2): 179–90.Google Scholar
Petty, Richard E. and Cacioppo, John T.. 1986. “The Elaboration Likelihood Model of Persuasion.” In Advances in Experimental Social Psychology Volume 19, ed Leonard Berkowitz. New York, NY: Academic Press, 123–205.Google Scholar
Potter, Philip B. K. and Gray, Julia. 2008. “Does Costly Signaling Matter? Preliminary Evidence from a Field Experiment.” Working paper, available at http://www.belfercenter.org/sites/default/files/files/publication/Potterr%202008%20FINAL%20DOC.pdfGoogle Scholar
Rogers, Todd, and Nickerson, David W.. 2013. “Can Inaccurate Beliefs about Incumbents Be Changed? And Can Reframing Change Votes?” Working Paper RWP13-018, Harvard Kennedy School. URL: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2271654.Google Scholar
Rogers, Todd, and Middleton, Joel. 2015. “Are Ballot Initiative Outcomes Influenced by the Campaigns of Independent Groups? A Precinct-Randomized Field Experiment Showing That They Are.” Political Behavior 37 (3): 567–93.Google Scholar
Sadin, Meredith L. 2014. “Campaigning with Class: Ambivalent Stereotypes and Candidate Wealth in U.S. Elections.” PhD thesis, Princeton University.Google Scholar
Schickler, Eric. 2016. Racial Realignment: The Transformation of American Liberalism, 1932–1965. Princeton, NJ: Princeton University Press.Google Scholar
Sears, David O. and Funk, Carolyn L.. 1999. “Evidence of the Long-term Persistence of Adults’ Political Predispositions.” The Journal of Politics 61 (1): 1–28.Google Scholar
Shaw, Daron, Blunt, Christopher, and Seaborn, Brent. 2017. “Testing Overall and Synergistic Campaign Effects in a Partisan Statewide Election.” Working paper.Google Scholar
Shaw, Daron R., Green, Donald P., Gimpel, James G., and Gerber, Alan S.. 2012. “Do Robotic Calls From Credible Sources Influence Voter Turnout or Vote Choice? Evidence From a Randomized Field Experiment.” Journal of Political Marketing 11 (4): 231–45.Google Scholar
Shaw, Daron R., and Gimpel, James G.. 2012. “What if We Randomize the Governor’s Schedule? Evidence on Campaign Appearance Effects From a Texas Field Experiment.” Political Communication 29 (2): 137–59.Google Scholar
Sides, John, and Vavreck, Lynn. 2013. The Gamble. Princeton, NJ: Princeton University Press.Google Scholar
Simon, Paul. 1994. P.S.: The Autobiography of Paul Simon. Boulder, CO: Taylor Trade Publishing.Google Scholar
Smidt, Corwin D. 2017. “Polarization and the Decline of the American Floating Voter.” American Journal of Political Science 61 (2): 365–81.Google Scholar
Sniderman, Paul M. and Theriault, Sean M.. 2004. “The Structure of Political Argument and the Logic of Issue Framing.” In Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change, ed. Saris, Willem E. and Sniderman, Paul M., Princeton, NJ: Princeton University Press: 133–65.CrossRefGoogle Scholar
Strauss, Aaron B. 2009. “Political Ground Truth: How Personal Issue Experience Counters Partisan Biases” PhD thesis, Princeton University.Google Scholar
Ternovski, John, Green, Jennifer, and Kalla, Joshua. 2012. Working America’s 2011 Persuasion Canvass Experiment on SB5 in Ohio. Technical report Analyst Institute.Google Scholar
Vavreck, Lynn. 2009. The Message Matters: The Economy and Presidential Campaigns. Princeton, NJ: Princeton University Press.Google Scholar
Wantchekon, Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World politics 55 (3): 399–422.Google Scholar
Wlezien, Christopher, and Erikson, Robert S.. 2002. “The Timeline of Presidential Election Campaigns.” Journal of Politics 64 (4): 969–93.CrossRefGoogle Scholar
Zaller, John R. 1992. The Nature and Origins of Mass Opinion. New York, NY: Cambridge University Press.Google Scholar
Supplementary material: PDF

Kalla and Broockman supplementary material 1

Appendix

Download Kalla and Broockman supplementary material 1(PDF)
PDF 2.2 MB
Supplementary material: Link

Kalla and Broockman Dataset

Dataset

http://dx.doi.org/10.7910/DVN/SMXWA9
Link
Submit a response

Comments

No Comments have been published for this article.