Skip to main content Accessibility help
×
Home

Socially Mediated Internet Surveys: Recruiting Participants for Online Experiments

  • Erin C. Cassese (a1), Leonie Huddy (a2), Todd K. Hartman (a3), Lilliana Mason (a4) and Christopher R. Weber (a5)...

Abstract

The Socially Mediated Internet Survey (SMIS) method is a cost-effective technique used to obtain web-based, adult samples for experimental research in political science. SMIS engages central figures in online social networks to help recruit participants among visitors to these websites, yielding sizable samples for experimental research. We present data from six samples collected using the SMIS method and compare them to those gathered by other sampling approaches such as Amazon's Mechanical Turk. While not representative of the general adult population, our SMIS samples are significantly more diverse than undergraduate convenience samples, not only demographically but also politically. We discuss the applicability of the method to experimental research and its usefulness for obtaining samples of special, politically relevant subpopulations such as political sophisticates and activists. We argue that the diversity of SMIS samples, along with the ability to capture highly engaged citizens, can circumvent questions about the artificiality of political behavior experiments entirely based on student samples and help to document sources of heterogeneous experimental treatment effects.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Socially Mediated Internet Surveys: Recruiting Participants for Online Experiments
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Socially Mediated Internet Surveys: Recruiting Participants for Online Experiments
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Socially Mediated Internet Surveys: Recruiting Participants for Online Experiments
      Available formats
      ×

Copyright

References

Hide All
Abramowitz, Alan. 2010. The Disappearing Center: Engaged Citizens, Polarization, and American Democracy. New Haven, CT: Yale University Press.
Alvarez, R. Michael, Sherman, Robert P., and Van Beselaere, Carla. 2003. “Subject Acquisition for Web-Based Surveys.” Political Analysis 11 (1): 2343.
Berinsky, Adam J., Huber, Gregory A., and Lenz, Gabriel S.. 2012. “Using Mechanical Turk as a Subject Recruitment Tool for Experimental Research.” Political Analysis 20 (3): 351–68.
Berrens, Robert P., Bohara, Alok K., Jenkins-Smith, Hank, Silva, Carol, and Weimer, David L.. 2003. “The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples.” Political Analysis 11 (1): 122.
Birnbaum, Michael H. 2004. “Human Research and Data Collection via the Internet.” Annual Review of Psychology 55: 803–32.
Brace, Paul, Arceneaux, Kevin, Johnson, Martin, and Ulbig, Stacey G.. 2004. “Does State Political Ideology Change over Time?Political Research Quarterly 57 (4): 529–40.
Buhrmeister, Michael, Kwang, Tracy, and Gosling, Samuel D.. 2011. “Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High Quality, Data?Perspectives on Psychological Science 6 (1): 35.
Chang, LinChiat, and Krosnick, Jon A.. 2009. “National Surveys via RDD Telephone Interviewing versus the Internet: Comparing Sample Representativeness and Response Quality.” Public Opinion Quarterly 73 (4): 641–78.
Converse, Philip E. 1964. “The Nature of Belief Systems in Mass Publics.” In Ideology and Discontent, ed. Apter, David E., 206–61. New York: Free Press.
Dillman, Don A., Smyth, Jolene D., and Chrisian, Leah Melani. 2009. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Hoboken, NJ: Wiley.
Druckman, James N., Green, Donald P., Kuklinski, James H., and Lupia, Arthur. 2006. “The Growth and Development of Experimental Research in Political Science.” American Political Science Review 100 (4): 627–35.
Druckman, James N., Green, Donald P., Kuklinski, James H., and Lupia, Arthur. 2011. Cambridge Handbook of Experimental Political Science. New York: Cambridge University Press.
Druckman, James N., and Nelson, Kjersten R.. 2003. “Framing and Deliberation: How Citizens' Conversations Limit Elite Influence.” American Journal of Political Science 47 (4): 729–45.
Gaines, Brian J., and Kuklinski, James H.. 2011. “Experimental Estimation of Heterogeneous Treatment Effects Related to Self-Selection.” American Journal of Political Science 55 (3): 724–36.
Glaser, Jack, Dixit, Jay, and Green, Donald P.. 2002. “Studying Hate Crime with the Internet: What Makes Racists Advocate Racial Violence?Journal of Social Issues 58 (1): 177–93.
Gomez, Brad T., and Wilson, J. Matthew. 2001. “Political Sophistication and Economic Voting in the American Electorate: A Theory of Heterogeneous Attribution.” American Journal of Political Science 45 (4): 899914.
Green, Donald P., and Gerber, Alan S.. 2004. Get Out the Vote! How to Increase Voter Turnout. Washington, DC: Brookings Institution Press.
Hartman, Todd K. 2012. “Toll Booths on the Information Superhighway? Policy Metaphors in the Case of Net Neutrality.” Political Communication 29: 278–98.
Henry, P. J. 2008. “College Sophomores in the Laboratory Redux: Influences of a Narrow Database on Social Psychology's View of the Nature of Prejudice.” Psychological Inquiry 19 (2): 4971.
Imai, Kosuke, Keele, Luke, Tingley, Dustin, and Yamamoto, Teppei. 2011. “Unpacking the Black Box: Learning About Causal Mechanisms from Experimental and Observational Studies.” American Political Science Review 105 (4): 765–89.
Imai, Kosuke, and Strauss, Aaron. 2011. “Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign.” Political Analysis 19 (1): 119.
Kam, Cindy D., Wilking, Jennifer R., and Zechmeister, Elizabeth J.. 2007. “Beyond the ‘Narrow Database’: Another Convenience Sample for Experimental Research.” Political Behavior 29 (4): 415–40.
Kinder, David R. 2007. “Curmudgeonly Advice.” Journal of Communication 57: 155–62.
Kinder, David R., and Palfrey, Thomas R.. 1993. Experimental Foundations of Political Science. Ann Arbor: University of Michigan Press.
Klar, Malte, and Kasser, Tim. 2009. “Some Benefits of Being an Activist: Measuring Activism and its Role in Psychological Well-Being.” Political Psychology 30 (5): 755–77.
Krupnikov, Yanna, and Levine, Adam S.. 2013. “What is the Preferred Population? Comparing ‘College Sophomores’ and Adults in Political Science Experiments.” Working Paper.
Malhotra, Neil, and Krosnick, Jon A.. 2007. “The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples.” Political Analysis 15 (3): 286323.
Mason, Winter, and Suri, Siddharth. 2012. “Conducting Behavioral Research on Amazon's Mechanical Turk.” Behavior Research Methods 44 (1): 123.
Mathy, Robin M., Schillace, Mark, Coleman, Sarah M., and Berquist, Barrie E.. 2002. “Methodological Rigor with Internet Samples: New Ways to Reach Underrepresented Populations.” Cyber-Psychology and Behavior 5 (3): 253–66.
Miller, Joanne M., and Krosnick, John A.. 2004. “Threat as a Motivator of Political Activism: A Field Experiment.” Political Psychology 25 (4): 507–23.
Miller, Patrick R. 2011. “The Emotional Citizen: Emotion as a Function of Political Sophistication.” Political Psychology 32 (4): 575600.
Morton, Rebecca B., and Williams, Kenneth C.. 2010. Experimental Political Science and the Study of Causality: From Nature to the Lab. Cambridge: Cambridge University Press.
Mutz, Diana. 2011. Population-Based Survey Experiments. Princeton, NJ: Princeton University Press.
Orr, Shannon K. 2005. “New Technology and Research: An Analysis of Internet Survey Methodology in Political Science.” PS: Political Science & Politics 38 (2): 263–67.
Paolacci, Gabriele, Chandler, Jesse, and Ipeirotis, Panagiotis G.. 2010. “Running Experiments on Amazon's Mechanical Turk.” Judgment and Decision Making 5 (5): 411–19.
Reips, Ulf-Dietrich. 2000. “The Web Experiment Method: Advantages, Disadvantages, and Solutions.” In Psychological Experiments on the Internet, ed. Birnbaum, Michael H., 89116. San Diego, CA: Academic Press.
Reips, Ulf-Dietrich, and Lengler, Ralph. 2005. “The Web Experiment List: A Web Service for the Recruitment of Participants and Archiving of Internet-Based Experiments.” Behavior Research Methods 37 (2): 287–92.
Rivers, Douglass, and Bailey, Delia. 2009. “Inference from Matched Samples in the 2008 U.S. National Elections.” Proceedings of the Joint Statistical Meetings 627–39.
Sears, David O. 1986. “College Sophomores in the Laboratory: Influences of a Narrow Data Base on Social Psychology's View of Human Nature.” Journal of Personality and Social Psychology 51 (3): 515–30.
Simon, Bernd, and Klandermans, Bert. 2001. “Politicized Collective Identity: A Social Psychological Analysis.” American Psychologist 56 (4): 319–31.
Singla, Parag, and Richardson, Matthew. 2008. “Yes, There is a Correlation: From Social Networks to Personal Behavior on the Web.” Proceedings from the 27th International Conference on the World Wide Web. [Online] http://research.microsoft.com/en-us/um/people/mattri/papers/www2008/socialnetworksandsearch.pdf (accessed January 15, 2013).
Skitka, Linda, and Sargis, Edward G.. 2006. “The Internet as Psychological Laboratory.” Annual Review of Psychology 57: 529–55.
Taber, Charles S., and Lodge, Milton. 2006. “Motivated Skepticism in the Evaluation of Political Beliefs.” American Journal of Political Science 50 (3): 755–69.
Thomas, Emma F., McGarty, Craig, and Mavor, Kennith I.. 2009. “Transforming Apathy into Movement: The Role of Prosocial Emotions in Motivating Action for Social Change.” Personality and Social Psychology Bulletin 13: 310–33.
Warschauer, Mark. 2003. Technology and Social Inclusion: Rethinking the Digital Divide. Cambridge, MA: MIT Press.
Wattenberg, Martin P. 2011. Is Voting for Young People? Upper Saddle River, NJ: Pearson.
Yaeger, David S., Krosnick, John A., Chang, LinChiat, Javitz, Harold S., Levendusky, Mathew S., Simpser, Alberto, and Wang, Rui. 2009. “Comparing the Accuracy of RDD Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples.” Unpublished manuscript. [Online] http://comm.stanford.edu/faculty/krosnick/Mode%2004 (accessed January 15, 2013).
Zaller, John. 1992. The Nature and Origins of Mass Opinion. New York: Cambridge University Press.
Type Description Title
PDF
Supplementary materials

Cassese et al. supplementary material
Appendix

 PDF (224 KB)
224 KB

Socially Mediated Internet Surveys: Recruiting Participants for Online Experiments

  • Erin C. Cassese (a1), Leonie Huddy (a2), Todd K. Hartman (a3), Lilliana Mason (a4) and Christopher R. Weber (a5)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed