Skip to main content Accessibility help
×
Home

Quota sampling using Facebook advertisements

  • Baobao Zhang (a1), Matto Mildenberger (a2), Peter D. Howe (a3), Jennifer Marlon (a4), Seth A. Rosenthal (a4) and Anthony Leiserowitz (a4)...

Abstract

Researchers in different social science disciplines have successfully used Facebook to recruit subjects for their studies. However, such convenience samples are not generally representative of the population. We developed and validated a new quota sampling method to recruit respondents using Facebook advertisements. Additionally, we published an R package to semi-automate this quota sampling process using the Facebook Marketing API. To test the method, we used Facebook advertisements to quota sample 2432 US respondents for a survey on climate change public opinion. We conducted a contemporaneous nationally representative survey asking identical questions using a high-quality online survey panel whose respondents were recruited using probability sampling. Many results from the Facebook-sampled survey are similar to those from the online panel survey; furthermore, results from the Facebook-sampled survey approximate results from the American Community Survey (ACS) for a set of validation questions. These findings suggest that using Facebook to recruit respondents is a viable option for survey researchers wishing to approximate population-level public opinion.

Copyright

Corresponding author

*Corresponding author. Email: baobao.zhang@yale.edu

References

Hide All
Berinsky, AJ, Huber, GAandLenz, GS (2012) Evaluating Online Labor Markets for Experimental Research: Amazon. com’s Mechanical Turk. Political Analysis 20(3), 351368.
Bond, RandMessing, S (2015) Quantifying Social Media’s Political Space: Estimating Ideology from Publicly Revealed Preferences on Facebook. American Political Science Review 109(01), 6278.
Broockman, DEandGreen, DP (2014) Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments. Political Behavior 36(2), 263289.
Hirano, S, Lenz, GS, Pinkovskiy, MandSnyder, JM (2015) Voter Learning in State Primary Elections. American Journal of Political Science 59(1), 91108.
Huff, CandTingley, D (2015) “Who Are These People?” Evaluating the Demographic Characteristics and Political Preferences of Mturk Survey Respondents. Research & Politics 2(3), 2053168015604648.
Jäger, K (2017) The potential of online sampling for studying political activists around the world and across time. Political Analysis 25(3), 329343.
Kapp, JM, Peters, CandOliver, DP (2013) Research Recruitment Using Facebook Advertising: Big Potential, Big Challenges. Journal of Cancer Education 28(1), 134137.
Kosinski, M, Matz, SC, Gosling, SD, Popov, VandStillwell, D (2015) Facebook as a Research Tool for the Social Sciences: Opportunities, Challenges, Ethical Considerations, and Practical Guidelines. American Psychologist 70(6), 543.
Kosinski, M, Stillwell, DandGraepel, T (2013) Private Traits and Attributes Are Predictable from Digital Records of Human Behavior. Proceedings of the National Academy of Sciences 110(15), 58025805.
Ramo, DEandProchaska, JJ (2012) Broad Reach and Targeted Recruitment Using Facebook for an Online Survey of Young Adult Substance Use. Journal of Medical Internet Research 14(1), e28.
Ramo, DE, Rodriguez, TM, Chavez, K, Sommer, MJandProchaska, JJ (2014) Facebook Recruitment of Young Adult Smokers for a Cessation Trial: Methods, Metrics, and Lessons Learned. Internet Interventions 1(2), 5864.
Ryan, TJ (2012) What Makes Us Click? Demonstrating Incentives for Angry Discourse with Digital-Age Field Experiments. The Journal of Politics 74(4), 11381152.
Samuels, DJandZucco, C (2013) Using Facebook as a Subject Recruitment Tool for Survey-Experimental Research. Working Paper, available at SSRN 2101458.
Samuels, DJandZucco, C (2014) The Power of Partisanship in Brazil: Evidence from Survey Experiments. American Journal of Political Science 58(1), 212225.
Sances, MW (2017) Ideology and Vote Choice in US Mayoral Elections: Evidence from Facebook Surveys. Political Behavior 40(3), 737762.
Sances, MW (2018) Missing the Target? Using Surveys to Validate Social Media Ad Targeting. Working Paper, Unpublished paper.
Santoso, LP, Stein, RandStevenson, R (2016) Survey Experiments with Google Consumer Surveys: Promise and Pitfalls for Academic Research in Social Science. Political Analysis 24(3), 356373.
Smith, AandAnderson, M (2018) Social Media Use in 2018. Survey Report, Washington, DC.
Stewart, N, Ungemach, C, Harris, AJ, Bartels, DM, Newell, BR, Paolacci, GandChandler, J (2015) The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers. Judgment and Decision Making 10(5), 479.
Youyou, W, Kosinski, MandStillwell, D (2015) Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans. Proceedings of the National Academy of Sciences 112(4), 10361040.

Related content

Powered by UNSILO
Type Description Title
PDF
Supplementary materials

Zhang et al. supplementary material
Zhang et al. supplementary material

 PDF (460 KB)
460 KB
UNKNOWN
Supplementary materials

Zhang et al. Dataset
Dataset

 Unknown

Quota sampling using Facebook advertisements

  • Baobao Zhang (a1), Matto Mildenberger (a2), Peter D. Howe (a3), Jennifer Marlon (a4), Seth A. Rosenthal (a4) and Anthony Leiserowitz (a4)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.