Skip to main content Accessibility help

A Convenient Solution: Using MTurk To Sample From Hard-To-Reach Populations

  • Nicholas A. Smith (a1), Isaac E. Sabat (a2), Larry R. Martinez (a1), Kayla Weaver (a1) and Shi Xu (a1)...


We agree with Landers and Behrend's (2015) proposition that Amazon's Mechanical Turk (MTurk) may provide great opportunities for organizational research samples. However, some groups are characteristically difficult to recruit because they are stigmatized or socially disenfranchised (Birman, 2005; Miller, Forte, Wilson, & Greene, 2006; Sullivan & Cain, 2004; see Campbell, Adams, & Patterson, 2008, for a review). These groups may include individuals who have not previously been the focus of much organizational research, such as those of low socioeconomic status; individuals with disabilities; lesbian, gay, bisexual, or transgender (LGBT) individuals; or victims of workplace harassment. As Landers and Behrend (2015) point out, there is an overrepresentation of research using “Western, educated, industrialized, rich, and democratic” participants. It is important to extend research beyond these samples to examine workplace phenomena that are specific to special populations. We contribute to this argument by noting the particular usefulness that MTurk can provide for sampling from hard-to-reach populations, which we characterize as groups that are in the numerical minority in terms of nationwide representation. To clarify, we focus our discussion on populations that are traditionally hard to reach in the context of contemporary organizational research within the United States.


Corresponding author

Correspondence concerning this article should be addressed to Nicholas Smith, School of Hospitality Management, The Pennsylvania State University, 201 Mateer Building, University Park, PA 16802. E-mail:


Hide All
Birman, D. (2005). Ethical issues in research with immigrants and refugees. In Trimble, J. E. & Fisher, C. B. (Eds.), The handbook of ethical research with ethnocultural populations and communities (pp. 155178). Thousand Oaks, CA: Sage.
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 35. doi:10.1177/1745691610393980
Campbell, R., Adams, A. E., & Patterson, D. (2008). Methodological challenges of collecting evaluation data from traumatized clients/consumers: A comparison of three methods. American Journal of Evaluation, 29, 369381. doi:10.1177/1098214008320736
Carr, A. (2014). An exploration of Mechanical Turk as a feasible recruitment platform for cancer survivors (Unpublished undergraduate honors theses, Paper 59). Retrieved from
Eisenberg, M., & Wechsler, H. (2003). Substance use behaviors among college students with same-sex and opposite-sex experience: Results from a national study. Addictive Behaviors, 28, 899913. doi:10.1016/S0306-4603(01)00286-6
Feitosa, J., Joseph, D. L., & Newman, D. A. (2015). Crowdsourcing and personality measurement equivalence: A warning about countries whose primary language is not English. Personality and Individual Differences, 75, 4752. doi:10.1016/j.paid.2014.11.017
Gonzalez, J. A., & DeNisi, A. S. (2009). Cross-level effects of demography and diversity climate on organizational attachment and firm effectiveness. Journal of Organizational Behavior, 30, 2140. doi:10.1002/job.498
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93104. doi:10.1037/0003-066X.59.2.93
Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Industrial and Organizational Psychology: Perspectives on Science and Practice. Advance online publication. doi:10.1017/iop.2015.14
Law, C. L., Martinez, L. R., Ruggs, E. N., Hebl, M. R., & Akers, E. (2011). Trans-parency in the workplace: How the experiences of transsexual employees can be improved. Journal of Vocational Behavior, 79, 710723. doi:10.1016/j.jvb.2011.03.018
Lenox, R. A., & Subich, L. M. (1994). The relationship between self-efficacy beliefs and inventoried vocational interests. The Career Development Quarterly, 42, 302313. doi:10.1002/j.2161-0045.1994.tb00514.x
Levine, S., Ancill, R. J., & Roberts, A. P. (1989). Assessment of suicide risk by computer-delivered self-rating questionnaire: Preliminary findings. Acta Psychiatrica Scandinavica, 80, 216220. doi:10.1111/j.1600-0447.1989.tb01330.x
Locke, S. D., & Gilbert, B. O. (1995). Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats. Journal of Social Behavior & Personality, 10, 255263. Retrieved from
McCormack, M. (2014). Innovative sampling and participant recruitment in sexuality research. Journal of Social and Personal Relationships, 31, 475481. doi:10.1177/0265407514522889
McKay, P. F., Avery, D. R., & Morris, M. A. (2009). A tale of two climates: Diversity climate from subordinates’ and managers’ perspectives and their role in store unit sales performance. Personnel Psychology, 62, 767791. doi:10.1111/j.1744-6570.2009.01157.x
Miller, R. L., Forte, D., Wilson, B. D. M., & Greene, G. J. (2006). Protecting sexual minority youth from research risks: Conflicting perspectives. American Journal of Community Psychology, 37, 341348. doi:10.1007/s10464-006-9053-4
Papa, A., Lancaster, N. G., & Kahler, J. (2014). Commonalities in grief responding across bereavement and non-bereavement losses. Journal of Affective Disorders, 161, 136143. doi:10.1016/j.jad.2014.03.018
Skitka, L. J., & Sargis, E. G. (2005). Social psychological research and the Internet: The promise and peril of a new methodological frontier. In Amichai-Hamburger, Y. (Ed.), The social net: The social psychology of the Internet (pp. 126). Oxford, United Kingdom: Oxford University Press.
Storey, D. J. (1994). Understanding the small business sector. New York, NY: Routledge.
Sullivan, C. M., & Cain, D. (2004). Ethical and safety considerations when obtaining information from or about battered women for research purposes. Journal of Interpersonal Violence, 19, 603618. doi:10.1177/0886260504263249
Tenenbaum, R. Z., Byrne, C. J., & Dahling, J. J. (2014). Interactive effects of physical disability severity and age of disability onset on RIASEC self-efficacies. Journal of Career Assessment, 22, 274289. doi:10.1177/1069072713493981
Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867873. doi:10.1126/science.280.5365.867
U.S. Census Bureau. (2011). Statistics of U.S. businesses (SUSB) main [Data file]. Retrieved from
Vaughn, A. A., Cronan, S. B., & Beavers, A. J. (2015). Resource effects on in-group boundary formation with regard to sexual identity. Social Psychological and Personality Science, 6, 292299. doi:10.1177/1948550614559604

A Convenient Solution: Using MTurk To Sample From Hard-To-Reach Populations

  • Nicholas A. Smith (a1), Isaac E. Sabat (a2), Larry R. Martinez (a1), Kayla Weaver (a1) and Shi Xu (a1)...


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed