Skip to main content Accessibility help
×
×
Home

Relaxing the No Liars Assumption in List Experiment Analyses

  • Yimeng Li (a1)

Abstract

The analysis of list experiments depends on two assumptions, known as “no design effect” and “no liars”. The no liars assumption is strong and may fail in many list experiments. I relax the no liars assumption in this paper, and develop a method to provide bounds for the prevalence of sensitive behaviors or attitudes under a weaker behavioral assumption about respondents’ truthfulness toward the sensitive item. I apply the method to a list experiment on the anti-immigration attitudes of California residents and on a broad set of existing list experiment datasets. The prevalence of different items and the correlation structure among items on the list jointly determine the width of the bound estimates. In particular, the bounds tend to be narrower when the list consists of items of the same category, such as multiple groups or organizations, different corporate activities, and various considerations for politician decision-making. My paper illustrates when the full power of the no liars assumption is most needed to pin down the prevalence of the sensitive behavior or attitude, and facilitates estimation of the prevalence robust to violations of the no liars assumption for many list experiment applications.

Copyright

Corresponding author

Footnotes

Hide All

Author’s note: I thank Ines Levin for collecting and sharing with me the data I use in Section 3.1. Previous versions of this research were presented as a poster at the 34th Annual Meeting of the Society for Political Methodology (Polmeth 2017), and in a paper session at the 2018 Annual Meeting of the Midwest Political Science Association (MPSA 2018). I thank R. Michael Alvarez, Jonathan N. Katz, Seo-young Silvia Kim, Ines Levin, Lucas Núñez, Alejandro Robinson-Cortés, Robert Sherman, Matthew Shum, and participants at my poster and paper presentations at Polmeth 2017 and MPSA 2018 for discussions and comments. All errors are my own. Replication data are available in Li (2018) and an R function to implement the proposed bounds is available at https://github.com/lymolympic/list_relaxed_liars.

Contributing Editor: Jeff Gill

Footnotes

References

Hide All
Ahlquist, J. S. 2018. “List Experiment Design, Non-Strategic Respondent Error, and Item Count Technique Estimators.” Political Analysis 26(1):3453.
Ahlquist, J. S., Mayer, K. R., and Jackman, S.. 2014. “Alien Abduction and Voter Impersonation in the 2012 U.S. General Election: Evidence from a Survey List Experiment.” Election Law Journal: Rules, Politics, and Policy 13(4):460475.
Alvarez, R. M., Atkeson, L. R., Levin, I., and Li, Y.. 2019. “Paying Attention to Inattentive Survey Respondents.” Political Analysis 27(2):145162.
Aronow, P. M., Coppock, A., Crawford, F. W., and Green, D. P.. 2015. “Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence.” Journal of Survey Statistics and Methodology 3(1):4366.
Berinsky, A. J., Margolis, M. F., and Sances, M. W.. 2014. “Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self-Administered Surveys.” American Journal of Political Science 58(3):739753.
Blair, G., and Imai, K.. 2012. “Statistical Analysis of List Experiments.” Political Analysis 20(1):4777.
Blair, G., Imai, K., and Lyall, J.. 2014. “Comparing and Combining List and Endorsement Experiments: Evidence from Afghanistan.” American Journal of Political Science 58(4):10431063.
Coffman, K. B., Coffman, L. C., and Marzilli Ericson, K. M.. 2017. “The Size of the LGBT Population and the Magnitude of Antigay Sentiment Are Substantially Underestimated.” Management Science 63(10):31683186.
Coppock, A. 2017. “Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-Representative List Experiment.” Statistics, Politics and Policy 8(1):2940.
Corstange, D. 2009. “Sensitive Questions, Truthful Answers? Modeling the List Experiment with LISTIT.” Political Analysis 17(1):4563.
Eady, G. 2017. “The Statistical Analysis of Misreporting on Sensitive Survey Questions.” Political Analysis 25(2):241259.
Frye, T., Gehlbach, S., Marquardt, K. L., and Reuter, O. J.. 2016. “Is Putin’s Popularity Real? Post-Soviet Affairs 33(1):115.
Glynn, A. N.2010. “What Can We Learn with Statistical Truth Serum? Design and Analysis of the List Experiment.” Unpublished manuscript.
Glynn, A. N. 2013. “What Can We Learn with Statistical Truth Serum? Design and Analysis of the List Experiment.” Public Opinion Quarterly 77(S1):159172.
González-Ocantos, E. et al. . 2012. “Vote Buying and Social Desirability Bias: Experimental Evidence from Nicaragua.” American Journal of Political Science 56(1):202217.
González-Ocantos, E., Kiewiet de Jonge, C., and Nickerson, D. W.. 2015. “Legitimacy Buying: The Dynamics of Clientelism in the Face of Legitimacy Challenges.” Comparative Political Studies 48(9):11271158.
Heerwig, J. A., and McCabe, B. J.. 2009. “Education and Social Desirability Bias: The Case of a Black Presidential Candidate.” Social Science Quarterly 90(3):674686.
Holbrook, A. L., and Krosnick, J. A.. 2010. “Social Desirability Bias in Voter Turnout Reports: Tests Using the Item Count Technique.” Public Opinion Quarterly 74(1):3767.
Imai, K. 2011. “Multivariate Regression Analysis for the Item Count Technique.” Journal of the American Statistical Association 106(494):407416.
Imai, K., Park, B., and Greene, K. F.. 2015. “Using the Predicted Responses from List Experiments as Explanatory Variables in Regression Models.” Political Analysis 23(2):180196.
Imbens, G. W., and Manski, C. F.. 2004. “Confidence Intervals for Partially Identified Parameters.” Econometrica 72(6):18451857.
Kane, J. G., Craig, S. C., and Wald, K. D.. 2004. “Religion and Presidential Politics in Florida: A List Experiment.” Social Science Quarterly 85(2):281293.
Kiewiet de Jonge, C. P. 2015. “Who Lies About Electoral Gifts? Experimental Evidence from Latin America.” Public Opinion Quarterly 79(3):710739.
Kiewiet de Jonge, C. P., and Nickerson, D. W.. 2014. “Artificial Inflation or Deflation? Assessing the Item Count Technique in Comparative Surveys.” Political Behavior 36(3):659682.
Köszegi, B. 2006. “Ego Utility, Overconfidence, and Task Choice.” Journal of the European Economic Association 4(4):673707.
Kramon, E., and Weghorst, K. R.. 2012. “Measuring Sensitive Attitudes in Developing Countries: Lessons from Implementing the List Experiment.” Newsletter of the APSA Experimental Section 3(2):1424.
Kuklinski, J. H., Cobb, M. D., and Gilens, M.. 1997. “Racial Attitudes and the ‘New South’.” The Journal of Politics 59(2):323349.
Lax, J. R., Phillips, J. H., and Stollwerk, A. F.. 2016. “Are Survey Respondents Lying about Their Support for Same-Sex Marriage? Lessons from a List Experiment.” Public Opinion Quarterly 80(2):510533.
Li, Y.2018. “Replication Data for: Relaxing the No Liars Assumption in List Experiment Analyses.” https://doi.org/10.7910/DVN/DFRSKZ, Harvard Dataverse, V1.
Malesky, E. J., Gueorguiev, D. D., and Jensen, N. M.. 2015. “Monopoly Money: Foreign Investment and Bribery in Vietnam, a Survey Experiment.” American Journal of Political Science 59(2):419439.
Meng, T., Pan, J., and Yang, P.. 2014. “Conditional Receptivity to Citizen Participation: Evidence From a Survey Experiment in China.” Comparative Political Studies 50(4):399433.
Oppenheimer, D. M., Meyvis, T., and Davidenko, N.. 2009. “Instructional Manipulation Checks: Detecting Satisficing to Increase Statistical Power.” Journal of Experimental Social Psychology 45(4):867872.
Redlawsk, D. P., Tolbert, C. J., and Franko, W.. 2010. “Voters, Emotions, and Race in 2008: Obama as the First Black President.” Political Research Quarterly 63(4):875889.
Rosenfeld, B., Imai, K., and Shapiro, J. N.. 2016. “An Empirical Validation Study of Popular Survey Methodologies for Sensitive Questions.” American Journal of Political Science 60(3):783802.
Stoye, J. 2009. “More on Confidence Intervals for Partially Identified Parameters.” Econometrica 77(4):12991315.
Streb, M. J., Burrell, B., Frederick, B., and Genovese, M. A.. 2008. “Social Desirability Effects and Support for a Female American President.” Public Opinion Quarterly 72(1):7689.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax

Keywords

Type Description Title
UNKNOWN
Supplementary materials

Li supplementary material
Li supplementary material 1

 Unknown (888 KB)
888 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed