Hostname: page-component-77c89778f8-9q27g Total loading time: 0 Render date: 2024-07-20T20:11:50.690Z Has data issue: false hasContentIssue false
  • English
  • Français

Looking Beyond Demographics: Panel Attrition in the ANES and GSS

Published online by Cambridge University Press:  04 January 2017

Laura Lazarus Frankel  and
D. Sunshine Hillygus
Laura Lazarus Frankel
Affiliation:
Department of Political Science, Duke University, 140 Science Drive, Box 90204, Durham, NC 27708
D. Sunshine Hillygus*
Affiliation:
Department of Political Science, Duke University, 140 Science Drive, Box 90204, Durham, NC 27708
*
e-mail: hillygus@duke.edu (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

Longitudinal or panel surveys offer unique benefits for social science research, but they typically suffer from attrition, which reduces sample size and can result in biased inferences. Previous research tends to focus on the demographic predictors of attrition, conceptualizing attrition propensity as a stable, individual-level characteristic—some individuals (e.g., young, poor, residentially mobile) are more likely to drop out of a study than others. We argue that panel attrition reflects both the characteristics of the individual respondent as well as her survey experience, a factor shaped by the design and implementation features of the study. In this article, we examine and compare the predictors of panel attrition in the 2008–2009 American National Election Study, an online panel, and the 2006–2010 General Social Survey, a face-to-face panel. In both cases, survey experience variables are predictive of panel attrition above and beyond the standard demographic predictors, but the particular measures of relevance differ across the two surveys. The findings inform statistical corrections for panel attrition bias and provide study design insights for future panel data collections.

Type
Symposium on Advances in Survey Methodology
Information
Political Analysis , Volume 22 , Issue 3 , Summer 2014 , pp. 336 - 353
Copyright
Copyright © The Author 2014. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors' note: Laura Lazarus Frankel is a Ph.D. candidate in political science at Duke University. D. Sunshine Hillygus is an Associate Professor of Political Science at Duke University. An earlier version of this paper was presented at the Annual Meeting of the American Association of Public Opinion Research. Replication data are available on the Political Analysis Dataverse.

References

Atrostic, B. K., Bates, Nancy, Burt, Geraldine, and Silberstein, Adriana. 2001. Nonresponse in US government household surveys: Consistent measures, recent trends, and new insights. Journal of Official Statistics, Stockholm 17: 209–26.Google Scholar
Bartels, Larry M. 1999. Panel effects in the American National Election Studies. Political Analysis 8: 1–20.Google Scholar
Bartels, Larry M. 2006. Three virtues of panel data for the analysis of campaign effects. In Capturing campaign effects, eds. Brady, Henry E. and Johnston, Richard, 134–63. Ann Arbor: University of Michigan Press.Google Scholar
Behr, Andreas, Bellgardt, Egon, and Rendtel, Ulrich. 2005. Extent and determinants of panel attrition in the European Community Household Panel. European Sociological Review 23: 81–97.Google Scholar
Brehm, John. 1993. The phantom respondents: Opinion surveys and political representation. Ann Arbor: University of Michigan Press.Google Scholar
Burchell, Brendan, and Marsh, Catherine. 1992. The effect of questionnaire length on survey response. Quality & Quantity 26(3): 233–44.Google Scholar
Burden, Barry C. 2000. Voter turnout and the National Election Studies. Political Analysis 8(4): 389.Google Scholar
Campanelli, Pamela, and O'Muircheartaigh, Colm. 1999. Interviewers, interviewer continuity, and panel survey nonresponse. Quality & Quantity 33(1): 59–76.Google Scholar
Clinton, Joshua D. 2001. Panel bias from attrition and conditioning: A case study of the knowledge networks panel. In AAPOR 55th Annual Conference.Google Scholar
Couper, Mick P., and Miller, Peter V. 2009. Web survey methods: Introduction. Public Opinion Quarterly 72: 831–5.Google Scholar
Couper, Mick P., and Groves, Robert M. 1992. The role of the interviewer in survey participation. Survey Methodology 18(2): 263–78.Google Scholar
Davis, Darren W. 1997. Nonrandom measurement error and race of interviewer effects among African Americans. Public Opinion Quarterly 61(1): 183–207.Google Scholar
de Leeuw, Edith Desiree. 1992. Data quality in mail, telephone and face to face surveys. Amsterdam, The Netherlands: TT Publikaties.Google Scholar
Debell, Matthew, Krosnick, Jon A., and Lupia, Arthur. 2010. Methodology report and users guide for the 2008–2009 ANES Panel Study. Technical Report, Stanford University and University of Michigan.Google Scholar
Deng, Yiting, Sunshine Hillygus, D., Reiter, Jerome P., Si, Yajuan, and Zheng, Siyu. 2013. Handling attrition in longitudinal studies: The case for refreshment samples. Statistical Science 28(2): 238–56.Google Scholar
Dennis, J. Micahel, and Li, Rick. 2003. Effects of panel attrition on survey estimates. Paper presented at Annual Meeting of the American Association for Public Opinion Research, Nashville, TN.Google Scholar
Dillman, Don A., Phelps, Glenn, Tortora, Robert, Swift, Karen, Kohrell, Julie, Berck, Jodi, and Messer, Benajmin L. 2009. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR), and the Internet. Social Science Research 38(1): 1–18.Google Scholar
Durrant, Gabriele B., Groves, Robert M., Staetsky, Laura, and Steele, Fiona. 2010. Effects of interviewer attitudes and behaviors on refusal in household surveys. Public Opinion Quarterly 74(1): 1–36.Google Scholar
Fitzgerald, John, Gottschalk, Peter, and Moffitt, Robert. 1998. An analysis of sample attrition in panel data: The Michigan Panel Study of Income Dynamics. Technical Report, National Bureau of Economic Research.Google Scholar
Frankel, Joanne. 1980. Measurement of respondent burden: Study design and early findings. Technical Report. Washington, DC: Bureau of Social Science Research.Google Scholar
Gray, Rebecca, Campanelli, Pamela, Deepchand, Kavita, and Prescott-Clarke, Patricia. 1996. Exploring survey non-response: The effect of attrition on a follow-up of the 1984–85 Health and Life Style Survey. Statistician 45(2): 163–83.Google Scholar
Groves, Robert M., Dillman, Don, Eltinge, John L., and Little, Roderick J.A. 2002. Survey nonresponse. New York: Wiley.Google Scholar
Groves, Robert M., Fowler, Floyd J., Couper, Mick P., Lepkowski, James M., and Singer, Eleanor. 2009. Survey methodology, vol. 561. Hoboken, NJ: John Wiley & Sons.Google Scholar
Groves, Robert M., and Couper, Mick P. 1998. Nonresponse in household interview surveys. New York: Wiley.Google Scholar
Groves, Robert M., Cialdini, Robert B., and Couper, Mick P. 1992. Understanding the decision to participate in a survey. Public Opinion Quarterly 56(4): 475.Google Scholar
Henderson, Michael, Sunshine Hillygus, D., and Tompson, Trevor. 2010. “Sour grapes” or rational voting? Voter decision making among thwarted primary voters in 2008. Public Opinion Quarterly 74(3): 499–529.CrossRefGoogle Scholar
Hill, Daniel H., and Willis, Robert J. 2001. Reducing panel attrition: A search for effective policy instruments. Journal of Human Resources 36(2): 416–38.Google Scholar
Honaker, James, and King, Gary. 2010. What to do about missing values in time-series cross-section data. American Journal of Political Science 54(2): 561–81.Google Scholar
Honaker, J., King, G., and Blackwell, M. 2011. Amelia II: A program for missing data. Journal of Statistical Software 45(7): 1–47.Google Scholar
Imai, Kosuke, King, Gary, and Lau, Olivia. 2006. Zelig: Everyone's statistical software. R package version, 2–7. http://gking.harvard.edu/zelig (accessed February 2013).Google Scholar
Kalton, Graham, Lepkowski, James, Montanari, G. E., and Maligalig, Dalisay. 1990. Characteristics of second-wave nonrespondents in a panel survey. In Proceedings of the Section on Survey Research Methods, American Statistical Association, 462–7.Google Scholar
Keeter, Scott, Kennedy, Courtney, Dimock, Michael, Best, Jonathon, and Craighill, Peyton. 2006. Gauging the impact of growing nonresponse on estimates from a National RDD Telephone Survey. Public Opinion Quarterly 70: 759–79.Google Scholar
Lepkowski, James, and Couper, Mick P. 2002. Nonresponse in the second wave of longitudinal household surveys. In Survey nonresponse, eds. Groves, Robert M., Dillman, Don, Eltinge, John L., and Little, Roderick J. A., 259–72. New York: Wiley.Google Scholar
Lipps, Oliver. 2007. Attrition in the Swiss Household Panel. Methoden-Daten-Analysen 1(1): 45–68.Google Scholar
Lipps, Oliver. 2010. Effects of different incentives on attrition and fieldwork effort in telephone household panel surveys. Survey Research Methods 4(2): 81–90.Google Scholar
Little, Roderick J.A., and Rubin, Donald B. 2002. Statistical analysis with missing data. Hoboken, NJ: Wiley-Interscience.Google Scholar
Loosveldt, Geert, and Carton, Ann. 1997. Evaluation of nonresponse in the Belgian Election Panel Study 91–95. In Proceedings of the Survey Research Methods Section, American Statistical Association.Google Scholar
Loosveldt, Geert, and Carton, Ann. 2001. An empirical test of a limited model for panel refusals. International Journal of Public Opinion Research 13(2): 173.Google Scholar
Loosveldt, Geert, Pickery, Jan, and Billiet, Jaak. 2002. Item nonresponse as a predictor of unit nonresponse in a panel survey. Journal of Official Statistics, Stockholm 18(4): 545–58.Google Scholar
Lynn, Peter, Buck, Nicholas, Burton, Jonathan, Jackle, Annette, and Laurie, Heather. 2005. A review of methodological research pertinent to longitudinal survey design and data collection. Working paper, Institute for Social and Economic Research, University of Essex.Google Scholar
Martin, Elizabeth, Abreu, Denise, and Winters, Franklin. 2001. Money and motive: Effects of incentives on panel attrition in the survey of income and program participation. Journal of Official Statistics, Stockholm 17(2): 267–84.Google Scholar
Olsen, Randall J. 2005. The problem of respondent attrition: Survey methodology is key. Monthly Labor Review 128: 63–71.Google Scholar
Olson, Kristen, and Witt, Lindsey. 2011. Are we keeping the people who used to stay? Changes in correlates of panel survey attrition over time. Social Science Research 40(4): 1037–50.Google Scholar
Pasek, Josh, Tahk, Alexander, Lelkes, Yphtach, Krosnick, Jon A., Keith Payne, B., Akhtar, Omair, and Tompson, Trevor. 2009. Determinants of turnout and candidate choice in the 2008 US presidential election: Illuminating the impact of racial prejudice and other considerations. Public Opinion Quarterly 73: 943–94.Google Scholar
Peress, Michael. 2010. Correcting for survey nonresponse using variable response propensity. Journal of the American Statistical Association 105(492): 1418–30.Google Scholar
Sharp, Laurie M., and Frankel, Joanne. 1983. Respondent burden: A test of some common assumptions. Public Opinion Quarterly 47(1): 36.Google Scholar
Singer, Eleanor. 2002. The use of incentives to reduce nonresponse in household surveys. In Survey nonresponse, eds. Robert, M., Dillman, Don, Eltinge, John L., and Little, Roderick J. A., 163–77. New York: Wiley.Google Scholar
Smith, Tom W., and Son, Jaesok. 2010. An analysis of panel attrition and panel change on the 2006–2008 General Social Survey Panel. Technical Report, NORC, University of Chicago.Google Scholar
Smith, Tom W., and Davis, James A. 1992. The 1992 General Social Survey reinterview. GSS Methodological Report 75. Technical Report, NORC, University of Chicago.Google Scholar
Traugott, Santa, and Morchio, Giovanna. 1990. Assessment of bias due to attrition and sample selection in the ANES 1989 Pilot Study. ANES Technical Report Series, No. NES010154. Technical Report, Center for Political Studies.Google Scholar
Traugott, Santa, and Rosenstone, Steven J. 1994. Panel attrition among the 1990–1992 panel respondents. ANES Technical Report Series, No. NES010163. Technical Report, Center for Political Studies.Google Scholar
Uhrig, S. C. Noah. 2008. The nature and causes of attrition in the British Household Panel Survey. Technical Report, Institute for Social and Economic Research, University of Essex.Google Scholar
Vandecasteele, Leen, and Debels, Annelies. 2007. Attrition in panel data: The effectiveness of weighting. European Sociological Review 23(1): 81–97.Google Scholar
Voogt, Robert J. J. 2005. An alternative approach to correcting response and nonresponse bias in election research. Acta Politica 40(1): 94–116.Google Scholar
Voogt, Robert J.J., and Saris, Willem E. 2003. To participate or not to participate: The link between survey participation, electoral participation, and political interest. Political Analysis 11(2): 164–79.Google Scholar
Watson, Nicole, and Wooden, Mark. 2009. Identifying factors affecting longitudinal survey response. In Methodology of longitudinal surveys, 157–81. West Sussex, UK: John Wiley & Sons.Google Scholar
Wawro, Gregory. 2002. Estimating dynamic panel data models in political science. Political Analysis 10: 25–48.Google Scholar
Zabel, Jeffrey E. 1998. An analysis of attrition in the panel study of income dynamics and the survey of income and program participation with an application to a model of labor market behavior. Journal of Human Resources 33(2): 479–506.Google Scholar