Skip to main content Accessibility help
×
Home

Interviewer effects in food acquisition surveys

  • Ai Rene Ong (a1), Mengyao Hu (a1), Brady T West (a1) and John A Kirlin (a2)

Abstract

Objective

To understand the effects of interviewers on the responses they collect for measures of food security, income and selected survey quality measures (i.e. discrepancy between reported Supplemental Nutrition Assistance Program (SNAP) status and administrative data, length of time between initial and final interview, and missing income data) in the US Department of Agriculture’s National Household Food Acquisition and Purchase Survey (FoodAPS).

Design

Using data from FoodAPS, multilevel models with random interviewer effects were fitted to estimate the variance in each outcome measure arising from effects of the interviewers. Covariates describing each household’s socio-economic status, demographics and experience in taking the survey, and interviewer-level experience were included as fixed effects. The variance components in the outcomes due to interviewers were estimated. Outlier interviewers were profiled.

Setting

Non-institutionalized households in the continental USA (April 2012–January 2013).

Subjects

Individuals (n 14 317) in 4826 households who responded to FoodAPS.

Results

There was a substantial amount of variability in the distributions of the outcomes examined (i.e. time between initial and final interview, reported values for food security, individual income, missing income) among the FoodAPS interviewers, even after accounting for the fixed effects of the household- and interviewer-level covariates and removing extreme outlier interviewers.

Conclusions

Interviewers may introduce error in food acquisition survey data when they are asked to interact with the respondents. Managers of future surveys with similarly complex data collection procedures could consider using multilevel models to adaptively identify and retrain interviewers who have extreme effects on data collection outcomes.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Interviewer effects in food acquisition surveys
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Interviewer effects in food acquisition surveys
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Interviewer effects in food acquisition surveys
      Available formats
      ×

Copyright

Corresponding author

* Corresponding author: Email aireneo@umich.edu

Footnotes

Hide All

The views expressed are those of the authors and should not be attributed to the Economic Research Service or the USDA.

Footnotes

References

Hide All
1. Ricciuto, LE & Tarasuk, VS (2007) An examination of income-related disparities in the nutritional quality of food selections among Canadian households from 1986–2001. Soc Sci Med 64, 186198.
2. Marchioni, DM, Claro, RM, Levy, RB et al. (2011) Patterns of food acquisition in Brazilian households and associated factors: a population-based survey. Public Health Nutr 14, 15861592.
3. Griffin, MK & Sobal, J (2014) Consumer food system participation: a community analysis. Ecol Food Nutr 53, 579595.
4. US Department of Agriculture, Economic Research Service (2017) National Household Food Acquisition and Purchase Survey (FoodAPS). http://www.ers.usda.gov/foodaps (accessed October 2017).
5. West, BT & Blom, AG (2017) Explaining interviewer effects: a research synthesis. J Surv Stat Methodol 5, 175211.
6. Eckman, S & Kreuter, F (2013) Undercoverage rates and undercoverage bias in traditional housing unit listing. Sociol Methods Res 42, 264293.
7. O’Muircheartaigh, C & Campanelli, P (1999) A multilevel exploration of the role of interviewers in survey non-response. J R Stat Soc Ser A Stat Soc 162, 437446.
8. Korbmacher, J & Schroeder, M (2013) Consent when linking survey data with administrative records: the role of the interviewer. Surv Res Methods 7, 115131.
9. Schaeffer, NC, Dykema, J & Maynard, DW (2010) Interviewers and interviewing. In Handbook of Survey Research, 2nd ed., pp. 437470 [PV Marsden and JD Wright, editors] Bingley: Emerald Group Publishing Limited.
10. Liu, M & Stainback, K (2013) Interviewer gender effects on survey responses to marriage-related questions. Public Opin Q 77, 606618.
11. Kim, Y, Choi, Y & Emery, S (2013) Logistic regression with multiple random effects: a simulation study of estimation methods and statistical packages. Am Stat 67, 171182.
12. West, BT & Olson, K (2010) How much of interviewer variance is really non-response error variance? Public Opin Q 74, 10041026.
13. Kreider, B, Pepper, JV, Gundersen, C et al. (2012) Identifying the effects of SNAP (Food Stamps) on child health outcomes when participation is endogenous and misreported. J Am Stat Assoc 107, 958975.
14. Gundersen, C, Kreider, B & Pepper, J (2012) The impact of the National School Lunch Program on child health: a nonparametric bounds analysis. J Econom 166, 7991.
15. Turrell, G, Hewitt, B, Patterson, C et al. (2002) Socioeconomic differences in food purchasing behaviour and suggested implications for diet-related health promotion. J Hum Nutr Diet 15, 355364.
16. Turrell, G & Kavanagh, AM (2006) Socio-economic pathways to diet: modelling the association between socio-economic position and food purchasing behaviour. Public Health Nutr 9, 375383.
17. US Department of Agriculture (2016) Definitions of Food Security. http://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/definitions-of-food-security/ (accessed April 2017).
18. Hu, M, Gremel, GW, Kirlin, JA et al. (2007) Nonresponse and underreporting errors increase over the data collection week based on paradata from the national household food acquisition and purchase survey. J Nutr 147, 964975.
19. American Association for Public Opinion Research (2015) Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions2015_8theditionwithchanges_April2015_logo.pdf (accessed February 2017).
20. US Department of Agriculture (2012) US Adult Food Security Survey Module: Three-Stage Design, With Screeners. http://www.ers.usda.gov/media/8279/ad2012.pdf (accessed November 2017).
21. West, BT, Welch, KB & Galecki, AT (2014) Linear Mixed Models: A Practical Guide Using Statistical Software, 3rd ed. Boca Raton: FL: Chapman Hall/CRC Press.
22. Gumedze, FN & Jackson, D (2011) A random effects variance shift model for detecting and accommodating outliers in meta-analysis. BMC Med Res Methodol 11, 19.
23. West, BT, Kreuter, F & Jaenichen, U (2013) ‘Interviewer’ effects in face-to-face surveys: a function of sampling, measurement error or nonresponse? J Off Stat 29, 277297.
24. Hox, JJ, Moerbeek, M and van de Schoot, R (2018) Multilevel Analysis: Techniques and Applications, 3rd ed. New York/Abingdon: Routledge.

Keywords

Type Description Title
WORD
Supplementary materials

Ong et al. supplementary material
Tables S1-S4

 Word (30 KB)
30 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed