Obesity is a major public health concern. About one-third of US adults and 17 % of children and adolescents are obese( Reference Flegal, Caroll and Ogden 1 , Reference Ogden, Carroll and Curtin 2 ). Epidemiological studies indicate an association between increased fruit and vegetable consumption and reduced long-term obesity risk( Reference He, Hu and Colditz 3 – Reference Champagne, Broyles and Moran 9 ) as well as reduced risk of heart disease and some cancers( Reference Dauchet, Amouyel and Hercberg 10 – Reference Brock, Ke and Gridley 15 ). Nevertheless, relatively few Americans consume the recommended amounts of fruits and vegetables( Reference Guenther, Dodd and Reedy 16 , Reference Briefel and Johnson 17 ). This is particularly true among low-income families and participants in federal nutrition assistance programmes like the Supplemental Nutrition Assistance Program (SNAP), populations least likely to meet recommended fruit and vegetable intake levels( Reference Casagrande, Wang and Anderson 18 – Reference Kamphuis, Giskes and de Bruijn 20 ).
In response to this growing public health crisis, some have advocated for targeted incentive or voucher programmes to encourage fruit and vegetable purchases( 21 – Reference Lin and Guthrie 23 ). Such programmes make nutritious foods like fresh fruits and vegetables more affordable compared with less nutritious choices by affecting relative prices and enhancing overall purchasing power( 21 ). Farmers’ markets in particular have been targeted by a number of incentive programmes as ideal sources for fresh fruits and vegetables within urban and low-income settings( Reference Ver Ploeg, Breneman and Farrigan 24 – Reference Briggs, Fisher and Lott 26 ).
The Health Bucks initiative, introduced in 2005, is a pioneering example of a farmers’ market incentive programme. Funded by the New York City Department of Health and Mental Hygiene (DOHMH) and the Human Resources Administration, the Health Bucks programme’s stated objectives are to: (i) increase consumption of fresh fruits and vegetables and thereby promote healthful diets; and (ii) expand access to locally grown produce at farmers’ markets in low-income neighbourhoods. The programme operates through three District Public Health Offices (DPHO) established by DOHMH to reduce health inequalities in designated high-need New York City neighbourhoods (the South Bronx, North and Central Brooklyn, and East and Central Harlem; hereafter referred to as ‘DPHO neighbourhoods’).
‘Health Bucks’ are $US 2 coupons redeemable for the purchase of fresh fruits and vegetables at participating farmers’ markets; in 2011, sixty-five New York City farmers’ markets accepted Health Bucks as a form of payment. The majority of Health Bucks are distributed directly to shoppers at farmers’ markets in DPHO neighbourhoods as a SNAP incentive. Participating markets dispense one $US 2 Health Buck coupon for every $US 5 in benefits spent by SNAP participants using their electronic benefit transfer (EBT) cards, with no ceiling amount. In 2011, nearly 90 000 Health Bucks were distributed as incentives to SNAP participants, with a 93 % redemption rate. In addition, the DOHMH issues Health Bucks to community-based organizations in DPHO neighbourhoods, which distribute the coupons directly to clients, often as an incentive to attend nutrition workshops or health and fitness programmes. Although there is no specific income-based eligibility requirement for Health Bucks distributed via this mechanism, most participating community-based organizations served predominantly low-income clients. This distribution method promotes the programme to neighbourhood residents and encourages more first-time shoppers to purchase produce from their local farmers’ markets. In 2011, more than 28 000 Health Bucks were distributed through over 150 community-based organizations in DPHO neighbourhoods, with redemption rates topping 70 %( 27 ). Detailed information on Health Bucks history and implementation has been published elsewhere( Reference Payne, Wethington and Olsho 28 ).
Although incentive programmes at farmers’ markets are growing as an approach to increase access to fresh fruits and vegetables, little is known about their effects on fruit and vegetable spending and intake( Reference McCormack, Laska and Larson 29 – Reference Blair, Giesecke and Sherman 31 ). The current paper presents results from a mixed-methods evaluation to assess the effectiveness of Health Bucks in increasing access to and awareness of farmers’ markets, and increasing purchase and consumption of fruits and vegetables.
Methods
Study design
Our mixed-methods evaluation employed descriptive analysis of cross-sectional survey data collected directly from New York City farmers’ market shoppers and residents of DPHO neighbourhoods and a quasi-experimental analysis of differential time trends in consumption before and after programme introduction using secondary Community Health Survey (CHS) data.
Data sources
The evaluation results reported here employed three data sources (two primary and one secondary). We provide a brief overview of data sources below; further detail is available by request from the corresponding author and survey instruments are provided as part of the online Health Bucks Evaluation Toolkit( Reference Olsho, Baronberg and Abel 32 ). The study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Abt Associates Institutional Review Board and the New York City DOHMH Institutional Review Board. Verbal informed consent was obtained from all subjects. Verbal consent was witnessed and formally recorded by interviewers.
Farmers’ market shopper survey
Trained interviewers conducted intercept surveys on site with 2287 shoppers at eighty-six New York City farmers’ markets during the 2010 farmers’ market season (July–November). Interview sites included all Health Bucks markets (1416 shoppers at forty-six markets in DPHO neighbourhoods) plus a random sample of non-participating markets (871 shoppers at forty markets outside DPHO neighbourhoods). Interviewers were stationed at high-traffic locations, approaching all eligible shoppers over age 18 years who had made market purchases that day, offering a round-trip MetroCard ($US 4·50 value) as incentive for completion. Interviews were conducted in English and Spanish. The final sample reflects a completion rate of 72 % of shoppers approached, excluding incompletes due to screen-outs, refusals and language barriers. We asked shoppers about market purchases that day, and access to and consumption of fruits and vegetables from farmers’ markets and other sources. In Health Bucks markets only, we asked about Health Bucks knowledge and experiences. Supplemental Table 1 (see online supplementary material) provides demographic data on farmers’ market shopper survey respondents.
Neighbourhood resident survey
Near the end of the 2010 farmers’ market season, we conducted a random-digit-dial telephone survey of 1025 residents stratified by DPHO neighbourhood, with about a third of respondents in each of the three neighbourhoods. A zip exchange analysis was used to identify telephone exchanges in each neighbourhood; cell phone exchanges were not included. Eligible respondents were residents of DPHO-neighbourhood zip codes aged 18 years and above who were the primary food shopper for their household. Interviews were conducted in English and Spanish. The overall response rate was 39 %. The survey’s objective was to obtain information on a representative sample of DPHO-neighbourhood households, to characterize overall programme knowledge and attitudes and provide a point of comparison for the shopper survey. Respondents reported on experiences with farmers’ markets, access to and consumption of fruits and vegetables from farmers’ markets and other sources, and Health Bucks knowledge and experiences. Supplemental Table 2 (see online supplementary material) provides demographic data on neighbourhood resident survey respondents.
Community health survey
The New York City Community Health Survey (CHS) is an annual cross-sectional telephone survey conducted for DOHMH by Abt SRBI Inc. Its objective is to provide representative data on the health of New Yorkers at the neighbourhood, borough and citywide level, including estimates on prevalence of chronic disease and behavioural risk factors( 33 ). Although many CHS questions are repeated annually, others are asked only on a one-time or periodic basis, to allow timely survey coverage of a wider breadth of topics and issues of emerging interest without undue respondent burden. Our analysis uses a consumption measure (total servings of fruits and vegetables eaten on the previous day) that was collected in four survey years: 2002 and 2004, prior to the 2005 implementation of Health Bucks, and 2008 and 2009, after the programme was well established. The overall response rate ranged from 29 % to 36 % across the four survey years. Across all four years, there were 35 606 individual CHS observations.
Analyses
We synthesized available information from each data source in this mixed-methods evaluation on three outcome areas of interest (awareness and access, purchasing, and consumption) to provide a range of evidence on programme impacts. Our primary data collection activities allowed us to directly observe associations between Health Bucks exposure and outcomes, but because these data were collected at a single point in time, ability to draw causal inference is limited. The secondary CHS data are repeated over time both before and after Health Bucks implementation, providing a more robust basis for causal inference by allowing us to compare trends in DPHO and non-DPHO neighbourhoods after programme introduction. However, because we cannot directly observe respondent use of Health Bucks, residence in a DPHO neighbourhood is the nearest available CHS proxy for Health Bucks exposure. In this sense, our primary and secondary data analyses represent useful complements, together providing a more complete perspective on programme exposure and impacts.
Cross-sectional survey analyses
We used questions from the farmers’ market shopper and neighbourhood resident surveys to construct measures of respondent exposure to Health Bucks. Neighbourhood residents were categorized by whether they had ever shopped at a farmers’ market; farmers’ market shoppers were categorized by whether the interview occurred at a market accepting Health Bucks or a non-participating market. Respondents were then further categorized by prior awareness of Health Bucks (ever heard of Health Bucks); any previous Health Bucks use (ever used Health Bucks to make purchases); and current Health Bucks use (use on the day of the interview for farmers’ market shoppers or during the current farmers’ market season for neighbourhood residents), with each subsequent sub-categorization representing a higher level of programme exposure (see online supplementary material, Supplemental Fig. 1 and Supplemental Fig. 2). For each set of subgroups, we report χ 2 tests for differences in categorical outcome measures and t tests for differences in means for continuous outcome measures, along with P values indicating statistically significant differences. Shopper survey analyses incorporate sampling weights to account for probability of market selection and neighbourhood survey analyses incorporate sampling weights to account for household selection probability. We separately report results for the respondent sub-sample who participated in SNAP, to enable comparisons by Health Bucks exposure level within a more homogeneous sub-population. Regression adjustment for differences in demographic characteristics (age, sex and race/ethnicity) did not materially alter findings; we therefore report unadjusted means and percentages here for simplicity of interpretation.
Difference-in-differences analysis of community health survey data
Finally, we estimated programme impacts on trends in fruit and vegetable consumption using individual-level CHS data. The analysis proceeds on the following intuition. If Health Bucks increases fruit and vegetable consumption, we would expect a more favourable trend in DPHO neighbourhoods after programme introduction. However, simple pre–post comparisons within DPHO neighbourhoods could be confounded by broader citywide factors influencing consumption over the same time period. We thus use individuals residing in non-DPHO New York City neighbourhoods as an outside comparison group, since we would not expect Health Bucks to influence consumption trends in these areas. Any differences in trends after programme introduction in DPHO neighbourhoods as compared with non-DPHO neighbourhoods are interpreted as Health Bucks programme impacts. This approach is known in the evaluation literature as a difference-in-differences design.
Specifically, we used linear regression to estimate the difference in fruit and vegetable consumption trends in DPHO neighbourhoods after Health Bucks implementation. Formally, we have self-reported data on fruit and vegetable consumption, y, of individuals, i, in neighbourhoods, g, in time period, t:
$$y_{{g,i,t}} =\alpha {\plus}H_{{g,t}} \beta {\plus}M_{g} \mu {\plus}T_{t} \tau {\plus}X_{{i,t}} \gamma {\plus}W_{{g,t}} \delta {\plus}{\varepsilon}_{{g,i,t}} ,$$
where H is an indicator variable equal to 1 in DPHO neighbourhoods after Health Bucks introduction, and 0 otherwise; M is an indicator variable equal to 1 in DPHO neighbourhoods in all years, and 0 otherwise; and T is a series of indicator variables for survey year (2004, 2008 or 2009, with 2002 as the excluded year). β is the key parameter of interest, representing programme impact as the difference in fruit and vegetable consumption in DPHO neighbourhoods after Health Bucks began, adjusted for time-invariant neighbourhood-level effects captured by μ and citywide trends by year captured by τ. In some specifications we additionally include X, a vector of individual-level characteristics influencing consumption (sex, race/ethnicity, age group, employment, educational attainment); and W, a vector of time-varying neighbourhood-level characteristics (presence of food retail establishments from Census zip code business pattern data, proportion of neighbourhood residents below 200 % of the poverty line). We account for correlation in standard error ε across individuals residing in the same neighbourhood by clustering at the neighbourhood level( Reference Bertrand, Duflo and Mullainathan 34 ). Finally, we incorporate appropriate sampling weights and techniques to account for survey non-response and the complex CHS survey design, so results can be considered representative of New York City households with landline telephone service.
Results
Awareness of and perceived access to farmers’ markets
Sixty-three per cent of neighbourhood residents, and 68 % of SNAP participants in the neighbourhood resident survey, reported that there was a farmers’ market in their neighbourhood. For both the full sample and the SNAP participant sub-sample, farmers’ market awareness rose with Health Bucks exposure: those who had used Health Bucks during the current season were most likely to report that there was a farmers’ market in their neighbourhood, followed by those who had ever used Health Bucks, those who had heard of Health Bucks (whether or not they had actually used them) and those who said they sometimes shopped at a farmers’ market.
Neighbourhood resident survey respondents were also asked how long it would take to walk to the nearest source of fresh fruits and vegetables (under 5 min, 5–10 min or over 10 min), to assess perceived geographic accessibility. There were no statistically significant differences in reported walking time by Health Bucks exposure.
Purchasing patterns
Frequency of farmers’ market shopping
Figure 1 shows self-reported frequency of shopping at farmers’ markets by DPHO neighbourhood residents and shoppers at Health Bucks markets, with responses stratified by programme exposure level. The χ 2 tests indicated that greater Health Bucks exposure was significantly associated with more frequent farmers’ market shopping for both groups (P=0·001 in both cases). In particular, among neighbourhood residents, 24 % who had never heard of Health Bucks said they shopped at farmers’ markets once per week or more, as compared with 32 % who had heard of Health Bucks, 41 % who had used Health Bucks and 49 % who had used Health Bucks during 2010. At the other end of the scale, 35 % of neighbourhood survey respondents who had never heard of Health Bucks said they never shopped at farmers’ markets, as compared with 23 % who had heard of Health Bucks, 8 % who had used Health Bucks and 8 % who had used Health Bucks this season. Greater programme exposure was similarly significantly associated with more frequent farmers’ market visits for farmers’ market shoppers, although the distribution of shopper responses was skewed towards a higher frequency than in the resident survey.
Fig. 1 Frequency of shopping at farmers’ markets, by level of experience with Health Bucks (
, never heard of Health Bucks; 
, heard of Health Bucks; 
, ever used Health Bucks; 
, used Health Bucks in 2010 season (a)/used Health Buck that day (b)), among (a) DPHO neighbourhood residents† (n 997) and Health Bucks farmers’ market shoppers‡ (n 1416). †Neighbourhood resident survey analytic sample excludes twenty-eight ‘don’t know’ or ‘refused’ responses; ‡note that since all farmers’ market shopper surveys were conducted on site at the farmers’ market, ‘never shopped at a farmers’ market’ was not provided as a possible response category for this respondent group. All differences in frequencies across respondent subgroups by level of Health Bucks exposure were statistically significant (P=0·001, χ
2 test). (DPHO, District Public Health Office)
In both respondent groups, the majority directly credited Health Bucks for increasing frequency of farmers’ market visits. Fifty-four per cent of Health Bucks users at markets accepting Health Bucks strongly agreed that ‘I shop at farmers’ markets more often because of Health Bucks’ and another 18 % somewhat agreed. Among neighbourhood residents, 39 % of Health Bucks users strongly agreed and another 16 % somewhat agreed.
Purchase amounts
In addition to crediting Health Bucks for more frequent market visits, respondents reported that Health Bucks induced them to spend more at farmers’ markets overall. Fifty-seven per cent of Health Bucks users surveyed on site strongly agreed that ‘I buy more at farmers’ markets because of Health Bucks’ and another 15 % somewhat agreed. Among Health Bucks users residing in DPHO neighbourhoods, 18 % strongly agreed and 37 % somewhat agreed.
On site at farmers’ markets, most Health Bucks users also agreed that ‘I spend more in Food Stamps (aka SNAP or EBT benefits) at farmers’ markets because of Health Bucks’, with 39 % strongly agreeing and 18 % somewhat agreeing. For Health Bucks users in our neighbourhood resident survey, however, only 22 % strongly agreed and 15 % somewhat agreed.
Purchase types
Table 1 presents the percentage of farmers’ market shoppers surveyed on site reporting purchases of fruits and vegetables at the market on the interview day, and the percentage reporting purchases of other items, broken down by SNAP participation and programme exposure.
Table 1 Percentage of farmers’ market shoppers reporting purchases of fruits and vegetables and other items at the market that day, all shoppers and SNAP participants, by Health Bucks exposure levels
SNAP, Supplemental Nutrition Assistance Program.
Significant difference across respondent subgroups by level of Health Bucks exposure (χ 2 test): (*)P<0·10, *P<0·05, **P<0·01.
† Includes jams/juices, bread, cheese, meats/fish, baked goods, and other unspecified items. Respondents could report purchases of both fruits and vegetables and other items.
The likelihood of a fruit or vegetable purchase at the market that day (including both purchases with Health Bucks and purchases with other forms of payment) increased with Health Bucks exposure. Among all respondents, 99 % of those who had used Health Bucks that day had purchased fruits or vegetables,Footnote * as compared with only 95 % of shoppers who had not used Health Bucks that day (P=0·04). At successively lower degrees of Health Bucks exposure, 98 % of respondents who had ever used Health Bucks reported a fruit or vegetable purchase that day (compared with 95 % of those who never used Health Bucks, P=0·02), 97 % who had heard of Health Bucks reported a fruit or vegetable purchase that day (compared with 95 % of those who had never heard of Health Bucks, P=0·06) and 95 % of shoppers at Health Bucks markets reported a fruit or vegetable purchase that day (compared with 91 % of shoppers at non-Health Bucks markets P<0·001). Patterns were similar among the SNAP participant sub-sample. For purchases of other items (e.g. breads, jams/juices, cheese, etc.), however, this trend was reversed, with likelihood of purchase decreasing with programme exposure.
Fruit and vegetable consumption
Table 2 shows results from the neighbourhood resident and shopper surveys on the mean servings of fruits and vegetables consumed the prior day and the percentage who reported eating more fruits and vegetables now than one year ago.
Table 2 Self-reported fruit and vegetable consumption from neighbourhood resident and farmers’ market shopper surveys, all respondents and SNAP participants, by level of experience with farmers’ markets and Health Bucks
SNAP, Supplemental Nutrition Assistance Program.
Significant difference across respondent subgroups by level of Health Bucks exposure (χ 2 test): (*)P<0·10, *P<0·05, **P<0·01.
† Neighbourhood resident survey analytic sample excludes twenty-eight ‘don’t know’ or ‘refused’ responses.
Among neighbourhood residents, farmers’ market shoppers reported consuming a higher number of fruit and vegetable servings on the previous day, both in the full sample and in the SNAP sub-sample. However, there was no consistent pattern in consumption associated with neighbourhood resident knowledge or use of Health Bucks. Similarly, we did not observe higher levels of fruit and vegetable consumption by Health Bucks awareness and use among farmers’ market shoppers surveyed on-site. Furthermore, shoppers at Health Bucks markets reported fewer servings consumed than shoppers at non-Health Bucks markets.
When asked to assess fruit and vegetable consumption today v. consumption one year ago, however, Health Bucks users in both cross-sectional surveys were more likely to report increased consumption. Furthermore, 64 % of DPHO neighbourhood residents who had heard of Health Bucks and 81 % of shoppers at Health Bucks markets somewhat or strongly agreed that Health Bucks helped them to eat more fruits and vegetables.
In addition to correlational findings from our cross-sectional surveys, for fruit and vegetable consumption we report difference-in-differences findings based on CHS data. Table 3 shows estimated regression results for models including and excluding individual- and neighbourhood-level characteristics. We found no evidence of differentially larger increases in self-reported fruit and vegetable consumption in DPHO neighbourhoods as compared with other neighbourhoods. In particular, the positive and significant parameter τ 2009 shows that average fruit and vegetable consumption increased in all New York City neighbourhoods from 2002 to 2009. However, the impact parameter β was not statistically significant in any specification, indicating that this increase was not differentially larger in DPHO neighbourhoods after Health Bucks implementation.
Table 3 Difference-in-differences estimates of impacts on self-reported servings of fruits and vegetables on previous day, Community Health Survey respondents, 2002–2009
DPHO, District Public Health Office; N, no; Y, yes.
*Significant at 5 %; **significant at 1 %; ***significant at 1 %.
† Specifications 1–4 differ by inclusion/exclusion of individual characteristics (X) and time-varying neighbourhood characteristics (W).
Discussion
The present study’s goal was to evaluate the effectiveness of Health Bucks in improving access to and awareness of farmers’ markets and in increasing purchase and consumption of fruits and vegetables among targeted low-income populations.
While we found that greater Health Bucks exposure was associated with greater awareness of farmers’ markets, we did not find statistically significant increases in perceived access to farmers’ markets, as measured by reported walking distance to the nearest market. Note, however, that just over 50 % of farmers’ market shoppers we interviewed reported that the market where they were surveyed was over 10 min away. It may be that our simple walking time metric may not capture all important dimensions of ‘access’ in this context, as shoppers seem to be willing to travel longer distances to reach farmers’ markets if necessary. Prior research has indicated that incentive programmes like Health Bucks can improve access to farmers’ markets by removing financial and logistical burdens for markets and participants in food assistance programmes( Reference Freedman, Whiteside and Brandt 35 – Reference Evans, Jennings and Smiley 37 ).
Increased Health Bucks exposure was also found to be associated with increased self-reported frequency and amounts of farmers’ market purchases. Additionally, farmers’ market shoppers with greater exposure to Health Bucks reported a greater likelihood of having purchased fruits and vegetables at the market that day. While our analysis focused on effects of the programme on individual spending and shopping patterns, this finding is consistent with related work examining effects on spending from the perspective of the farmers’ market. For example, farmers’ market managers and vendors agreed that they made more money at the market and sold more fruits and vegetables due to Health Bucks( Reference Payne, Wethington and Olsho 28 ), and farmers’ markets accepting Health Bucks coupons averaged higher daily SNAP sales than markets not accepting the coupons( Reference Baronberg, Dunn and Nonas 38 ). Freedman and colleagues similarly found increased use of all forms of food assistance at the farmers’ market associated with introduction of an incentive programme( Reference Freedman, Mattison-Faye and Kassandra 39 ).
Finally, we found mixed evidence regarding the effect of Health Bucks on consumption of fruits and vegetables. Greater Health Bucks exposure was associated with a greater likelihood of a self-reported year-over-year increase in fruit and vegetable consumption among farmers’ market shoppers as well as residents of DPHO neighbourhoods. This is consistent with findings from an evaluation of Philly Food Bucks, a farmers’ market incentive programme for SNAP participants in low-income Philadelphia neighbourhoods, which similarly found self-reported increases in fruit and vegetable consumption among Philly Food Bucks users( Reference Young, Aquilante and Solomon 40 ). In addition, DPHO neighbourhood residents who shopped at farmers’ markets reported higher current levels of fruit and vegetable consumption, consistent with prior research indicating an association between fruit and vegetable consumption and farmers’ market use among low-income populations( Reference Evans, Jennings and Smiley 37 , Reference Jilcott Pitts, Gustafson and Wu 41 ).
However, we did not find evidence of an association between Health Bucks awareness or use and self-reported fruit and vegetable consumption. Furthermore, prior-day fruit and vegetable servings were lower among shoppers at Health Bucks markets than in other farmers’ markets, likely reflecting the fact that Health Bucks markets were intentionally located in underserved DPHO areas with lower levels of fruit and vegetable consumption overall. These findings suggest differences in fruit and vegetable consumption by use of farmers’ markets, but not necessarily by Health Bucks exposure.
In addition, we did not detect an impact of Health Bucks on fruit and vegetable consumption levels in DPHO neighbourhoods in our CHS difference-in-differences analysis. One possible explanation is that the scale of the programme may be insufficient to generate impacts detectable at the community level using available data. Only 21 % of respondents in DPHO neighbourhoods had heard of the Health Bucks programme, while 8 % reported that they had actually used Health Bucks and just 4 % that they had used Health Bucks that season. We may have detected a statistically significant impact either with a larger analytic sample size, greater programme size or a more finely grained measure of exposure to Health Bucks than our simple DPHO residence measure.
Strengths and limitations of the study
Our mixed-methods evaluation design includes multiple, complementary components, each with its own unique strengths and limitations.
The DPHO neighbourhood resident and farmers’ market shopper surveys incorporate detailed measures of respondents’ farmers’ market and Health Bucks use, along with a wide range of outcome measures of interest, including awareness of and perceived access to farmers’ markets, as well as self-reported fruit and vegetable spending and consumption measures. The representative sample of DPHO neighbourhood residents offers important contextual evidence on the programme’s target population and reach to inform other aspects of the study. The survey of farmers’ market shoppers in Health Bucks and non-Health Bucks markets, which is, to our knowledge, the largest field survey of farmers’ market shoppers to have been conducted, similarly provides information on outcomes among those most likely to actually participate in the programme, along with other farmers’ market shoppers as a point of comparison to allow assessment of similarities and differences among these groups. However, our ability to draw causal inference from these two surveys is limited, because data were collected during a single market season after programme implementation. Additionally, because the shopper survey was conducted on site at farmers’ markets, these data cannot be used to answer questions about the broader community; conversely, because the sample for the neighbourhood resident survey was randomly selected from communities targeted by Health Bucks, these results include responses from individuals who may not be familiar with farmers’ markets or with Health Bucks. As well, like all self-reported data, survey responses may be subject to social desirability bias, in which respondent reports are influenced by norms about the most socially acceptable response to survey questions. Finally, the relatively low response rate for the neighbourhood resident survey may raise concerns about non-response bias, and the exclusion of cell phone-only households may additionally bias the results.
Our difference-in-differences analysis of CHS data has strengths that directly complement limitations of our two primary data collection efforts. In particular, as noted in the introduction, the repeated cross-section design provides a basis for causal inference, in contrast to the single-point-in-time structure of the other two surveys. However, the CHS secondary data analyses are also limited in several ways. First, the CHS measure of fruit and vegetable consumption is the same measure we used in our cross-sectional surveys, offering a valid point of comparison, but the CHS did not include questions about Health Bucks or other outcomes of interest, like farmers’ market access and purchasing. Second, the CHS sample does not include households with no telephone service or cell phone-only households, so weighted results are representative only of New York City households with landline telephone service; since cell phone usage increased over this time period, it is possible that excluding cell phone-only households may have distorted observed time trends. Third, we cannot rule out the possibility of non-response bias due to differences in unobservable characteristics among CHS respondents or the impact of contextual and environmental policy differences related to food programmes or neighbourhood characteristics that we did not measure. For this reason, it may be difficult to statistically isolate effects of Health Bucks in the CHS analysis.
Conclusion
Public health advocates and policy makers are increasingly promoting farmers’ markets as a viable source of fresh fruits and vegetables in low-income, urban settings. As part of these broader promotional efforts, farmers’ market incentive programmes have sharply proliferated in communities across the USA in recent years( Reference Winch 42 ). Health Bucks is one of the earliest and most mature of these initiatives and through its successful implementation has demonstrated the feasibility and sustainability of such strategies, with steadily increasing numbers of participating farmers’ markets and participants over time( Reference Payne, Wethington and Olsho 28 ). With the recent announcement of the US Department of Agriculture’s Food Insecurity Nutrition Incentive Grant Program( 43 ), funding and support for similar programmes will expand considerably in the near future.
And yet, while nutrition incentives have been demonstrated as an effective strategy for improving fruit and vegetable consumption among SNAP participants in general( Reference Klerman, Bartlett and Wilde 44 , Reference Bartlett, Klerman and Olsho 45 ), to date there is still relatively little rigorous evidence on the effectiveness of these programmes in the farmers’ market context( Reference Baronberg, Dunn and Nonas 38 – Reference Young, Aquilante and Solomon 40 ). While our study provides promising evidence that use of farmers’ market incentives is associated with improved short-term outcomes such as awareness and use of farmers’ markets, additional research is needed to better understand impacts on outcomes such as fruit and vegetable consumption, and, ultimately, overall nutritional and health status.
Acknowledgements
Acknowledgements: The authors wish to acknowledge the valuable contributions of Yvonne Abel, Cheryl Austin, Cristina Booker, Diane Eggert, Diane Gibson, Jacey Greece, Kasey Holloway, Sonia Kim, Erin Lee, Jessica Levin, Bob Lewis, Tara Merry, Cathy Nonas, Leah Staub-DeLong, Alexis Stevens, Raegan Tuff, Andy Weiss and Holly Wethington to this evaluation study. They also thank the Abt Associates Journal Authors Support Group for valuable comments on the manuscript draft. Financial support: This evaluation study was funded by the Centers for Disease Control and Prevention (CDC) (under contract #200-2008-27954, TO #2009-LB-14). The CDC had no role in the design, analysis or writing of this article. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC. Conflicts of interest: None. Authorship: L.E.W.O. led development of the study design and led analysis, data collection and manuscript development. G.H.P. contributed to interpretation and synthesis of results, and manuscript development and revision. D.K.W. served as Principal Investigator for the study, with oversight for all design, data collection, analysis and writing tasks. S.B. contributed to the study design and interpretation of results, as well as manuscript revisions. J.J. contributed to interpretation and synthesis of results, and manuscript development and revision. A.A. contributed to the study design and interpretation of results, as well as manuscript revisions. Ethics of human subject participation: This evaluation study was reviewed and approved by the Abt Associates Institutional Review Board and the New York City DOHMH Institutional Review Board.
Supplementary material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1368980015001056
