Children who grow up in poverty are more likely to experience chronic stressors that generate “wear” on stress regulatory systems including the hypothalamus–pituitary–adrenal (HPA) axis. This can have long-term consequences for health and well-being. Prior research has examined the role of proximal family and home contributions to HPA axis functioning. However, there is evidence to suggest that more distal levels of context, including neighborhoods, also matter. Prior evidence has primarily focused on adolescents and adults, with little evidence linking the neighborhood context with HPA activity in infancy and toddlerhood. We tested whether neighborhood disadvantage (indexed by US Census data) was associated with basal salivary cortisol levels at 7, 15, and 24 months of child age in a large sample of families (N = 1,292) residing in predominately low-income and rural communities in the United States. Multilevel models indicated that neighborhood disadvantage was positively associated with salivary cortisol levels and that this effect emerged across time. This effect was moderated by the race/ethnicity of children such that the association was only observed in White children in our sample. Findings provide preliminary evidence that the neighborhood context is associated with stress regulation during toddlerhood, elucidating a need for future work to address possible mechanisms.