Household food insecurity (HFI) is a major concern in South Asia. The pathways by which HFI may reduce child growth remain inadequately understood. In a cohort study of 12 693 maternal–infant dyads in rural Bangladesh, we examined association and likely explanatory pathways linking HFI, assessed using a validated nine-item perception-based index, to infant size at 6 months. Mothers were assessed early in pregnancy for anthropometric status, dietary diversity and socio-economic status. Infants were assessed for weight, length, and arm, chest and head circumferences and breast and complementary feeding status at birth and 6 months of age. Extent of HFI shared a negative, dose–response association with all measures of infant size at 6 months and odds of wasting and stunting; 57–89 % of variances in the unadjusted models were explained by prenatal factors (maternal nutritional status and dietary diversity), and birth size adjusted for gestational age. Postnatal infant breast and complementary feeding and morbidity exposures explained the remaining fraction of the significant association between HFI and differences in infant arm and chest circumferences and odds of underweight. Contextual (i.e. socio-economic) factors finally brought remaining non-significant fractions of the food insecurity-related mid-infancy growth deficit to practically zero. Improving food security prior to pregnancy and during gestation would likely improve infant growth the most in rural Bangladesh.