The aim of this study was to identify social and biological drivers of fetal growth by examining associations with household, preconception, and pregnancy factors in a cohort from Soweto, South Africa. Complete data and ultrasound scans were collected on 519 women between 2013 and 2016 at 6 time points during pregnancy (<14, 14–18, 19–23, 24–28, 29–33 weeks, and 34–38 weeks). Household-level factors, preconception health, baseline body mass index (BMI), and demographic data were collected at the first visit. During pregnancy, gestational weight gain (GWG; kg/week) was calculated. At 24–28 weeks of gestation, oral glucose tolerance test was used to determine gestational diabetes mellitus (GDM) status, and hypertension status was characterised. Longitudinal growth in head circumference, abdominal circumference, biparietal diameter, and femur length were modelled using the Superimposition by Translation and Rotation, a shape-invariant model which produces growth curves against gestational age. A priori identified exposure variables were then included in a series of sex-stratified hierarchical regression models for each fetal growth outcome. No household-level factors were associated with fetal growth. Maternal BMI at baseline was positively associated with all outcome parameters in males and females. Both GWG (in males and females) and GDM (in males) were significant positive predictors of abdominal growth. Males showed more responsiveness to abdominal growth, while females were more responsive to linear growth. Thus, fetal growth was largely predicted by maternal biological factors, and sexual dimorphism in the responsiveness of fetal biometry to biological exposures was evident.