Studies examining associations between fetal serotonin reuptake inhibitor (SRI) exposure and child autism spectrum disorder (ASD) diagnoses or delayed language remain mixed and rarely prospectively follow children or employ gold-standard assessments. We prospectively followed a cohort of mother–child dyads from pregnancy through early elementary school (N = 178), and obtained maternal and alternate–caregiver ratings of behaviors related to ASD (N = 137), as well as direct, gold-standard assessments of child ASD symptoms and pragmatic language among dyads who experienced prenatal depression and either took SRIs or were medication free during pregnancy (N = 44). Prenatal SRI exposure was related to maternal ratings of ASD-related behaviors (β = 0.24 95% confidence interval; CI [0.07, 0.48]), and, among boys, alternative caregiver ratings (males-only β = 0.28 95% CI [0.02, 0.55], females-only β = −0.21 95% CI [–0.63, 0.08]). However, results of our direct assessments suggest an association between SRI exposure and reduced pragmatic language scores (β = –0.27, 95% CI [–0.53, –0.01], but not ASD (Autism Diagnostic Observation Schedule β = 0.14 95% CI [–0.15, 0.41]; Social Responsiveness Scale β = 0.08 95% CI [–0.25, 0.40]). These discrepancies point to issues regarding how ASD is assessed, and the possibility that SRIs may be more strongly associated with language or other broader behaviors that coincide with ASD. Larger prospective studies that incorporate thorough, gold-standard assessments of ASD, language, and other ASD-related behaviors are needed.