The present study investigated the associations of mothers' and fathers' lifetime depression and anxiety symptoms, and of infants' negative temperament with parents' and infants' gaze, facial expressions of emotion, and synchrony. We observed infants' (age between 3.5 and 5.5 months, N = 101) and parents' gaze and facial expressions during 4-min naturalistic face-to-face interactions. Parents' lifetime symptoms of depression and anxiety were assessed with clinical interviews, and infants' negative temperament was measured with standardized observations. Parents with more depressive symptoms and their infants expressed less positive and more neutral affect. Parents' lifetime anxiety symptoms were not significantly related to parents' expressions of affect, while they were linked to longer durations of gaze to parent, and to more positive and negative affect in infants. Parents' lifetime depression or anxiety was not related to synchrony. Infants' temperament did not predict infants' or parents' interactive behavior. The study reveals that more depression symptoms in parents are linked to more neutral affect from parents and from infants during face-to-face interactions, while parents' anxiety symptoms are related to more attention to parent and less neutral affect from infants (but not from parents).