Depression and cognitive impairment are highly prevalent in type 2 diabetes (T2D), yet little is known about how their relationship varies by sex. We examined this question in a large T2D sample (N = 897) of non-demented elderly (≥ 65) participating in the Israel Diabetes and Cognitive Decline (IDCD) Study. Cognition was evaluated by a comprehensive neuropsychological battery and depressive symptoms were assessed by the Geriatric Depression Scale (GDS). The results showed that in all but the executive function domain, the association of depressive symptoms with poorer cognitive function was stronger in women than men, with a significant interaction for language/semantic categorization and missed significance for episodic memory. When defining clinical depression as GDS of ≥6, women with depression had significantly poorer language/semantic categorization, episodic memory, and overall cognitive function. Inclusion of antidepressants in the model did not alter substantively the associations. Our results suggest that depressed T2D women may have poorer cognitive performance, highlighting the significance of sex-specific personalized management of depression in elderly diabetics.