Impaired cognitive functioning constitutes an important symptom of major depressive disorder (MDD), potentially associated with elevated cortisol levels. Adverse childhood experiences (ACE) enhance the risk for MDD and can contribute to disturbances in the stress systems, including cortisol and cognitive functions. In healthy participants, cortisol administration as well as acute stress can affect cognitive performance. In the current study, we tested cognitive performance in MDD patients with (N = 32) and without (N = 52) ACE and healthy participants with (N = 22) and without (N = 37) ACE after psychosocial stress induction (Trier Social Stress Test, TSST) and a control condition (Placebo-TSST). MDD predicted lower performance in verbal learning and both selective and sustained attention, while ACE predicted lower performance in psychomotoric speed and working memory. There were no interaction effects of MDD and ACE. After stress, MDD patients were more likely to show lower performance in working memory as well as in selective and sustained attention compared with participants without MDD. Individuals with ACE were more likely to show lower performance in verbal memory after stress compared with individuals without ACE. Our results indicate negative effects of MDD and ACE on distinct cognitive domains. Furthermore, MDD and/or ACE seem to enhance susceptibility for stress-related cognitive impairments.