From a “successful aging” perspective, the subjective feeling of well-being is as important as “objective” health. Physical exercise is seen as being an effective way of staying healthy, but its link with well-being in a normal aging population remains largely unexplored. Based on two randomized surveys of the aging population, conducted in 1979 and 1994, respectively, with questionnaires including retrospective questions on activities and health, two cohorts of young-old (aged 64–74) were selected (cohort 1, born 1905–1914, N = 949; cohort 2, born 1920–1929, N = 602) and split into four groups, corresponding to their exercising trajectories (long-term exercisers LE, new exercisers NE, quitters Q, sedentary S). The link between the four trajectories and two indicators of well-being (self-rated health, self-assessed depression scale) was examined by means of regression analyses. In both cohorts, the LE group had a higher level of well-being than the Q and the S. The study also throws light on the case of the quitters (Q), who showed the lowest level of well-being. Scant research has hitherto been done on the causes and repercussions of abandoning exercise.