Objectives: The aim of this study was to assess the association between personality factors and age-related longitudinal cognitive performance, and explore interactions of stress-proneness with apolipoprotein E (APOE) ɛ4, a prevalent risk factor for Alzheimer’s disease (AD). Methods: A total of 510 neuropsychiatrically healthy residents of Maricopa County recruited through media ads (mean age 57.6±10.6 years; 70% women; mean education 15.8±2.4 years; 213 APOE ɛ4 carriers) had neuropsychological testing every 2 years (mean duration follow-up 9.1±4.4 years), and the complete Neuroticism Extraversion Openness Personality Inventory-Revised. Several tests were administered within each of the following cognitive domains: memory, executive skills, language, visuospatial skills, and general cognition. Primary effects on cognitive trajectories and APOE ɛ4 interactions were ascertained with quadratic models. Results: With personality factors treated as continuous variables, Neuroticism was associated with greater decline, and Conscientiousness associated with reduced decline consistently across tests in memory and executive domains. With personality factors trichotomized, the associations of Neuroticism and Conscientiousness were again highly consistent across tests within memory and to a lesser degree executive domains. While age-related memory decline was greater in APOE ɛ4 carriers as a group than ɛ4 noncarriers, verbal memory decline was mitigated in ɛ4 carriers with higher Conscientiousness, and visuospatial perception and memory decline was mitigated in ɛ4 carriers with higher Openness. Conclusions: Neuroticism and Conscientiousness were associated with changes in longitudinal performances on tests sensitive to memory and executive skills. APOE interactions were less consistent. Our findings are consistent with previous studies that have suggested that personality factors, particularly Neuroticism and Conscientiousness are associated with cognitive aging patterns. (JINS, 2016, 22, 765–776)