Objective: Increased levels of worry, age, and presence of the apolipoprotein-E (ApoE)-ε4 allele are associated with the risk of developing cognitive declines and Alzheimer’s disease (AD). Odor memory performance has been shown to vary as a function of age and ApoE genotype, and odor memory tests are sensitive to preclinical AD. Worry is known to influence verbal memory; however, its effects on odor memory are unknown. This study aimed to assess the relationships between worry, age, and ε4 status on odor memory. Method: Worry was evaluated for young (n = 53) and older (n = 45) adults using the Penn State Worry Questionnaire. Odor memory was assessed using the California Odor Learning Test, an olfactory analogue to the California Verbal Learning Test. Results: A significant main effect of worry on long-delay free recall was found, such that increasing worry was associated with better recall across age and ε4 status. A significant interaction effect between ε4 status and worry on both short-and long-delay cued recall was found, such that across age, higher worry was associated with increased cued recall scores among ε4-negative adults, and decreased scores among ε4-positive adults. Conclusions: Findings demonstrated that worry influences odor memory and exerts a particular effect on cued recall among ε4 carriers who are at a greater risk of developing AD. Worry is a modifiable predictor of cognitive decline and risk of dementia in aging. Future studies on the effects of treatments aimed at reducing worry (e.g., cognitive behavioral therapies for anxiety) on changes in cognitive functioning are warranted.