Childhood trauma is strongly associated with poor health outcomes. Although many studies have found associations between adverse childhood experiences (ACEs), a well-established indicator of childhood trauma and diet-related health outcomes, few have explored the relationship between ACEs and diet quality, despite growing literature in epidemiology and neurobiology suggesting that childhood trauma has an important but poorly understood relationship with diet. Thus, we performed a cross-sectional study of the association of ACEs and adult diet quality in the Southern Community Cohort Study, a largely low-income and racially diverse population in the southeastern United States. We used ordinal logistic regression to estimate the association of ACEs with the Healthy Eating Index-2010 (HEI-10) score among 30 854 adults aged 40–79 enrolled from 2002 to 2009. Having experienced any ACE was associated with higher odds of worse HEI-10 among all (odds ratio (OR) 1⋅22; 95 % confidence interval (CI) 1⋅17, 1⋅27), and for all race–sex groups, and remained significant after adjustment for adult income. The increasing number of ACEs was also associated with increasing odds of a worse HEI-10 (OR for 4+ ACEs: 1⋅34; 95 % CI 1⋅27, 1⋅42). The association with worse HEI-10 score was especially strong for ACEs in the household dysfunction category, including having a family member in prison (OR 1⋅34; 95 % CI 1⋅25, 1⋅42) and parents divorced (OR 1⋅25; 95 % CI 1⋅20, 1⋅31). In summary, ACEs are associated with poor adult diet quality, independent of race, sex and adult income. Research is needed to explore whether trauma intervention strategies can impact adult diet quality.