Background: Previous studies examined caffeine use and caffeine dependence and risk for the symptoms, or diagnosis, of psychiatric disorders. The current study aimed to determine if generalized anxiety disorder (GAD), panic disorder, phobias, major depressive disorder (MDD), anorexia nervosa (AN), or bulimia nervosa (BN) shared common genetic or environmental factors with caffeine use, caffeine tolerance, or caffeine withdrawal. Method: Using 2,270 women from the Virginia Adult Twin Study of Psychiatric and Substance Use Disorders, bivariate Cholesky decomposition models were used to determine if any of the psychiatric disorders shared genetic or environmental factors with caffeine use phenotypes. Results: GAD, phobias, and MDD shared genetic factors with caffeine use, with genetic correlations estimated to be 0.48, 0.25, and 0.38, respectively. Removal of the shared genetic and environmental parameter for phobias and caffeine use resulted in a significantly worse fitting model. MDD shared unique environmental factors (environmental correlation = 0.23) with caffeine tolerance; the genetic correlation between AN and caffeine tolerance and BN and caffeine tolerance were 0.64 and 0.49, respectively. Removal of the genetic and environmental correlation parameters resulted in significantly worse fitting models for GAD, phobias, MDD, AN, and BN, which suggested that there was significant shared liability between each of these phenotypes and caffeine tolerance. GAD had modest genetic correlations with caffeine tolerance, 0.24, and caffeine withdrawal, 0.35. Conclusions: There was suggestive evidence of shared genetic and environmental liability between psychiatric disorders and caffeine phenotypes. This might inform us about the etiology of the comorbidity between these phenotypes.