Skip to main content Accessibility help
×
Home

Machine learning enhances prediction of illness course: a longitudinal study in eating disorders

  • Ann F. Haynos (a1), Shirley B. Wang (a2), Sarah Lipson (a2), Carol B. Peterson (a1) (a3), James E. Mitchell (a4), Katherine A. Halmi (a5), W. Stewart Agras (a6) and Scott J. Crow (a1) (a3)...

Abstract

Background

Psychiatric disorders, including eating disorders (EDs), have clinical outcomes that range widely in severity and chronicity. The ability to predict such outcomes is extremely limited. Machine-learning (ML) approaches that model complexity may optimize the prediction of multifaceted psychiatric behaviors. However, the investigations of many psychiatric concerns have not capitalized on ML to improve prognosis. This study conducted the first comparison of an ML approach (elastic net regularized logistic regression) to traditional regression to longitudinally predict ED outcomes.

Methods

Females with heterogeneous ED diagnoses completed demographic and psychiatric assessments at baseline (n = 415) and Year 1 (n = 320) and 2 (n = 277) follow-ups. Elastic net and traditional logistic regression models comprising the same baseline variables were compared in ability to longitudinally predict ED diagnosis, binge eating, compensatory behavior, and underweight BMI at Years 1 and 2.

Results

Elastic net models had higher accuracy for all outcomes at Years 1 and 2 [average Area Under the Receiving Operating Characteristics Curve (AUC) = 0.78] compared to logistic regression (average AUC = 0.67). Model performance did not deteriorate when the most important predictor was removed or an alternative ML algorithm (random forests) was applied. Baseline ED (e.g. diagnosis), psychiatric (e.g. hospitalization), and demographic (e.g. ethnicity) characteristics emerged as important predictors in exploratory predictor importance analyses.

Conclusions

ML algorithms can enhance the prediction of ED symptoms for 2 years and may identify important risk markers. The superior accuracy of ML for predicting complex outcomes suggests that these approaches may ultimately aid in advancing precision medicine for serious psychiatric disorders.

Copyright

Corresponding author

Author for correspondence: Ann F. Haynos, E-mail: afhaynos@umn.edu

Footnotes

Hide All
*

The first two authors contributed equally to this work.

Footnotes

References

Hide All
Agras, W. S., Crow, S., Mitchell, J. E., Halmi, K. A., & Bryson, S. (2009). A 4-year prospective study of eating disorder NOS compared with full eating disorder syndromes. The International Journal of Eating Disorders, 42, 565570. doi: 10.1002/eat.20708
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR (4th ed.). text revision. Washington, DC: American Psychiatric Association.
Askland, K. D., Garnaat, S., Sibrava, N. J., Boisseau, C. L., Strong, D., Mancebo, M., … Eisen, J. (2015). Prediction of remission in obsessive compulsive disorder using a novel machine learning strategy. International Journal of Methods in Psychiatric Research, 24, 156169. doi: 10.1002/mpr.1463
Bardone-Cone, A. M., Hunt, R. A., & Watson, H. J. (2018). An overview of conceptualizations of eating disorder recovery, recent findings, and future directions. Current Psychiatry Reports, 20, 79. doi: 10.1007/s11920-018-0932-9
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.
Beck, A. T., Steer, R. A., & Carbin, M. G. (1988). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8, 77100. doi: 10.1016/0272-7358(88)90050-5
Berg, K. C., Peterson, C. B., Frazier, P., & Crow, S. J. (2012a). Psychometric evaluation of the eating disorder examination and eating disorder examination-questionnaire: A systematic review of the literature. The International Journal of Eating Disorders, 4, 428438. doi: 10.1002/eat.20931
Berg, K. C., Stiles-Shields, E. C., Swanson, S. A., Peterson, C. B., Lebow, J., & Le Grange, D. (2012b). Diagnostic concordance of the interview and questionnaire versions of the eating disorder examination. The International Journal of Eating Disorders, 45, 850855. doi: 10.1002/eat.20948
Berkman, N. D., Bulik, C. M., Brownley, K. A., Lohr, K. N., Sedway, J. A., Rooks, A., & Gartlehner, G. (2006). Management of eating disorders. Evidence Report/Technology Assessment, 135, 1166.
Bilder, R. M., Howe, A. G., & Sabb, F. W. (2013). Multilevel models from biology to psychology: Mission impossible? Journal of Abnormal Psychology, 122, 917927. doi: 10.1037/a0032263
Burke, T. A., Ammerman, B. A., & Jacobucci, R. (2019). The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: A systematic review. Journal of Affective Disorders, 245, 869884. doi: 10.1016/j.jad.2018.11.073
Bzdok, D., & Meyer-Lindenberg, A. (2018). Machine learning for precision psychiatry: Opportunities and challenges. Biological Psychiatry. Cognitive Neuroscience and Neuroimaging, 3, 223230. doi: 10.1016/j.bpsc.2017.11.007
Canty, A., & Ripley, B. (2019). boot: Bootstrap R (S-Plus) Functions. Version 1.3-22.
Carter, J. C., Blackmore, E., Sutandar-Pinnock, K., & Woodside, D. B. (2004). Relapse in anorexia nervosa: A survival analysis. Psychological Medicine, 34, 671679.
Carter, J. C., Mercer-Lynn, K. B., Norwood, S. J., Bewell-Weiss, C. V., Crosby, R. D., Woodside, D. B., & Olmsted, M. P. (2012). A prospective study of predictors of relapse in anorexia nervosa: Implications for relapse prevention. Psychiatry Research, 200, 518523. doi: 10.1016/j.psychres.2012.04.037
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019). Artificial intelligence, bias and clinical safety. Artificial intelligence, bias and clinical safety. BMJ Quality & Safety, 28, 231237. doi: 10.1136/bmjqs-2018-008370
Cho, G., Yim, J., Choi, Y., Ko, J., & Lee, S.-H. (2019). Review of machine learning algorithms for diagnosing mental illness. Psychiatry Investigation, 16, 262269. doi: 10.30773/pi.2018.12.21.2
Cooper, Z., & Fairburn, C. (1987). The eating disorder examination: A semi-structured interview for the assessment of the specific psychopathology of eating disorders. The International Journal of Eating Disorders, 6, 18. doi: 10.1002/1098-108X(198701)6:1<1::AID-EAT2260060102>3.0.CO;2-9
Crow, S. J., Agras, W. S., Halmi, K., Mitchell, J. E., & Kraemer, H. C. (2002). Full syndromal versus subthreshold anorexia nervosa, bulimia nervosa, and binge eating disorder: A multicenter study. The International Journal of Eating Disorders, 32, 309318. doi: 10.1002/eat.10088
Crow, S. J., Peterson, C. B., Swanson, S. A., Raymond, N. C., Specker, S., Eckert, E. D., & Mitchell, J. E. (2009). Increased mortality in bulimia nervosa and other eating disorders. The American Journal of Psychiatry, 166, 13421346. doi: 10.1176/appi.ajp.2009.09020247
DuBois, R. H., Rodgers, R. F., Franko, D. L., Eddy, K. T., & Thomas, J. J. (2017). A network analysis investigation of the cognitive-behavioral theory of eating disorders. Behavior Research and Therapy, 97, 213221. doi: 10.1016/j.brat.2017.08.004
Eddy, K. T., Tabri, N., Thomas, J. J., Murray, H. B., Keshaviah, A., Hastings, E., … Franko, D. L. (2017). Recovery from anorexia nervosa and bulimia nervosa at 22-year follow-up. The Journal of Clinical Psychiatry, 78, 184189. doi: 10.4088/JCP.15m10393
First, M. B., Gibbon, M., Spitzer, R. L., Williams, J. B. W., & Benjamin, L. S. (1997). Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II). Washington, DC: American Psychiatric Press, Inc.
First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1995). Structured Clinical Interview for DSM-IV Axis I Disorders: Patient Edition (SCID-I/P). Version 2.0. Biometrics Research Department, New York State Psychiatric Institute: New York.
Fox, K. R., Huang, X., Linthicum, K. P., Wang, S. B., Franklin, J. C., & Ribeiro, J. D. (2019). Model complexity improves the prediction of nonsuicidal self-injury. Journal of Consulting and Clinical Psychology, 87, 684692. doi: 10.1037/ccp0000421
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., … Nock, M. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143, 187232. doi: 10.1037/bul0000084
Franko, D. L., Tabri, N., Keshaviah, A., Murray, H. B., Herzog, D. B., & Thomas, J. J. (2018). Predictors of long-term recovery in anorexia nervosa and bulimia nervosa: Data from a 22-year longitudinal study. Journal of Psychiatry Research, 96, 183188. doi: 10.1016/j.jpsychires.2017.10.008
Gormally, J., Black, S., Daston, S., & Rardin, D. (1982). The assessment of binge eating severity among obese persons. Addictive Behaviors, 7, 4755. doi: 10.1016/0306-4603(82)90024-7
Greeno, C. G., Marcus, M. D., & Wing, R. R. (1995). Diagnosis of binge eating disorder: Discrepancies between a questionnaire and clinical interview. The International Journal of Eating Disorders, 17, 153160. doi: 10.1002/1098-108x(199503)17:2<153::aid-eat2260170208>3.0.co;2-v
Guarda, A. S., Wonderlich, S., Kaye, W., & Attia, E. (2018). A path to defining excellence in intensive treatment for eating disorders. The International Journal of Eating Disorders, 51, 10511055. doi: 10.1002/eat.22899
He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21, 12631284. doi: 10.1109/TKDE.2008.239
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349, 255260. doi: 10.1126/science.aaa8415
Kazdin, A. E., Fitzsimmons-Craft, E. E., & Wilfley, D. E. (2017). Addressing critical gaps in the treatment of eating disorders. The International Journal of Eating Disorders, 50, 170189. doi: 10.1002/eat.22670
Kessler, R. C., van Loo, H. M., Wardenaar, K. J., Bossarte, R. M., Brenner, L. A., Cai, T., … Zaslavsky, A. M. (2016). Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Molecular Psychiatry, 21, 13661371. doi: 10.1038/mp.2015.198
Koutsouleris, N., Kahn, R. S., Chekroud, A. M., Leucht, S., Falkai, P., & Wobrock, T. (2016). Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: A machine learning approach. The Lancet. Psychiatry, 3, 935946. doi: 10.1016/S2215-0366(16)30171-7
Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28, 126. doi: 10.18637/jss.v028.i05
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. New York: Springer.
Linthicum, K. P., Schafer, K. M., & Ribeiro, J. D. (2019). Machine learning in suicide science: Applications and ethics. Behavioral Science and the Law, 37, 214222. doi: 10.1002/bsl.2392
Lock, J., Agras, W. S., Le Grange, D., Couturier, J., Safer, D., & Bryson, S. W. (2013). Do end of treatment assessments predict outcome at follow-up in eating disorders? The International Journal of Eating Disorders, 46, 771778. doi: 10.1002/eat.22175
Marquand, A. F., Wolfers, T., Mennes, M., Buitelaar, J., & Beckmann, C. F. (2016). Beyond lumping and splitting: A review of computational approaches for stratifying psychiatric disorders. Biological Psychiatry. Cognitive Neuroscience and Neuroimaging, 1, 433447. doi: 10.1016/j.bpsc.2016.04.002.
Marucci, S., Ragione, L. D., De Iaco, G., Mococci, T., Vicini, M., Guastamacchia, E., & Triggiani, V. (2018). Anorexia nervosa and comorbid psychopathology. Endocrine, Metabolic, and Immune Disorders Drug Targets, 18, 316324. doi: 10.2174/1871530318666180213111637
McMahon, F. J. (2014). Prediction of treatment outcomes in psychiatry – where do we stand? Dialogues in Clinical Neuroscience, 16, 455464.
Mitchell, E., & Crow, E. (2006). Medical complications of anorexia nervosa and bulimia nervosa. Current Opinions in Psychiatry, 19, 438443. doi: 10.1097/01.yco.0000228768.79097.3e
Ogutu, J. O., Schulz-Streeck, T., & Piepho, H.-P. (2012). Genomic selection using regularized linear regression models: Ridge regression, lasso, elastic net and their extensions. BMC Proceedings, 6(Suppl 2), S10. doi: 10.1186/1753-6561-6-S2-S10
R Development Core Team (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
Ribeiro, J. D., Huang, X., Fox, K. R., Walsh, C. G., & Linthicum, K. P. (2019). Predicting imminent suicidal thoughts and nonfatal attempts: The role of complexity. Journal of Consulting and Clinical Psychology, 87, 684692. doi: 10.1177/2167702619838464
Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.
Smink, F. R., van Hoeken, D., & Hoek, H. W. (2012). Epidemiology of eating disorders: Incidence, prevalence and mortality rates. Current Psychiatry Reports, 14, 406414. doi: 10.1007/s11920-012-0282-y
Smith, G. C. S., Seaman, S. R., Wood, A. M., Royston, P., & White, I. R. (2014). Correcting for optimistic prediction in small data sets. American Journal of Epidemiology, 180, 318324. doi: 10.1093/aje/kwu140
Steinhausen, H.-C. (2002). The outcome of anorexia nervosa in the 20th century. The American Journal of Psychiatry, 159, 12841293. doi: 10.1176/appi.ajp.159.8.1284
Steinhausen, H.-C., & Weber, S. (2009). The outcome of bulimia nervosa: Findings from one-quarter century of research. The American Journal of Psychiatry, 166, 13311341. doi: 10.1176/appi.ajp.2009.09040582
Stice, E., Fisher, M., & Lowe, M. R. (2004). Are dietary restraint scales valid measures of acute dietary restriction? Unobtrusive observational data suggest not. Psychological Assessment, 16, 5159. doi: 10.1037/1040-3590.16.1.51
Stunkard, A. J., & Messick, S. (1985). The three-factor eating questionnaire to measure dietary restraint, disinhibition and hunger. Journal of Psychosomatic Research, 29, 7183. doi: 10.1016/0022-3999(85)90010-8
Suvisaari, J., Mantere, O., Keinänen, J., Mäntylä, T., Rikandi, E., Lindgren, M., … Raij, T. T. (2018). Is it possible to predict the future in first-episode psychosis? Frontiers in Psychiatry, 9, 580. doi: 10.3389/fpsyt.2018.00580
Thompson-Brenner, H., Franko, D. L., Thompson, D. R., Grilo, C. M., Boisseau, C. L., Roehrig, J. P., … Wilson, G. T. (2013). Race/ethnicity, education, and treatment parameters as moderators and predictors of outcome in binge eating disorder. Journal of Consulting and Clinical Psychology, 81, 710721. doi: 10.1037/a0032946
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58, 267288. doi: 10.1111/j.2517-6161.1996.tb02080.x
Tikhonov, A. N. (1963). Solution of incorrectly formulated problems and the regularization method. Soviet Mathematics, 4, 10351038.
Traviss-Turner, G. D., West, R. M., & Hill, A. J. (2017). Guided self-help for eating disorders: A systematic review and metaregression. European Eating Disorder Review, 25, 148164. doi: 10.1002/erv.2507
Vall, E., & Wade, T. D. (2015). Predictors of treatment outcome in individuals with eating disorders: A systematic review and meta-analysis. The International Journal of Eating Disorders, 48, 946971. doi: 10.1002/eat.22411
van Buuren, S. (2018). Flexible imputation of missing data (2nd ed.). Boca Raton, FL: CRC/Chapman & Hall.
Von Hippel, P. T. (2017). Regression with missing ys: An improved strategy for analyzing multiply imputed data. Sociological Methodology, 37, 83117. doi: 10.1111/j.1467-9531.2007.00180.x
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B, 67, 301320.

Keywords

Type Description Title
UNKNOWN
Supplementary materials

Haynos et al. supplementary material
Haynos et al. supplementary material

 Unknown (84 KB)
84 KB

Machine learning enhances prediction of illness course: a longitudinal study in eating disorders

  • Ann F. Haynos (a1), Shirley B. Wang (a2), Sarah Lipson (a2), Carol B. Peterson (a1) (a3), James E. Mitchell (a4), Katherine A. Halmi (a5), W. Stewart Agras (a6) and Scott J. Crow (a1) (a3)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.