Hostname: page-component-8448b6f56d-sxzjt Total loading time: 0 Render date: 2024-04-24T13:18:44.465Z Has data issue: false hasContentIssue false
  • English
  • Français

Development of a probability calculator for psychosis risk in children, adolescents, and young adults

Published online by Cambridge University Press:  12 January 2021

Tyler M. Moore Open the ORCID record for Tyler M. Moore [Opens in a new window] ,
Monica E. Calkins ,
Adon F. G. Rosen ,
Ellyn R. Butler ,
Kosha Ruparel ,
Paolo Fusar-Poli ,
Nikolaos Koutsouleris ,
Philip McGuire ,
Tyrone D. Cannon  and
Ruben C. Gur
...Show all authors
Tyler M. Moore*
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA Lifespan Brain Institute, Penn Medicine and Children's Hospital of Philadelphia, Philadelphia, PA 19104, USA
Monica E. Calkins
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA Lifespan Brain Institute, Penn Medicine and Children's Hospital of Philadelphia, Philadelphia, PA 19104, USA
Adon F. G. Rosen
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
Ellyn R. Butler
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
Kosha Ruparel
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA Lifespan Brain Institute, Penn Medicine and Children's Hospital of Philadelphia, Philadelphia, PA 19104, USA
Paolo Fusar-Poli
Affiliation:
Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK OASIS service, South London and Maudsley NHS Foundation Trust, London, UK Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
Nikolaos Koutsouleris
Affiliation:
Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University Munich, Germany
Philip McGuire
Affiliation:
Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
Tyrone D. Cannon
Affiliation:
Departments of Psychology and Psychiatry, Yale University, New Haven, CT 06520, USA
Ruben C. Gur
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA Lifespan Brain Institute, Penn Medicine and Children's Hospital of Philadelphia, Philadelphia, PA 19104, USA
Raquel E. Gur
Affiliation:
Department of Psychiatry, Brain Behavior Laboratory, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA Lifespan Brain Institute, Penn Medicine and Children's Hospital of Philadelphia, Philadelphia, PA 19104, USA
*
Author for correspondence: Tyler M. Moore, E-mail: tymoore@pennmedicine.upenn.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Assessment of risks of illnesses has been an important part of medicine for decades. We now have hundreds of ‘risk calculators’ for illnesses, including brain disorders, and these calculators are continually improving as more diverse measures are collected on larger samples.

Methods

We first replicated an existing psychosis risk calculator and then used our own sample to develop a similar calculator for use in recruiting ‘psychosis risk’ enriched community samples. We assessed 632 participants age 8–21 (52% female; 48% Black) from a community sample with longitudinal data on neurocognitive, clinical, medical, and environmental variables. We used this information to predict psychosis spectrum (PS) status in the future. We selected variables based on lasso, random forest, and statistical inference relief; and predicted future PS using ridge regression, random forest, and support vector machines.

Results

Cross-validated prediction diagnostics were obtained by building and testing models in randomly selected sub-samples of the data, resulting in a distribution of the diagnostics; we report the mean. The strongest predictors of later PS status were the Children's Global Assessment Scale; delusions of predicting the future or having one's thoughts/actions controlled; and the percent married in one's neighborhood. Random forest followed by ridge regression was most accurate, with a cross-validated area under the curve (AUC) of 0.67. Adjustment of the model including only six variables reached an AUC of 0.70.

Conclusions

Results support the potential application of risk calculators for screening and identification of at-risk community youth in prospective investigations of developmental trajectories of the PS.

Keywords

Feature-selectionprodromepsychosis spectrumrecruitingrisk calculator
Type
Original Article
Information
Psychological Medicine , Volume 52 , Issue 14 , October 2022 , pp. 3159 - 3167
Check for updates
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

These authors contributed equally to this work.

References

Adibi, A., Sadatsafavi, M., & Ioannidis, J. P. (2020). Validation and utility testing of clinical prediction models: Time to change the approach. JAMA, 324(3), 235–236.CrossRefGoogle ScholarPubMed
Bilimoria, K. Y., Liu, Y., Paruch, J. L., Zhou, L., Kmiecik, T. E., Ko, C. Y., & Cohen, M. E. (2013). Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons. Journal of the American College of Surgeons, 217(5), 833–842.CrossRefGoogle ScholarPubMed
Boehmke, B., & Greenwell, B. M. (2020). Hands-On machine learning with R. Boca Raton, FL: CRC Press.Google Scholar
Brown, J. L. (1963). Follow-up of children with atypical development (infantile psychosis). American Journal of Orthopsychiatry, 33(5), 855.CrossRefGoogle ScholarPubMed
Calkins, M. E., Merikangas, K. R., Moore, T. M., Burstein, M., Behr, M. A., Satterthwaite, T. D., … Gur, R. E. (2015). The Philadelphia neurodevelopmental cohort: Constructing a deep phenotyping collaborative. Journal of Child Psychology and Psychiatry, 56(12), 1356–1369. doi:10.1111/jcpp.12416.CrossRefGoogle ScholarPubMed
Calkins, M. E., Moore, T. M., Merikangas, K. R., Burstein, M., Satterthwaite, T. D., Bilker, W. B., … Gur, R. E. (2014). The psychosis spectrum in a young U.S. Community sample: Findings from the Philadelphia neurodevelopmental cohort. World Psychiatry, 13(3), 296–305. doi:10.1002/wps.20152.CrossRefGoogle Scholar
Calkins, M. E., Moore, T. M., Satterthwaite, T. D., Wolf, D. H., Turetsky, B. I., Roalf, D. R., … Gur, R. C. (2017). Persistence of psychosis spectrum symptoms in the Philadelphia neurodevelopmental cohort: A prospective two-year follow-up. World Psychiatry, 16, 62–76.CrossRefGoogle ScholarPubMed
Cannon, T. D., Yu, C., Addington, J., Bearden, C. E., Cadenhead, K. S., Cornblatt, B. A., … Perkins, D. O. (2016). An individualized risk calculator for research in prodromal psychosis. American Journal of Psychiatry, 173(10), 980–988.CrossRefGoogle ScholarPubMed
Carrión, R. E., Cornblatt, B. A., Burton, C. Z., Tso, I. F., Auther, A. M., Adelsheim, S., … Taylor, S. F. (2016). Personalized prediction of psychosis: External validation of the NAPLS-2 psychosis risk calculator with the EDIPPP project. American Journal of Psychiatry, 173(10), 989–996.CrossRefGoogle ScholarPubMed
Coiera, E., Ammenwerth, E., Georgiou, A., & Magrabi, F. (2018). Does health informatics have a replication crisis? Journal of the American Medical Informatics Association, 25(8), 963–968.CrossRefGoogle ScholarPubMed
Combe, G., Donkin, B., Buchanan, R., & Mackenzie, G. S. (1820). On inferring natural dispositions and talents from development of brain. Transactions of the Phrenological Society, 9, 306–379.Google Scholar
Cornblatt, B. A., Carrion, R. E., Addington, J., Seidman, L., Walker, E. F., Cannon, T. D., … Lencz, T. (2012). Risk factors for psychosis: Impaired social and role functioning. Schizophrenia Bulletin, 38(6), 1247–1257.CrossRefGoogle ScholarPubMed
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281.CrossRefGoogle ScholarPubMed
Davies, J., Sullivan, S., & Zammit, S. (2018). Adverse life outcomes associated with adolescent psychotic experiences and depressive symptoms. Society of Psychiatry and Psychiatric Epidemiology, 53(5), 497–507.CrossRefGoogle ScholarPubMed
Dawber, T. R., Moore, F. E., & Mann, G. V. (1957). Coronary heart disease in the Framingham study. American Journal of Public Health and the Nation's Health, 47, 4–24.CrossRefGoogle ScholarPubMed
Friedman, J. I., Harvey, P. D., Coleman, T., Moriarty, P. J., Bowie, C., Parrella, M., … Davis, K. L. (2001). Six-year follow-up study of cognitive and functional status across the lifespan in schizophrenia: A comparison with Alzheimer's disease and normal aging. American Journal of Psychiatry, 158(9), 1441–1448.CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Oliver, D., Spada, G., Patel, R., Stewart, R., Dobson, R., & McGuire, P. (2019). Real-world implementation of a transdiagnostic risk calculator for the automatic detection of individuals at risk of psychosis in clinical routine: Study protocol. Frontiers in Psychiatry, 10, 109.CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Rutigliano, G., Stahl, D., Davies, C., Bonoldi, I., Reilly, T., & McGuire, P. (2017). Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiatry, 74(5), 493–500.CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Schultze-Lutter, F., Cappucciati, M., Rutigliano, G., Bonoldi, I., Stahl, D., … Woods, S. W. (2016). The dark side of the moon: Meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophrenia Bulletin, 42(3), 732–743.CrossRefGoogle ScholarPubMed
Goeman, J., Meijer, R., & Chaturvedi, N. (2018). L1 and L2 penalized regression models. Vignette of R package penalized. Retrieved from https://cran.r-project.org/web/packages/penalized/penalized.pdf.Google Scholar
Gupta, P. K., Gupta, H., Sundaram, A., Kaushik, M., Fang, X., Miller, W. J., … Lynch, T. G. (2011). Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation, 124(4), 381–387.CrossRefGoogle ScholarPubMed
Gur, R. C., Ragland, J. D., Moberg, P. J., Turner, T. H., Bilker, W. B., Kohler, C., … Gur, R. E. (2001). Computerized neurocognitive scanning: I. Methodology and validation in healthy people. Neuropsychopharmacology, 25, 766–776.CrossRefGoogle ScholarPubMed
Gur, R. C., Richard, J., Hughett, P., Calkins, M. E., Macy, L., Bilker, W. B., … Gur, R. E. (2010). A cognitive neuroscience-based computerized battery for efficient measurement of individual differences: Standardization and initial construct validation. Journal of Neuroscience Methods, 187, 254–262.CrossRefGoogle ScholarPubMed
Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.CrossRefGoogle Scholar
Heikes, K. E., Eddy, D. M., Arondekar, B., & Schlessinger, L. (2008). Diabetes risk calculator: A simple tool for detecting undiagnosed diabetes and pre-diabetes. Diabetes Care, 31(5), 1040–1045.CrossRefGoogle ScholarPubMed
Kalman, J. L., Bresnahan, M., Schulze, T. G., & Susser, E. (2019). Predictors of persisting psychotic like experiences in children and adolescents: A scoping review. Schizophrenia Research, 209, 32–39.CrossRefGoogle ScholarPubMed
Kaufman, J., Birmaher, B., & Brent, D., Rao, U., Flynn, C., Moreci, P., … Ryan, N. (1997). Schedule for affective disorders and schizophrenia for school-age children – present and lifetime version (K- SADS-PL): Initial reliability and validity. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 980–988.CrossRefGoogle ScholarPubMed
Keith, S. J., & Matthews, S. M. (1991). The diagnosis of schizophrenia: A review of onset and duration issues. Schizophrenia Bulletin, 17(1), 51–68.CrossRefGoogle ScholarPubMed
Kobayashi, H., Nemoto, T., Koshikawa, H., Osono, Y., Yamazawa, R., Murakami, M., … Mizuno, M. (2008). A self-reported instrument for prodromal symptoms of psychosis: Testing the clinical validity of the PRIME screen—revised (PS-R) in a Japanese population. Schizophrenia Research, 106(2–3), 356–362.CrossRefGoogle Scholar
Le, T. T., Urbanowicz, R. J., Moore, J. H., & McKinney, B. A. (2019). Statistical inference relief (STIR) feature selection. Bioinformatics (Oxford, England), 35(8), 1358–1365.CrossRefGoogle ScholarPubMed
Lee, T. Y., Lee, J., Kim, M., Choe, E., & Kwon, J. S. (2018). Can we predict psychosis outside the clinical high-risk state? A systematic review of non-psychotic risk syndromes for mental disorders. Schizophrenia Bulletin, 44(2), 276–285.CrossRefGoogle Scholar
Leslie, W. D., & Lix, L. M. (2014). Comparison between various fracture risk assessment tools. Osteoporosis International, 25(1), 1–21.CrossRefGoogle ScholarPubMed
Liaw, A., & Wiener, M. (2002). Classification and regression by randomForest. R News, 2(3), 18–22.Google Scholar
Mahler, M. S. (1952). On child psychosis and schizophrenia: Autistic and symbiotic infantile psychoses. The Psychoanalytic Study of the Child, 7(1), 286–305.CrossRefGoogle Scholar
Mahmood, S. S., Levy, D., Vasan, R. S., & Wang, T. J. (2014). The Framingham Heart Study and the epidemiology of cardiovascular disease: A historical perspective. The Lancet, 383(9921), 999–1008.CrossRefGoogle ScholarPubMed
Maxwell, M. E. (1996). Manual for the FIGS. Bethesda: Clinical Neurogenetics Branch, Intramural Research Program, National Institute for Mental Health.Google Scholar
McGlashan, T. H., Miller, T. J., & Woods, S. W. (2003). Structured interview for prodromal syndromes, version 4.0. New Haven: Prime Clinic Yale School of Medicine.Google Scholar
McNeish, D. M. (2015). Using lasso for predictor selection and to assuage overfitting: A method long overlooked in behavioral sciences. Multivariate Behavioral Research, 50(5), 471–484.CrossRefGoogle ScholarPubMed
Miller, T. J., Cicchetti, D., & Markovich, P. J. (2004). The SIPS screen: A brief self- report screen to detect the schizophrenia prodrome. Schizophrenia Research, 70(Suppl. 1), 78.Google Scholar
Miller, T. J., McGlashan, T. H., Woods, S. W., Stein, K., Driesen, N., Corcoran, C. M., … Davidson, L. (1999). Symptom assessment in schizophrenic prodromal states. Psychiatric Quarterly, 70(4), 273–287.CrossRefGoogle ScholarPubMed
Moore, T. M., Martin, I. K., Gur, O. M., Jackson, C. T., Scott, J. C., Calkins, M. E., … Gur, R. E. (2016). Characterizing social environment's association with neurocognition using census and crime data linked to the Philadelphia neurodevelopmental cohort. Psychological Medicine, 46(3), 599–610.CrossRefGoogle ScholarPubMed
Moore, T. M., Reise, S. P., Gur, R. E., Hakonarson, H., & Gur, R. C. (2015). Psychometric properties of the Penn Computerized Neurocognitive Battery. Neuropsychology, 29(2), 235.CrossRefGoogle ScholarPubMed
Nelson, B., & McGorry, P. (2020). The prodrome of psychotic disorders: Identification, prediction, and preventive treatment. Child and Adolescent Psychiatric Clinics, 29(1), 57–69.CrossRefGoogle ScholarPubMed
Osborne, K. J., & Mittal, V. A. (2019). External validation and extension of the NAPLS-2 and SIPS-RC personalized risk calculators in an independent clinical high-risk sample. Psychiatry Research, 279, 9–14.CrossRefGoogle Scholar
Page, R. C., Krall, E. A., Martin, J., Mancl, L., & Garcia, R. I. (2002). Validity and accuracy of a risk calculator in predicting periodontal disease. The Journal of the American Dental Association, 133(5), 569–576.CrossRefGoogle ScholarPubMed
Parikh, P., Shiloach, M., Cohen, M. E., Bilimoria, K. Y., Ko, C. Y., Hall, B. L., & Pitt, H. A. (2010). Pancreatectomy risk calculator: An ACS-NSQIP resource. Hpb, 12(7), 488–497.CrossRefGoogle ScholarPubMed
Powers, A. R., Addington, J., Perkins, D. O., Bearden, C. E., Cadenhead, K. S., Cannon, T. D., … Walker, E. F. (2020). Duration of the psychosis prodrome. Schizophrenia Research, 216, 443–449.CrossRefGoogle ScholarPubMed
Radua, J., Ramella-Cravaro, V., Ioannidis, J. P., Reichenberg, A., Phiphopthatsanee, N., Amir, T., … Fusar-Poli, P. (2018). What causes psychosis? An umbrella review of risk and protective factors. World Psychiatry, 17(1), 49–66.CrossRefGoogle ScholarPubMed
Riecher-Rössler, A., & Studerus, E. (2017). Prediction of conversion to psychosis in individuals with an at-risk mental state: A brief update on recent developments. Current Opinion in Psychiatry, 30(3), 209–219.CrossRefGoogle Scholar
Rüsch, N., Heekeren, K., Theodoridou, A., Müller, M., Corrigan, P. W., Mayer, B., … Rössler, W. (2015). Stigma as a stressor and transition to schizophrenia after one year among young people at risk of psychosis. Schizophrenia Research, 166(1–3), 43–48.CrossRefGoogle ScholarPubMed
Sanfelici, R., Dwyer, D. B., Antonucci, L. A., & Koutsouleris, N. (2020). Individualized diagnostic and prognostic models for patients with psychosis risk syndromes: A meta-analytic view on the state-of-the-art. Biological Psychiatry, 88(4), 349–360.CrossRefGoogle ScholarPubMed
Satterthwaite, T. D., Elliott, M. A., Ruparel, K., Loughead, J., Prabhakaran, K., & Calkins, M. E., … Gur, R. E. (2014). Neuroimaging of the Philadelphia neurodevelopmental cohort. NeuroImage, 86, 544–553.CrossRefGoogle ScholarPubMed
Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., & Aluwahlia, S. (1983). A children's global assessment scale (CGAS). Archives of General Psychiatry, 40(11), 1228–1231.CrossRefGoogle ScholarPubMed
Taylor, J. H., Asabere, N., Calkins, M. E., Moore, T. M., Tang, S. X., Xavier, R. M., … Gur, R. E. (2020). Characteristics of youth with reported family history of psychosis spectrum symptoms in the Philadelphia neurodevelopmental cohort. Schizophrenia Research, 216, 104–110.CrossRefGoogle ScholarPubMed
Taylor, J. H., Calkins, M. E., & Gur, R. E. (2020). Markers of psychosis risk in the general population. Biological Psychiatry, 88(4), 337–348.CrossRefGoogle ScholarPubMed
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288.Google Scholar
Wigman, J. T., van Winkel, R., Raaijmakers, Q. A., Ormel, J., Verhulst, F. C., Reijneveld, S. A., … Vollebergh, W. A. (2011). Evidence for a persistent, environment-dependent and deteriorating subtype of subclinical psychotic experiences: A 6-year longitudinal general population study. Psychological Medicine, 41(11), 2317–2329.CrossRefGoogle ScholarPubMed
Woodberry, K. A., Shapiro, D. I., Bryant, C., & Seidman, L. J. (2016). Progress and future directions in research on the psychosis prodrome: A review for clinicians. Harvard Review of Psychiatry, 24(2), 87.CrossRefGoogle ScholarPubMed
Yang, L. H., Link, B. G., Ben-David, S., Gill, K. E., Girgis, R. R., Brucato, G., … Corcoran, C. M. (2015). Stigma related to labels and symptoms in individuals at clinical high-risk for psychosis. Schizophrenia Research, 168(1–2), 9–15.CrossRefGoogle ScholarPubMed
Yung, A. R., & McGorry, P. D. (1996). The prodromal phase of first-episode psychosis: Past and current conceptualizations. Schizophrenia Bulletin, 22(2), 353–370.CrossRefGoogle ScholarPubMed
Yung, A. R., Phillips, L. J., Yuen, H. P., Francey, S. M., McFarlane, C. A., Hallgren, M., & McGorry, P. D. (2003). Psychosis prediction: 12-month follow up of a high-risk (“prodromal”) group. Schizophrenia Research, 60(1), 21–32.CrossRefGoogle ScholarPubMed
Zhang, T., Li, H., Tang, Y., Niznikiewicz, M. A., Shenton, M. E., Keshavan, M. S., … Wang, J. (2018). Validating the predictive accuracy of the NAPLS-2 psychosis risk calculator in a clinical high-risk sample from the SHARP (Shanghai At risk for psychosis) program. American Journal of Psychiatry, 175(9), 906–908.CrossRefGoogle Scholar
Supplementary material: File

Moore et al. supplementary material

Moore et al. supplementary material

Download Moore et al. supplementary material(File)
File 93.2 KB