Skip to main content Accessibility help
×
Home

Prediction of transition to psychosis in patients with a clinical high risk for psychosis: a systematic review of methodology and reporting

  • E. Studerus (a1), A. Ramyead (a2) and A. Riecher-Rössler (a1)

Abstract

Background

To enhance indicated prevention in patients with a clinical high risk (CHR) for psychosis, recent research efforts have been increasingly directed towards estimating the risk of developing psychosis on an individual level using multivariable clinical prediction models. The aim of this study was to systematically review the methodological quality and reporting of studies developing or validating such models.

Method

A systematic literature search was carried out (up to 14 March 2016) to find all studies that developed or validated a clinical prediction model predicting the transition to psychosis in CHR patients. Data were extracted using a comprehensive item list which was based on current methodological recommendations.

Results

A total of 91 studies met the inclusion criteria. None of the retrieved studies performed a true external validation of an existing model. Only three studies (3.5%) had an event per variable ratio of at least 10, which is the recommended minimum to avoid overfitting. Internal validation was performed in only 14 studies (15%) and seven of these used biased internal validation strategies. Other frequently observed modeling approaches not recommended by methodologists included univariable screening of candidate predictors, stepwise variable selection, categorization of continuous variables, and poor handling and reporting of missing data.

Conclusions

Our systematic review revealed that poor methods and reporting are widespread in prediction of psychosis research. Since most studies relied on small sample sizes, did not perform internal or external cross-validation, and used poor model development strategies, most published models are probably overfitted and their reported predictive accuracy is likely to be overoptimistic.

Copyright

Corresponding author

*Address for correspondence: A. Riecher-Rössler, M.D., University of Basel Psychiatric Clinics, Center for Gender Research and Early Detection, Kornhausgasse 7, CH-4051 Basel, Switzerland. (Email: anita.riecher@upkbs.ch)

References

Hide All
Altman, DG, McShane, LM, Sauerbrei, W, Taube, SE (2012). Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): explanation and elaboration. PLoS Medicine 9, e1001216.
Altman, DG, Vergouwe, Y, Royston, P, Moons, KG (2009). Prognosis and prognostic research: validating a prognostic model. BMJ 338, b605.
Amminger, GP, Leicester, S, Yung, AR, Phillips, LJ, Berger, GE, Francey, SM, Yuen, HP, McGorry, PD (2006). Early-onset of symptoms predicts conversion to non-affective psychosis in ultra-high risk individuals. Schizophrenia Research 84, 6776.
Austin, PC, Steyerberg, EW (2014). Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Statistical Methods in Medical Research. Published online 19 November 2014. doi:10.1177/0962280214558972.
Bang, M, Kim, KR, Song, YY, Baek, S, Lee, E, An, SK (2015). Neurocognitive impairments in individuals at ultra-high risk for psychosis: who will really convert? Australian and New Zealand Journal of Psychiatry 49, 462470.
Bedi, G, Carrillo, F, Cecchi, GA, Slezak, DF, Sigman, M, Mota, NB, Ribeiro, S, Javitt, DC, Copelli, M, Corcoran, CM (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. NPJ Schizophrenia 1, 15030.
Bouwmeester, W, Zuithoff, NP, Mallett, S, Geerlings, MI, Vergouwe, Y, Steyerberg, EW, Altman, DG, Moons, KG (2012). Reporting and methods in clinical prediction research: a systematic review. PLoS Medicine 9, 112.
Cannon, TD, Cadenhead, K, Cornblatt, B, Woods, SW, Addington, J, Walker, E, Seidman, LJ, Perkins, D, Tsuang, M, McGlashan, T, Heinssen, R (2008). Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Archives of General Psychiatry 65, 2837.
Chan, MK, Krebs, MO, Cox, D, Guest, PC, Yolken, RH, Rahmoune, H, Rothermundt, M, Steiner, J, Leweke, FM, van Beveren, NJ, Niebuhr, DW, Weber, NS, Cowan, DN, Suarez-Pinilla, P, Crespo-Facorro, B, Mam-Lam-Fook, C, Bourgin, J, Wenstrup, RJ, Kaldate, RR, Cooper, JD, Bahn, S (2015). Development of a blood-based molecular biomarker test for identification of schizophrenia before disease onset. Translational Psychiatry 5, e601.
Collins, GS, Mallett, S, Omar, O, Yu, LM (2011). Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Medicine 9, 103.
Collins, GS, Ogundimu, EO, Cook, JA, Manach, YL, Altman, DG (2016). Quantifying the impact of different approaches for handling continuous predictors on the performance of a prognostic model. Statistics in Medicine 35, 4124–4135.
Collins, GS, Omar, O, Shanyinde, M, Yu, LM (2013). A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. Journal of Clinical Epidemiology 66, 268277.
Collins, GS, Reitsma, JB, Altman, DG, Moons, KGM (2015). Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Journal of Clinical Epidemiology 68, 112121.
Cornblatt, BA, Carrion, RE, Auther, A, McLaughlin, D, Olsen, RH, John, M, Correll, CU (2015). Psychosis prevention: a modified clinical high risk perspective from the Recognition and Prevention (RAP) Program. American Journal of Psychiatry 172, 986994.
D'Amico, G, Malizia, G, D'Amico, M (2016). Prognosis research and risk of bias. Internal and Emergency Medicine 11, 251260.
Demjaha, A, Valmaggia, L, Stahl, D, Byrne, M, McGuire, P (2012). Disorganization/cognitive and negative symptom dimensions in the at-risk mental state predict subsequent transition to psychosis. Schizophrenia Bulletin 38, 351359.
Derksen, S, Keselman, HJ (1992). Backward, forward and stepwise automated subset selection algorithms: frequency of obtaining authentic and noise variables. British Journal of Mathematical and Statistical Psychology 45, 265282.
DeVylder, JE, Muchomba, FM, Gill, KE, Ben-David, S, Walder, DJ, Malaspina, D, Corcoran, CM (2014). Symptom trajectories and psychosis onset in a clinical high-risk cohort: the relevance of subthreshold thought disorder. Schizophrenia Research 159, 278283.
Fusar-Poli, P, Bechdolf, A, Taylor, MJ, Bonoldi, I, Carpenter, WT, Yung, AR, McGuire, P (2013 a). At risk for schizophrenic or affective psychoses? A meta-analysis of DSM/ICD diagnostic outcomes in individuals at high clinical risk. Schizophrenia Bulletin 39, 923932.
Fusar-Poli, P, Bonoldi, I, Yung, AR, Borgwardt, S, Kempton, MJ, Valmaggia, L, Barale, F, Caverzasi, E, McGuire, P (2012). Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. Archives of General Psychiatry 69, 220229.
Fusar-Poli, P, Borgwardt, S, Bechdolf, A, Addington, J, Riecher-Rössler, A, Schultze-Lutter, F, Keshavan, M, Wood, S, Ruhrmann, S, Seidman, LJ, Valmaggia, L, Cannon, T, Velthorst, E, De Haan, L, Cornblatt, B, Bonoldi, I, Birchwood, M, McGlashan, T, Carpenter, W, McGorry, P, Klosterkötter, J, McGuire, P, Yung, A (2013 b). The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry 70, 107120.
Fusar-Poli, P, Cappucciati, M, Borgwardt, S, Woods, SW, Addington, J, Nelson, B, Nieman, DH, Stahl, DR, Rutigliano, G, Riecher-Rössler, A, Simon, AE, Mizuno, M, Lee, TY, Kwon, JS, Lam, MM, Perez, J, Keri, S, Amminger, P, Metzler, S, Kawohl, W, Rössler, W, Lee, J, Labad, J, Ziermans, T, An, SK, Liu, CC, Woodberry, KA, Braham, A, Corcoran, C, McGorry, P, Yung, AR, McGuire, PK (2016). Heterogeneity of psychosis risk within individuals at clinical high risk: a meta-analytical stratification. JAMA Psychiatry 73, 113120.
Fusar-Poli, P, Schultze-Lutter, F (2016). Predicting the onset of psychosis in patients at clinical high risk: practical guide to probabilistic prognostic reasoning. Evidence-Based Mental Health 19, 1015.
Gorelick, MH (2006). Bias arising from missing data in predictive models. Journal of Clinical Epidemiology 59, 11151123.
Harrell, FE (2001). Regression Modeling Strategies with Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer: New York.
Healey, KM, Penn, DL, Perkins, D, Woods, SW, Addington, J (2013). Theory of mind and social judgments in people at clinical high risk of psychosis. Schizophrenia Research 150, 498504.
Hidalgo, B, Goodman, M (2013). Multivariate or multivariable regression? American Journal of Public Health 103, 3940.
Hollander, N, Sauerbrei, W, Schumacher, M (2004). Confidence intervals for the effect of a prognostic factor after selection of an ‘optimal’ cutpoint. Statistics in Medicine 23, 17011713.
Huang, JT, Leweke, FM, Tsang, TM, Koethe, D, Kranaster, L, Gerth, CW, Gross, S, Schreiber, D, Ruhrmann, S, Schultze-Lutter, F, Klosterkötter, J, Holmes, E, Bahn, S (2007). CSF metabolic and proteomic profiles in patients prodromal for psychosis. PLoS ONE 2, e756.
Kempton, MJ, Bonoldi, I, Valmaggia, L, McGuire, P, Fusar-Poli, P (2015). Speed of psychosis progression in people at ultra-high clinical risk: a complementary meta-analysis. JAMA Psychiatry 72, 622623.
Keshavan, MS, Nasrallah, HA, Tandon, R (2011). Schizophrenia, “Just the Facts” 6. Moving ahead with the schizophrenia concept: from the elephant to the mouse. Schizophrenia Research 127, 313.
Koutsouleris, N, Borgwardt, S, Meisenzahl, EM, Bottlender, R, Moller, HJ, Riecher-Rössler, A (2012 a). Disease prediction in the at-risk mental state for psychosis using neuroanatomical biomarkers: results from the FePsy study. Schizophrenia Bulletin 38, 12341246.
Koutsouleris, N, Davatzikos, C, Bottlender, R, Patschurek-Kliche, K, Scheuerecker, J, Decker, P, Gaser, C, Moller, HJ, Meisenzahl, EM (2012 b). Early recognition and disease prediction in the at-risk mental states for psychosis using neurocognitive pattern classification. Schizophrenia Bulletin 38, 12001215.
Koutsouleris, N, Kambeitz, J (2016). Pattern recognition methods in the prediction of psychosis. In Early Detection and Intervention in Psychosis – State of the Art and Future Perspectives (ed. Riecher-Rössler, A and McGorry, PD), pp. 95102. Karger: Basel.
Koutsouleris, N, Meisenzahl, EM, Davatzikos, C, Bottlender, R, Frodl, T, Scheuerecker, J, Schmitt, G, Zetzsche, T, Decker, P, Reiser, M, Moller, HJ, Gaser, C (2009). Use of neuroanatomical pattern classification to identify subjects in at-risk mental states of psychosis and predict disease transition. Archives of General Psychiatry 66, 700712.
Koutsouleris, N, Riecher-Rössler, A, Meisenzahl, EM, Smieskova, R, Studerus, E, Kambeitz-Ilankovic, L, von Saldern, S, Cabral, C, Reiser, M, Falkai, P, Borgwardt, S (2015). Detecting the psychosis prodrome across high-risk populations using neuroanatomical biomarkers. Schizophrenia Bulletin 41, 471482.
Krstajic, D, Buturovic, LJ, Leahy, DE, Thomas, S (2014). Cross-validation pitfalls when selecting and assessing regression and classification models. Journal of Cheminformatics 6, 10.
Leeflang, MM, Moons, KG, Reitsma, JB, Zwinderman, AH (2008). Bias in sensitivity and specificity caused by data-driven selection of optimal cutoff values: mechanisms, magnitude, and solutions. Clinical Chemistry 54, 729737.
Lencz, T, Smith, CW, McLaughlin, D, Auther, A, Nakayama, E, Hovey, L, Cornblatt, BA (2006). Generalized and specific neurocognitive deficits in prodromal schizophrenia. Biological Psychiatry 59, 863871.
Mallett, S, Royston, P, Dutton, S, Waters, R, Altman, DG (2010). Reporting methods in studies developing prognostic models in cancer: a review. BMC Medicine 8, 20.
Mason, O, Startup, M, Halpin, S, Schall, U, Conrad, A, Carr, V (2004). Risk factors for transition to first episode psychosis among individuals with ‘at-risk mental states’. Schizophrenia Research 71, 227237.
Michel, C, Ruhrmann, S, Schimmelmann, BG, Klosterkötter, J, Schultze-Lutter, F (2014). A stratified model for psychosis prediction in clinical practice. Schizophrenia Bulletin 40, 15331542.
Mittal, VA, Walker, EF, Bearden, CE, Walder, D, Trottman, H, Daley, M, Simone, A, Cannon, TD (2010). Markers of basal ganglia dysfunction and conversion to psychosis: neurocognitive deficits and dyskinesias in the prodromal period. Biological Psychiatry 68, 9399.
Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, Group, P (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine 6, e1000097.
Moons, KG, Altman, DG, Reitsma, JB, Ioannidis, JP, Macaskill, P, Steyerberg, EW, Vickers, AJ, Ransohoff, DF, Collins, GS (2015). Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Annals of Internal Medicine 162, W1–W73.
Moons, KG, Altman, DG, Vergouwe, Y, Royston, P (2009 a). Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338, b606.
Moons, KG, de Groot, JA, Bouwmeester, W, Vergouwe, Y, Mallett, S, Altman, DG, Reitsma, JB, Collins, GS (2014). Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Medicine 11, e1001744.
Moons, KG, Kengne, AP, Woodward, M, Royston, P, Vergouwe, Y, Altman, DG, Grobbee, DE (2012). Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart 98, 683690.
Moons, KG, Royston, P, Vergouwe, Y, Grobbee, DE, Altman, DG (2009 b). Prognosis and prognostic research: what, why, and how? BMJ 338, b375.
Mushkudiani, NA, Hukkelhoven, CW, Hernandez, AV, Murray, GD, Choi, SC, Maas, AI, Steyerberg, EW (2008). A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes. Journal of Clinical Epidemiology 61, 331343.
Nelson, B, Yuen, HP, Wood, SJ, Lin, A, Spiliotacopoulos, D, Bruxner, A, Broussard, C, Simmons, M, Foley, DL, Brewer, WJ, Francey, SM, Amminger, GP, Thompson, A, McGorry, PD, Yung, AR (2013). Long-term follow-up of a group at ultra high risk (“prodromal”) for psychosis: the PACE 400 study. JAMA Psychiatry 70, 793802.
Nieman, DH, Ruhrmann, S, Dragt, S, Soen, F, van Tricht, MJ, Koelman, JH, Bour, LJ, Velthorst, E, Becker, HE, Weiser, M, Linszen, DH, de Haan, L (2014). Psychosis prediction: stratification of risk estimation with information-processing and premorbid functioning variables. Schizophrenia Bulletin 40, 14821490.
Nieman, DH, Velthorst, E, Becker, HE, de Haan, L, Dingemans, PM, Linszen, DH, Birchwood, M, Patterson, P, Salokangas, RK, Heinimaa, M, Heinz, A, Juckel, G, von Reventlow, HG, Morrison, A, Schultze-Lutter, F, Klosterkotter, J, Ruhrmann, S, group E (2013). The Strauss and Carpenter Prognostic Scale in subjects clinically at high risk of psychosis. Acta Psychiatrica Scandinavica 127, 5361.
Núñez, E, Steyerberg, EW, Núñez, J (2011). [Regression modeling strategies]. Revista Española de Cardiología 64, 501507.
O'Donoghue, B, Nelson, B, Yuen, HP, Lane, A, Wood, S, Thompson, A, Lin, A, McGorry, P, Yung, AR (2015). Social environmental risk factors for transition to psychosis in an ultra-high risk population. Schizophrenia Research 161, 150155.
Perkins, DO, Jeffries, CD, Addington, J, Bearden, CE, Cadenhead, KS, Cannon, TD, Cornblatt, BA, Mathalon, DH, McGlashan, TH, Seidman, LJ, Tsuang, MT, Walker, EF, Woods, SW, Heinssen, R (2015 a). Towards a psychosis risk blood diagnostic for persons experiencing high-risk symptoms: preliminary results from the NAPLS project. Schizophrenia Bulletin 41, 419428.
Perkins, DO, Jeffries, CD, Cornblatt, BA, Woods, SW, Addington, J, Bearden, CE, Cadenhead, KS, Cannon, TD, Heinssen, R, Mathalon, DH, Seidman, LJ, Tsuang, MT, Walker, EF, McGlashan, TH (2015 b). Severity of thought disorder predicts psychosis in persons at clinical high-risk. Schizophrenia Research 169, 169–177.
Perlis, RH (2013). A clinical risk stratification tool for predicting treatment resistance in major depressive disorder. Biological Psychiatry 74, 714.
Pettersson-Yeo, W, Benetti, S, Marquand, AF, Dell'acqua, F, Williams, SC, Allen, P, Prata, D, McGuire, P, Mechelli, A (2013). Using genetic, cognitive and multi-modal neuroimaging data to identify ultra-high-risk and first-episode psychosis at the individual level. Psychological Medicine 43, 25472562.
Piskulic, D, Addington, J, Cadenhead, KS, Cannon, TD, Cornblatt, BA, Heinssen, R, Perkins, DO, Seidman, LJ, Tsuang, MT, Walker, EF, Woods, SW, McGlashan, TH (2012). Negative symptoms in individuals at clinical high risk of psychosis. Psychiatry Research 196, 220224.
Raballo, A, Nelson, B, Thompson, A, Yung, A (2011). The comprehensive assessment of at-risk mental states: from mapping the onset to mapping the structure. Schizophrenia Research 127, 107114.
Ramyead, A, Studerus, E, Kometer, M, Uttinger, M, Gschwandtner, U, Fuhr, P, Riecher-Rössler, A (2016). Prediction of psychosis using neural oscillations and machine learning in neuroleptic-naive at-risk patients. World Journal of Biological Psychiatry 17, 285295.
Riecher-Rössler, A, Pflueger, MO, Aston, J, Borgwardt, SJ, Brewer, WJ, Gschwandtner, U, Stieglitz, RD (2009). Efficacy of using cognitive status in predicting psychosis: a 7-year follow-up. Biological Psychiatry 66, 10231030.
Royston, P, Altman, DG (2013). External validation of a Cox prognostic model: principles and methods. BMC Medical Research Methodology 13, 33.
Royston, P, Altman, DG, Sauerbrei, W (2006). Dichotomizing continuous predictors in multiple regression: a bad idea. Statistics in Medicine 25, 127141.
Royston, P, Moons, KG, Altman, DG, Vergouwe, Y (2009). Prognosis and prognostic research: developing a prognostic model. BMJ 338, b604.
Ruhrmann, S, Klosterkötter, J, Bodatsch, M, Nikolaides, A, Julkowski, D, Hilboll, D, Schultz-Lutter, F (2012). Chances and risks of predicting psychosis. European Archives of Psychiatry and Clinical Neuroscience 262 (Suppl. 2), S85S90.
Ruhrmann, S, Schultze-Lutter, F, Salokangas, RK, Heinimaa, M, Linszen, D, Dingemans, P, Birchwood, M, Patterson, P, Juckel, G, Heinz, A, Morrison, A, Lewis, S, von Reventlow, HG, Klosterkötter, J (2010). Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European Prediction of Psychosis Study. Archives of General Psychiatry 67, 241251.
Rüsch, N, Heekeren, K, Theodoridou, A, Muller, M, Corrigan, PW, Mayer, B, Metzler, S, Dvorsky, D, Walitza, S, Rossler, W (2015). Stigma as a stressor and transition to schizophrenia after 1 year among young people at risk of psychosis. Schizophrenia Research 166, 4348.
Schultze-Lutter, F, Klosterkötter, J, Michel, C, Winkler, K, Ruhrmann, S (2012). Personality disorders and accentuations in at-risk persons with and without conversion to first-episode psychosis. Early Intervention in Psychiatry 6, 389398.
Schultze-Lutter, F, Klosterkötter, J, Picker, H, Steinmeyer, E-M, Ruhrmann, S (2007). Predicting first-episode psychosis by basic symptom criteria. Clinical Neuropsychiatry 4, 1122.
Schultze-Lutter, F, Michel, C, Schmidt, SJ, Schimmelmann, BG, Maric, NP, Salokangas, RK, Riecher-Rössler, A, van der Gaag, M, Nordentoft, M, Raballo, A, Meneghelli, A, Marshall, M, Morrison, A, Ruhrmann, S, Klosterkötter, J (2015). EPA guidance on the early detection of clinical high risk states of psychoses. European Psychiatry 30, 405416.
Seel, RT, Steyerberg, EW, Malec, JF, Sherer, M, Macciocchi, SN (2012). Developing and evaluating prediction models in rehabilitation populations. Archives of Physical Medicine and Rehabilitation 93, S138S153.
Seidman, LJ, Giuliano, AJ, Meyer, EC, Addington, J, Cadenhead, KS, Cannon, TD, McGlashan, TH, Perkins, DO, Tsuang, MT, Walker, EF, Woods, SW, Bearden, CE, Christensen, BK, Hawkins, K, Heaton, R, Keefe, RS, Heinssen, R, Cornblatt, BA; North American Prodrome Longitudinal Study (NAPLS) Group (2010). Neuropsychology of the prodrome to psychosis in the NAPLS consortium: relationship to family history and conversion to psychosis. Archives of General Psychiatry 67, 578588.
Simon, AE, Borgwardt, S, Riecher-Rössler, A, Velthorst, E, de Haan, L, Fusar-Poli, P (2013). Moving beyond transition outcomes: meta-analysis of remission rates in individuals at high clinical risk for psychosis. Psychiatry Research 209, 266272.
Sterne, JA, White, IR, Carlin, JB, Spratt, M, Royston, P, Kenward, MG, Wood, AM, Carpenter, JR (2009). Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 338, b2393.
Steyerberg, EW (2009). Clinical Prediction Models: a Practical Approach to Development, Validation, and Updating. Springer: New York.
Steyerberg, EW, Eijkemans, MJ, Habbema, JD (1999). Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. Journal of Clinical Epidemiology 52, 935942.
Steyerberg, EW, Eijkemans, MJ, Harrell, FE Jr., Habbema, JD (2001). Prognostic modeling with logistic regression analysis: in search of a sensible strategy in small data sets. Medical Decision Making 21, 4556.
Steyerberg, EW, van der Ploeg, T, Van Calster, B (2014). Risk prediction with machine learning and regression methods. Biometrical Journal 56, 601606.
Steyerberg, EW, Vergouwe, Y (2014). Towards better clinical prediction models: seven steps for development and an ABCD for validation. European Heart Journal 35, 19251931.
Steyerberg, EW, Vickers, AJ, Cook, NR, Gerds, T, Gonen, M, Obuchowski, N, Pencina, MJ, Kattan, MW (2010). Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21, 128138.
Stowkowy, J, Liu, L, Cadenhead, KS, Cannon, TD, Cornblatt, BA, McGlashan, TH, Perkins, DO, Seidman, LJ, Tsuang, MT, Walker, EF, Woods, SW, Bearden, CE, Mathalon, DH, Addington, J (2016). Early traumatic experiences, perceived discrimination and conversion to psychosis in those at clinical high risk for psychosis. Social Psychiatry and Psychiatric Epidemiology 51, 497503.
Strobl, EV, Eack, SM, Swaminathan, V, Visweswaran, S (2012). Predicting the risk of psychosis onset: advances and prospects. Early Intervention in Psychiatry 6, 368379.
Sun, GW, Shook, TL, Kay, GL (1996). Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. Journal of Clinical Epidemiology 49, 907916.
Thompson, A, Nelson, B, Yung, A (2011). Predictive validity of clinical variables in the “at risk” for psychosis population: international comparison with results from the North American Prodrome Longitudinal Study. Schizophrenia Research 126, 5157.
Tibshirani, R (1997). The lasso method for variable selection in the Cox model. Statistics in Medicine 16, 385395.
van der Net, JB, Janssens, ACJW, Eijkemans, MJC, Kastelein, JJP, Sijbrands, EJG, Steyerberg, EW (2008). Cox proportional hazards models have more statistical power than logistic regression models in cross-sectional genetic association studies. European Journal of Human Genetics 16, 11111116.
van der Ploeg, T, Austin, PC, Steyerberg, EW (2014). Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Medical Research Methodology 14, 137.
van der Ploeg, T, Nieboer, D, Steyerberg, EW (2016). Modern modeling techniques had limited external validity in predicting mortality from traumatic brain injury. Journal of Clinical Epidemiology 78, 83–89.
van Oort, L, van den Berg, T, Koes, BW, de Vet, RH, Anema, HJ, Heymans, MW, Verhagen, AP (2012). Preliminary state of development of prediction models for primary care physical therapy: a systematic review. Journal of Clinical Epidemiology 65, 12571266.
Velthorst, E, Derks, EM, Schothorst, P, Becker, H, Durston, S, Ziermans, T, Nieman, DH, de Haan, L (2013 a). Quantitative and qualitative symptomatic differences in individuals at ultra-high risk for psychosis and healthy controls. Psychiatry Research 210, 432437.
Velthorst, E, Nelson, B, Wiltink, S, de Haan, L, Wood, SJ, Lin, A, Yung, AR (2013 b). Transition to first episode psychosis in ultra high risk populations: does baseline functioning hold the key? Schizophrenia Research 143, 132137.
Walder, DJ, Holtzman, CW, Addington, J, Cadenhead, K, Tsuang, M, Cornblatt, B, Cannon, TD, McGlashan, TH, Woods, SW, Perkins, DO, Seidman, LJ, Heinssen, R, Walker, EF (2013). Sexual dimorphisms and prediction of conversion in the NAPLS psychosis prodrome. Schizophrenia Research 144, 4350.
Wessler, BS, Lai Yh, L, Kramer, W, Cangelosi, M, Raman, G, Lutz, JS, Kent, DM (2015). Clinical prediction models for cardiovascular disease: tufts predictive analytics and comparative effectiveness clinical prediction model database. Circulation. Cardiovascular Quality and Outcomes 8, 368375.
Wynants, L, Collins, GS, Van Calster, B (2016). Key steps and common pitfalls in developing and validating risk models. BJOG: An International Journal of Obstetrics and Gynaecology. Published online 30 June 2016. doi:10.1111/1471-0528.14170.
Xu, L, Zhang, T, Zheng, L, Li, H, Tang, Y, Luo, X, Sheng, J, Wang, J (2016). Psychometric Properties of Prodromal Questionnaire-Brief Version among Chinese help-seeking individuals. PLOS ONE 11, e0148935.
Yung, AR, Phillips, LJ, McGorry, PD, McFarlane, CA, Francey, S, Harrigan, S, Patton, GC, Jackson, HJ (1998). Prediction of psychosis. A step towards indicated prevention of schizophrenia. British Journal of Psychiatry. Supplement 172, 1420.
Yung, AR, Phillips, LJ, Yuen, HP, Francey, SM, McFarlane, CA, Hallgren, M, McGorry, PD (2003). Psychosis prediction: 12-month follow up of a high-risk (“prodromal”) group. Schizophrenia Research 60, 2132.
Yung, AR, Phillips, LJ, Yuen, HP, McGorry, PD (2004). Risk factors for psychosis in an ultra high-risk group: psychopathology and clinical features. Schizophrenia Research 67, 131142.
Ziermans, T, de Wit, S, Schothorst, P, Sprong, M, van Engeland, H, Kahn, R, Durston, S (2014). Neurocognitive and clinical predictors of long-term outcome in adolescents at ultra-high risk for psychosis: a 6-year follow-up. PLOS ONE 9, e93994.

Keywords

Type Description Title
WORD
Supplementary materials

Studerus supplementary material
Table S2

 Word (36 KB)
36 KB
WORD
Supplementary materials

Studerus supplementary material
Table S1

 Word (374 KB)
374 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed