Skip to main content Accessibility help

Development of a stage-dependent prognostic model to predict psychosis in ultra-high-risk patients seeking treatment for co-morbid psychiatric disorders

  • H. K. Ising (a1), S. Ruhrmann (a2), N. A. F. M. Burger (a1), J. Rietdijk (a1) (a3), S. Dragt (a4), R. M. C. Klaassen (a5), D. P. G. van den Berg (a1), D. H. Nieman (a4), N. Boonstra (a6), D. H. Linszen (a4), L. Wunderink (a6), F. Smit (a3) (a7) (a8), W. Veling (a9) and M. van der Gaag (a1) (a3)...



Current ultra-high-risk (UHR) criteria appear insufficient to predict imminent onset of first-episode psychosis, as a meta-analysis showed that about 20% of patients have a psychotic outcome after 2 years. Therefore, we aimed to develop a stage-dependent predictive model in UHR individuals who were seeking help for co-morbid disorders.


Baseline data on symptomatology, and environmental and psychological factors of 185 UHR patients (aged 14–35 years) participating in the Dutch Early Detection and Intervention Evaluation study were analysed with Cox proportional hazard analyses.


At 18 months, the overall transition rate was 17.3%. The final predictor model included five variables: observed blunted affect [hazard ratio (HR) 3.39, 95% confidence interval (CI) 1.56–7.35, p < 0.001], subjective complaints of impaired motor function (HR 5.88, 95% CI 1.21–6.10, p = 0.02), beliefs about social marginalization (HR 2.76, 95% CI 1.14–6.72, p = 0.03), decline in social functioning (HR 1.10, 95% CI 1.01–1.17, p = 0.03), and distress associated with suspiciousness (HR 1.02, 95% CI 1.00–1.03, p = 0.01). The positive predictive value of the model was 80.0%. The resulting prognostic index stratified the general risk into three risk classes with significantly different survival curves. In the highest risk class, transition to psychosis emerged on average ⩾8 months earlier than in the lowest risk class.


Predicting a first-episode psychosis in help-seeking UHR patients was improved using a stage-dependent prognostic model including negative psychotic symptoms (observed flattened affect, subjective impaired motor functioning), impaired social functioning and distress associated with suspiciousness. Treatment intensity may be stratified and personalized using the risk stratification.


Corresponding author

*Address for correspondence: H. Ising, MSc, Department of Psychiatry, Parnassia Psychiatric Institute, Zoutkeetsingel 40, 2512 HN The Hague, The Netherlands. (Email:


Hide All
Addington, J, Penn, D, Woods, SW, Addington, D, Perkins, DO (2008). Social functioning in individuals at clinical high risk for psychosis. Schizophrenia Research 99, 119124.
Andrews, G, Peters, L (1998). The psychometric properties of the Composite International Diagnostic Interview. Social Psychiatry and Psychiatric Epidemiology 33, 8088.
APA (1987). Diagnostic and Statistical Manual of Mental Disorders: DSM-III-R. Washington, DC: American Psychiatric Association.
Beck, A, Steer, R, Brown, G (1996). Manual for Beck Depression Inventory – II. Psychological Corporation: San Antonio, TX.
Birchwood, M, Jackson, C, Brunet, K, Holden, J, Barton, K (2012). Personal beliefs about illness questionnaire-revised (PBIQ-R): reliability and validation in a first episode sample. British Journal of Clinical Psychology 51, 448458.
Brett, C, Heriot-Maitland, C, McGuire, P, Peters, E (2014). Predictors of distress associated with psychotic-like anomalous experiences in clinical and non-clinical populations. British Journal of Clinical Psychology 53, 213227.
Cannon, TD, Cadenhead, K, Cornblatt, B, Woods, SW, Addington, J, Walker, E, Seidman, LJ, Perkins, D, Tsuang, M, McGlashan, T, Heinssen, R (2008). Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Archives of General Psychiatry 65, 2837.
Cornblatt, BA, Carrión, RE, Addington, J, Seidman, L, Walker, EF, Cannon, TD, Cadenhead, KS, McGlashan, TH, Perkins, DO, Tsuang, MT, Woods, SW, Heinssen, R, Lencz, T (2012). Risk factors for psychosis: impaired social and role functioning. Schizophrenia Bulletin 38, 12471257.
Cornblatt, BA, Carrión, RE, Auther, A, McLaughlin, D, Olsen, RH, John, M, Correll, CU (2015). Psychosis prevention: a modified clinical high risk perspective from the recognition and prevention (RAP) program. American Journal of Psychiatry 172, 986994.
Cornblatt, BA, Lencz, T, Smith, CW, Correll, CU, Auther, AM, Nakayama, E (2003). The schizophrenia prodrome revisited: a neurodevelopmental perspective. Schizophrenia Bulletin 29, 633651.
Cornblatt, B, Lencz, T, Obuchowski, M (2002). The schizophrenia prodrome: treatment and high-risk perspectives. Schizophr Research 57, 177186.
Demjaha, A, Valmaggia, L, Stahl, D, Byrne, M, McGuire, P (2012). Disorganization/cognitive and negative symptom dimensions in the at-risk mental state predict subsequent transition to psychosis. Schizophrenia Bulletin 38, 351359.
Fusar-Poli, P, Bonoldi, I, Yung, AR, Borgwardt, S, Kempton, MJ, Valmaggia, L, Barale, F, Caverzasi, E, McGuire, P (2012). Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. Archives of General Psychiatry 69, 220229.
Fusar-Poli, P, Byrne, M, Valmaggia, L, Day, F, Tabraham, P, Johns, L, McGuire, P (2010). Social dysfunction predicts two years clinical outcome in people at ultra high risk for psychosis. Journal of Psychiatric Research 44, 294301.
Fusar-Poli, P, Meneghelli, A, Valmaggia, L, Allen, P, Galvan, F, McGuire, P, Cocchi, A (2009). Duration of untreated prodromal symptoms and 12-month functional outcome of individuals at risk of psychosis. British Journal of Psychiatry 194, 181182.
Goldman, HH, Skodol, AE, Lave, TR (1992). Revising axis V for DSM-IV: a review of measures of social functioning. American Journal of Psychiatry 149, 11481156.
Guillem, F, Pampoulova, T, Stip, E, Lalonde, P, Todorov, C (2005). The relationships between symptom dimensions and dysphoria in schizophrenia. Schizophrenia Research 75, 8396.
Hosmer, DW Jr., Lemeshow, S, May, S (1999). Applied Survival Analysis: Regression Modeling of Time to Event Data. John Wiley & Sons: New York.
Huber, G (1983). The concept of substrate-close basic symptoms and its significance for the theory and therapy of schizophrenic diseases. Der Nervenarzt 54, 2332.
Ising, HK, Veling, W, Loewy, RL, Rietveld, MW, Rietdijk, J, Dragt, S, Klaassen, RMC, Nieman, DH, Wunderink, L, Linszen, DH, van der Gaag, M (2012). The validity of the 16-item version of the Prodromal Questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophrenia Bulletin 38, 12881296.
Krabbendam, L, Myin-Germeys, I, Hanssen, M, Bijl, RV, de Graaf, R, Vollebergh, W, Bak, M, van Os, J (2004). Hallucinatory experiences and onset of psychotic disorder: evidence that the risk is mediated by delusion formation. Acta Psychiatrica Scandinavica 110, 264272.
Lencz, T, Smith, CW, Auther, A, Correll, CU, Cornblatt, B (2004). Nonspecific and attenuated negative symptoms in patients at clinical high-risk for schizophrenia. Schizophrenia Research 68, 3748.
Loughin, TM (1998). On the bootstrap and monotone likelihood in the cox proportional hazards regression model. Lifetime Data Analysis 4, 393403.
Manschreck, TC, Chun, J, Merrill, AM, Maher, BA, Boshes, RA, Glatt, SJ, Faraone, SV, Tsuang, MT, Seidman, LJ (2015). Impaired motor performance in adolescents at familial high-risk for schizophrenia. Schizophrenia Research 168, 4449.
Mason, O, Startup, M, Halpin, S, Schall, U, Conrad, A, Carr, V (2004). Risk factors for transition to first episode psychosis among individuals with ‘at-risk mental states’. Schizophrenia Research 71, 227237.
Mattick, RP, Clarke, JC (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety11Editor's note: this article was written before the development of some contemporary measures of social phobia, such as the Social Phobia and Anxiety Inventory. Behaviour Research and Therapy 36, 455470.
McGorry, PD, Hickie, IB, Yung, AR, Pantelis, C, Jackson, HJ (2006). Clinical staging of psychiatric disorders: a heuristic framework for choosing earlier, safer and more effective interventions. Australian and New Zealand Journal of Psychiatry 40, 616622.
McGorry, PD, Yung, AR, Bechdolf, A, Amminger, P (2008). Back to the future: predicting and reshaping the course of psychotic disorder. Archives of General Psychiatry 65, 2527.
Michel, C, Ruhrmann, S, Schimmelmann, BG, Klosterkotter, J, Schultze-Lutter, F (2014). A stratified model for psychosis prediction in clinical practice. Schizophrenia Bulletin 40, 15331542.
Miller, TJ, McGlashan, TH, Rosen, JL, Somjee, L, Markovich, PJ, Stein, K, Woods, SW (2002). Prospective diagnosis of the initial prodrome for schizophrenia based on the Structured Interview for Prodromal Syndromes: preliminary evidence of interrater reliability and predictive validity. American Journal of Psychiatry 159, 863865.
Miller, TJ, Zipursky, RB, Perkins, D, Addington, J, Woods, SW, Hawkins, Ka, Hoffman, R, Preda, A, Epstein, I, Addington, D, Lindborg, S, Marquez, E, Tohen, M, Breier, A, McGlashan, TH (2003). The PRIME North America randomized double-blind clinical trial of olanzapine versus placebo in patients at risk of being prodromally symptomatic for psychosis. Schizophrenia Research 61, 1930.
Morrison, AP, French, P, Parker, S, Roberts, M, Stevens, H, Bentall, RP, Lewis, SW (2007). Three-year follow-up of a randomized controlled trial of cognitive therapy for the prevention of psychosis in people at ultrahigh risk. Schizophrenia Bulletin 33, 682687.
Nelson, B, Yuen, HP, Wood, SJ, Lin, A, Spiliotacopoulos, D, Bruxner, A, Broussard, C, Simmons, M, Foley, DL, Brewer, WJ, Francey, SM, Amminger, GP, Thompson, A, McGorry, PD, Yung, AR (2013). Long-term Follow-up of a Group at Ultra High Risk (‘Prodromal’) for Psychosis: The PACE 400 Study. JAMA Psychiatry 70, 793802.
Nieman, DH, Ruhrmann, S, Dragt, S, Soen, F, van Tricht, MJ, Koelman, JHTM, Bour, LJ, Velthorst, E, Becker, HE, Weiser, M, Linszen, DH, de Haan, L (2014). Psychosis prediction: stratification of risk estimation with information-processing and premorbid functioning variables. Schizophrenia Bulletin 40, 14821490.
Power, L, Polari, AR, Yung, AR, McGorry, PD, Nelson, B (2015). Distress in relation to attenuated psychotic symptoms in the ultra-high-risk population is not associated with increased risk of psychotic disorder. Early Intervention in Psychiatry. Published online: 15 March 2015. doi:10.1111/eip.12233.
Priebe, S, Huxley, P, Knight, S, Evans, S (1999). Application and results of the Manchester Short Assessment of Quality of Life (MANSA). International Journal of Social Psychiatry 45, 712.
Riecher-Rössler, A, Pflueger, MO, Aston, J, Borgwardt, SJ, Brewer, WJ, Gschwandtner, U, Stieglitz, R-D (2009). Efficacy of using cognitive status in predicting psychosis: a 7-year follow-up. Biological Psychiatry 66, 10231030.
Rietdijk, J, Dragt, S, Klaassen, R, Ising, H, Nieman, D, Wunderink, L, Delespaul, P, Cuijpers, P, Linszen, D, van der Gaag, M (2010). A single blind randomized controlled trial of cognitive behavioural therapy in a help-seeking population with an At Risk Mental State for psychosis: the Dutch Early Detection and Intervention Evaluation (EDIE-NL) trial. Trials 11, 30.
Rijnders, CAT, van den Berg, JFM, Hodiamont, PPG, Nienhuis, FJ, Furer, JW, Mulder, J, Giel, R (2000). Psychometric properties of the schedules for clinical assessment in neuropsychiatry (SCAN-2.1). Social Psychiatry and Psychiatric Epidemiology 35, 348352.
Ruhrmann, S, Schultze-Lutter, F, Klosterkötter, J (2010 a). Probably at-risk, but certainly ill – advocating the introduction of a psychosis spectrum disorder in DSM-V. Schizophrenia Research 120, 2337.
Ruhrmann, S, Schultze-Lutter, F, Salokangas, RKR, Heinimaa, M, Linszen, D, Dingemans, P, Birchwood, M, Patterson, P, Juckel, G, Heinz, A, Morrison, A, Lewis, S, von Reventlow, HG, Klosterkötter, J (2010 b). Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European prediction of psychosis study. Archives of General Psychiatry 67, 241251.
Schultze-Lutter, F (2009). Subjective symptoms of schizophrenia in research and the clinic: the basic symptom concept. Schizophrenia Bulletin 35, 58.
Schultze-Lutter, F, Ruhrmann, S, Picker, H, von Reventlow, HG, Brockhaus-Dumke, A, Klosterkötter, J (2007). Basic symptoms in early psychotic and depressive disorders. British Journal of Psychiatry. Supplement 51, s31s37.
Simon, AE, Umbricht, D (2010). High remission rates from an initial ultra-high risk state for psychosis. Schizophrenia Research 116, 168172.
Tropsha, A, Gramatica, P, Gombar, V (2003). The importance of being earnest: validation is the absolute essential for successful application and interpretation of QSPR models. QSAR & Combinatorial Science 22, 6977.
Valmaggia, LR, Stahl, D, Yung, AR, Nelson, B, Fusar-Poli, P, McGorry, PD, McGuire, PK (2013). Negative psychotic symptoms and impaired role functioning predict transition outcomes in the at-risk mental state: a latent class cluster analysis study. Psychological Medicine 43, 23112325.
van der Gaag, M, Nieman, D, van den Berg, D (2013). CBT for Those at Risk of a First Episode Psychosis: Evidence-based psychotherapy for people with an ‘At Risk Mental State’. Routledge Taylor & Francis Group: London and New York.
van der Gaag, M, Nieman, DH, Rietdijk, J, Dragt, S, Ising, HK, Klaassen, RMC, Koeter, M, Cuijpers, P, Wunderink, L, Linszen, DH (2012). Cognitive behavioral therapy for subjects at ultrahigh risk for developing psychosis: a randomized controlled clinical trial. Schizophrenia Bulletin 38, 11801188.
van Os, J, Linscott, RJ, Myin-Germeys, I, Delespaul, P, Krabbendam, L, Van Os, J (2009). A systematic review and meta-analysis of the psychosis continuum: evidence for a psychosis proneness-persistence-impairment model of psychotic disorder. Psychological Medicine 39, 179195.
Velthorst, E, Nieman, DH, Klaassen, RMC, Becker, HE, Dingemans, PM, Linszen, DH, De Haan, L (2011). Three-year course of clinical symptomatology in young people at ultra high risk for transition to psychosis. Acta Psychiatrica Scandinavica 123, 3642.
Velthorst, E, Nieman, DH, Linszen, D, Becker, H, de Haan, L, Dingemans, PM, Birchwood, M, Patterson, P, Salokangas, RKR, Heinimaa, M, Heinz, A, Juckel, G, von Reventlow, HG, French, P, Stevens, H, Schultze-Lutter, F, Klosterkötter, J, Ruhrmann, S (2010). Disability in people clinically at high risk of psychosis. British Journal of Psychiatry 197, 278284.
Vittinghoff, E, McCulloch, CE (2007). Relaxing the rule of ten events per variable in logistic and Cox regression. American Journal of Epidemiology 165, 710718.
Yung, AR, Buckby, Ja, Cosgrave, EM, Killackey, EJ, Baker, K, Cotton, SM, McGorry, PD (2007). Association between psychotic experiences and depression in a clinical sample over 6 months. Schizophrenia Research 91, 246253.
Yung, AR, Nelson, B, Stanford, C, Simmons, MB, Cosgrave, EM, Killackey, E, Phillips, LJ, Bechdolf, A, Buckby, J, McGorry, PD (2008). Validation of ‘prodromal’ criteria to detect individuals at ultra high risk of psychosis: 2 year follow-up. Schizophrenia Research 105, 1017.
Yung, AR, Nelson, B, Thompson, A, Wood, SJ (2010). The psychosis threshold in Ultra High Risk (prodromal) research: is it valid? Schizophrenia Research 120, 16.
Yung, AR, Phillips, LJ, Yuen, HP, Mcgorry, PD, Pan, H (2004). Risk factors for psychosis in an ultra high-risk group: psychopathology and clinical features. Schizophrenia Research 67, 131142.
Yung, AR, Stanford, C, Cosgrave, E, Killackey, E, Phillips, L, Nelson, B, McGorry, PD (2006). Testing the Ultra High Risk (prodromal) criteria for the prediction of psychosis in a clinical sample of young people. Schizophrenia Research 84, 5766.
Yung, AR, Yuen, HP, McGorry, PD, Phillips, LJ, Kelly, D, Dell'Olio, M, Francey, SM, Cosgrave, EM, Killackey, E, Stanford, C, Godfrey, K, Buckby, J (2005). Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Australian and New Zealand Journal of Psychiatry 39, 964971.


Type Description Title
Supplementary materials

Ising supplementary material
Table S1

 Word (68 KB)
68 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed