Hostname: page-component-848d4c4894-8bljj Total loading time: 0 Render date: 2024-06-22T15:57:14.608Z Has data issue: false hasContentIssue false
  • English
  • Français

Are we failing clinical trials? A case for strong aggregate outcomes

Published online by Cambridge University Press:  29 June 2017

D. W. Joyce ,
D. K. Tracy  and
S. S. Shergill
D. W. Joyce*
Affiliation:
Cognition Schizophrenia and Imaging Laboratory, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London SE5 8AF, PO63, UK
D. K. Tracy
Affiliation:
Cognition Schizophrenia and Imaging Laboratory, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London SE5 8AF, PO63, UK Oxleas NHS Foundation Trust, London, UK
S. S. Shergill
Affiliation:
Cognition Schizophrenia and Imaging Laboratory, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London SE5 8AF, PO63, UK
*
*Address for correspondence: D. W. Joyce, Cognition Schizophrenia and Imaging Laboratory, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London SE5 8AF, PO63, UK. (Email: danjoyce@doctors.org.uk)
Get access Rights & Permissions [Opens in a new window]

Abstract

Clinical trials in psychiatry inherit methods for design and statistical analysis from evidence-based medicine. However, trials in other clinical disciplines benefit from a more specific relationship between instruments that measure disease state (e.g. biomarkers, clinical signs), the underlying pathology and diagnosis such that primary outcomes can be readily defined. Trials in psychiatry use diagnosis (i.e. a categorical label for a syndrome) as a proxy for the underlying disorder, and outcomes are defined, for example, as a percentage change in a univariate total score on some clinical instrument. We label this approach to defining outcomes weak aggregation of disease state. Univariate measures are necessary, because statistical methodology is both tractable and well-developed for scalar outcomes, but we show that weak aggregate approaches do not capture disease state sufficiently, potentially leading to loss of information about response to intervention. We demonstrate how multivariate disease state can be captured using geometric concepts of spaces defined over routine clinical instruments, and show how clinically meaningful disease states (e.g. representing different profiles of symptoms, recovery or remission) can be defined as prototypes (geometric locations) in these spaces. Then, we show how to derive univariate (scalar) measures, which capture patient's relationships to these prototypes and argue these represent strong aggregates of disease state that may be a better basis for outcome measures. We demonstrate our proposal using a large publically available dataset. We conclude by discussing the impact of strong aggregates for analyses in traditional and novel trial designs.

Keywords

Trialsmethodologyoutcomesdimensional definitions of disorder
Type
Invited Review
Information
Psychological Medicine , Volume 48 , Issue 2 , January 2018 , pp. 177 - 186
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Andreasen, NC, Carpenter, WT, Kane, JM, Lasser, RA, Marder, SR and Weinberger, DR (2005) ‘Remission in Schizophrenia: Proposed Criteria and Rationale for Consensus’, Am J Psychiatry, 162(3), pp. 441–449.CrossRefGoogle ScholarPubMed
Annen, S, Roser, P and Brüne, M (2012) ‘Nonverbal behavior during clinical interviews: similarities and dissimilarities among schizophrenia, mania, and depression.’, The Journal of nervous and mental disease, 200, pp. 26–32.CrossRefGoogle ScholarPubMed
Barnow, S, Arens, EA, Sieswerda, S, Dinu-Biringer, R, Spitzer, C and Lang, S (2010) ‘Borderline personality disorder and psychosis: A review’, Current Psychiatry Reports, 12, pp. 186–195.CrossRefGoogle ScholarPubMed
Berkowitz, RL, Patel, U, Ni, Q, Parks, JJ and Docherty, JP (2012) ‘The impact of the clinical antipsychotic trials of intervention effectiveness (CATIE) on prescribing practices: An analysis of data from a large midwestern state’, Journal of Clinical Psychiatry, 73, pp. 498–503.CrossRefGoogle ScholarPubMed
Craddock, N, O'Donovan, MC and Owen, MJ (2009) ‘Psychosis genetics: Modeling the relationship between schizophrenia, bipolar disorder and mixed (or “schizoaffective”) psychoses’, Schizophrenia Bulletin, 35(3), pp. 482–490.CrossRefGoogle ScholarPubMed
Cuthbert, BN and Insel, TR (2013) ‘Toward the future of psychiatric diagnosis: the seven pillars of RDoC.’, BMC medicine, 11, p. 126.CrossRefGoogle ScholarPubMed
Demjaha, A, Morgan, K, Morgan, C, Landau, S, Dean, K, Reichenberg, A, Sham, P, Fearon, P, Hutchinson, G, Jones, PB, Murray, RM and Dazzan, P (2009) ‘Combining dimensional and categorical representation of psychosis: the way forward for DSM-V and ICD-11?’, Psychological medicine, 39, pp. 1943–1955.CrossRefGoogle ScholarPubMed
Demjaha, A, MacCabe, JH and Murray, RM (2012) ‘How genes and environmental factors determine the different neurodevelopmental trajectories of schizophrenia and bipolar disorder’, Schizophrenia Bulletin, 38, pp. 209–214.CrossRefGoogle ScholarPubMed
Frank, E, Prien, RF, Jarrett, RB, Keller, MB, Kupfer, DJ, Lavori, PW, Rush, AJ and Weissman, MM (1991) ‘Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Remission, recovery, relapse, and recurrence.’, Archives of general psychiatry, 48(9), pp. 851–5.Google Scholar
Glaser, JP, Van Os, J, Thewissen, V and Myin-Germeys, I (2010) ‘Psychotic reactivity in borderline personality disorder’, Acta Psychiatrica Scandinavica, 121, pp. 125–134.CrossRefGoogle ScholarPubMed
Green, MF (2006) ‘Cognitive impairment and functional outcome in schizophrenia and bipolar disorder.’, The Journal of clinical psychiatry, 67, pp. 3–8.Google Scholar
Insel, T, Cuthbert, B, Garvey, M, Heinssen, R, Pine, DS, Quinn, K, Sanislow, C and Wang, P (2010) ‘Research Domain Criteria (RDoC): Toward a new classification framework for research on mental disorders’, American Journal of Psychiatry, 167, pp. 748–751.CrossRefGoogle Scholar
Jabben, N, Arts, B, Krabbendam, L and Van Os, J (2009) ‘Investigating the association between neurocognition and psychosis in bipolar disorder: Further evidence for the overlap with schizophrenia’, Bipolar Disorders, 11, pp. 166–177.Google Scholar
Jablensky, A (2006) ‘Subtyping schizophrenia: implications for genetic research.’, Molecular psychiatry, 11(9), pp. 815–36.CrossRefGoogle ScholarPubMed
Jakubovski, E, Carlson, JP and Bloch, MH (2015) ‘Prognostic Subgroups for Remission, Response, and Treatment Continuation in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Trial’, The Journal of Clinical Psychiatry, 76(11), pp. 1535–1545.CrossRefGoogle ScholarPubMed
Joyce, DW, Kehagia, AA, Tracy, DK, Proctor, J and Shergill, SS (2017) ‘Realising stratified psychiatry using multidimensional signatures and trajectories’, Journal of Translational Medicine. BioMed Central, 15(1), p. 15.Google Scholar
Kay, SR, Fiszbein, A and Opler, L (1987) ‘The positive and negative syndrome scale (PANSS) for schizophrenia.’, Schizophrenia bulletin, 13, pp. 261–76.Google Scholar
Kendler, KS (2016) ‘The nature of psychiatric disorders.’, World psychiatry. World Psychiatric Association, 15(1), pp. 5–12.Google Scholar
Keshavan, MS, Morris, DW, Sweeney, JA, Pearlson, G, Thaker, G, Seidman, LJ, Eack, SM and Tamminga, C (2011) ‘A dimensional approach to the psychosis spectrum between bipolar disorder and schizophrenia: The Schizo-Bipolar Scale’, Schizophrenia Research, 133, pp. 250–254.Google Scholar
Kohonen, T (1995) Self-Organizing Maps. Berlin: Springer.Google Scholar
Krzanowski, WJ (2000) Principles of Multivariate Analysis. A User's Perspective, Oxford Statistical Science Series No. 23.Google Scholar
Leucht, S, Davis, JM, Engel, RR, Kissling, W and Kane, JM (2009) ‘Definitions of response and remission in schizophrenia: recommendations for their use and their presentation’, Acta Psychiatrica Scandinavica, 119(438), pp. 7–14.CrossRefGoogle Scholar
Lichtenstein, P, Yip, BH, Björk, C, Pawitan, Y, Cannon, TD, Sullivan, PF and Hultman, CM (2009) ‘Common genetic determinants of schizophrenia and bipolar disorder in Swedish families: a population-based study.’, Lancet, 373, pp. 234–239.Google Scholar
Lindenmayer, JP, Grochowski, S, Hyman, RB, Powchik, P and Davidson, M (1995) ‘Five factor model of schizophrenia: replication across samples.’, Schizophrenia research. Netherlands, Amsterdam, 14(3), pp. 229–34.Google Scholar
Marquand, AF, Wolfers, T, Mennes, M, Buitelaar, J and Beckmann, CF (2016) ‘Beyond Lumping and Splitting: A Review of Computational Approaches for Stratifying Psychiatric Disorders’, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1(5), pp. 433–447.Google ScholarPubMed
McCullagh, P and Nelder, JA (1989) Generalized Linear Models. Second Edi. Chapman & Hall/CRC.CrossRefGoogle Scholar
Moore, A, Derry, S, Eccleston, C and Kalso, E (2013) ‘Expect analgesic failure; pursue analgesic success’, British Medical Journal, 346(May), pp. 2690–2690.Google Scholar
Moore, R, Derry, S, McQuay, HJ, Straube, S, Aldington, D, Wiffen, P, Bell, RF, Kalso, E and Rowbotham, MC (2010) ‘Clinical effectiveness: An approach to clinical trial design more relevant to clinical practice, acknowledging the importance of individual differences’, Pain. International Association for the Study of Pain, 149(2), pp. 173–176.Google Scholar
Morris, SE and Cuthbert, BN (2012) ‘Research domain criteria: Cognitive systems, neural circuits, and dimensions of behavior’, Dialogues in Clinical Neuroscience, 14, pp. 29–37.Google Scholar
Nishizono-Maher, A, Ikuta, N, Ogiso, Y, Moriya, N, Miyake, Y and Minakawa, K (1993) ‘Psychotic symptoms in depression and borderline personality disorder’, Journal of Affective Disorders, 28, pp. 279–285.Google Scholar
Overall, JE and Gorham, DR (1962) ‘THE BRIEF PSYCHIATRIC RATING SCALE’, Psychological Reports. Ammons Scientific, 10(3), pp. 799–812.Google Scholar
Purcell, SM, Wray, NR, Stone, JL, Visscher, PM, O'Donovan, MC, Sullivan, PF and Sklar, P (2009) ‘Common polygenic variation contributes to risk of schizophrenia and bipolar disorder.’, Nature, 460, pp. 748–752.Google Scholar
Reininghaus, U, Böhnke, JR, Hosang, G, Farmer, A, Burns, T, McGuffin, P, Bentall, RP, Insel, TR, Adam, D, First, MB, Reed, GM, Hyman, SE, Saxena, S, Bebbington, P, Caspi, AHR, Belsky, DW, Goldman-Mellor, SJ, Harrington, H, Israel, S, Meier, MH, Carpenter, WT, Bustillo, JR, Thaker, GK, Os, J van, Krueger, RF, Green, MJ, Craddock, N, Owen, MJ, Cardno, AG, Rijsdijk, FV, Sham, PC, Murray, RM, McGuffin, P, Lichtenstein, P, Yip, BH, Bjork, C, Pawitan, Y, Cannon, TD, Sullivan, PF, Brown, AS, Os, J van, Driessens, C, Hoek, HW, Susser, ES, Clarke, MC, Harley, M, Cannon, M, Matheson, SL, Shepherd, AM, Pinchbeck, RM, Laurens, KR, Carr, VJ, Henquet, C, Krabbendam, L, Graaf, R, Have, M, Os, J van, Henquet, C, Murray, R, Linszen, D, Os, J van, Heinz, A, Deserno, L, Reininghaus, U, Kirkbride, JB, Errazuriz, A, Croudace, TJ, Morgan, C, Jackson, D, Boydell, J, Os, J van, Kapur, S, Reininghaus, U, Priebe, S, Bentall, RP, Russo, M, Levine, SZ, Demjaha, A, Forti, MD, Bonaccorso, S, Fearon, P, McGuffin, P, Farmer, A, Harvey, I, Burns, T, Creed, F, Fahy, T, Thompson, S, Tyrer, P, White, I, Cohen-Woods, S, Craig, I, Gaysina, D, Gray, J, Gunasinghe, C, Craddock, N, Spitzer, RL, Endicott, J, Robins, E, Rucker, J, Newman, S, Gray, J, Gunasinghe, C, Broadbent, M, Brittain, P, Chalmers, RP, Kass, RW, Wasserman, L, Peterson, LE, Coleman, MA, Kohavi, R, Efron, BG, Gong, G, Böhnke, JR, Croudace, TJ, Cardno, AG, Jones, LA, Murphy, KC, Murphy, KC, Asherson, P, Scott, LC, Markon, KE, Reininghaus, U, Morgan, C, Simpson, J, Dazzan, P, Morgan, K, Doody, G, Morgan, C, Reininghaus, U, Fearon, P, Hutchinson, G, Morgan, K, Dazzan, P, Reininghaus, U, Craig, T, Fisher, H, Hutchinson, G, Fearon, P, Morgan, K, Tamminga, CA, Ivleva, EI, Keshavan, MS, Pearlson, GD, Clementz, BA and Witte, B (2016) ‘Evaluation of the validity and utility of a transdiagnostic psychosis dimension encompassing schizophrenia and bipolar disorder.’, The British journal of psychiatry: the journal of mental science, 209(2), pp. 107–13.Google Scholar
Ripke, S, Neale, BM, Corvin, A, Walters, JTR, Farh, K-H, Holmans, Pa, Lee, P, Bulik-Sullivan, B, Collier, Da, Huang, H, Pers, TH, Agartz, I, Agerbo, E, Albus, M, Alexander, M, Amin, F, Bacanu, Sa, Begemann, M, Belliveau, Ra Jr, Bene, J, Bergen, SE, Bevilacqua, E, Bigdeli, TB, Black, DW, Bruggeman, R, Buccola, NG, Buckner, RL, Byerley, W, Cahn, W, Cai, G, Campion, D, Cantor, RM, Carr, VJ, Carrera, N, Catts, SV, Chambert, KD, Chan, RCK, Chen, RYL, Chen, EYH, Cheng, W, Cheung, EFC, Ann Chong, S, Robert Cloninger, C, Cohen, D, Cohen, N, Cormican, P, Craddock, N, Crowley, JJ, Curtis, D, Davidson, M, Davis, KL, Degenhardt, F, Del Favero, J, Demontis, D, Dikeos, D, Dinan, T, Djurovic, S, Donohoe, G, Drapeau, E, Duan, J, Dudbridge, F, Durmishi, N, Eichhammer, P, Eriksson, J, Escott-Price, V, Essioux, L, Fanous, AH, Farrell, MS, Frank, J, Franke, L, Freedman, R, Freimer, NB, Friedl, M, Friedman, JI, Fromer, M, Genovese, G, Georgieva, L, Giegling, I, Giusti-Rodríguez, P, Godard, S, Goldstein, JI, Golimbet, V, Gopal, S, Gratten, J, de Haan, L, Hammer, C, Hamshere, ML, Hansen, M, Hansen, T, Haroutunian, V, Hartmann, AM, Henskens, Fa, Herms, S, Hirschhorn, JN, Hoffmann, P, Hofman, A, Hollegaard, MV, Hougaard, DM, Ikeda, M, Joa, I, Julià, A, Kahn, RS, Kalaydjieva, L, Karachanak-Yankova, S, Karjalainen, J, Kavanagh, D, Keller, MC, Kennedy, JL, Khrunin, A, Kim, Y, Klovins, J, Knowles, Ja, Konte, B, Kucinskas, V, Ausrele Kucinskiene, Z, Kuzelova-Ptackova, H, Kähler, AK, Laurent, C, Lee Chee Keong, J, Hong Lee, S, Legge, SE, Lerer, B, Li, M, Li, T, Liang, K-Y, Lieberman, J, Limborska, S, Loughland, CM, Lubinski, J, Lönnqvist, J, Macek, M Jr, Magnusson, PKE, Maher, BS, Maier, W, Mallet, J, Marsal, S, Mattheisen, M, Mattingsdal, M, McCarley, RW, McDonald, C, McIntosh, AM, Meier, S, Meijer, CJ, Melegh, B, Melle, I, Mesholam-Gately, RI, Metspalu, A, Michie, PT, Milani, L, Milanova, V, Mokrab, Y, Morris, DW, Mors, O, Murphy, KC, Murray, RM, Myin-Germeys, I, Müller-Myhsok, B, Nelis, M, Nenadic, I, Nertney, Da, Nestadt, G, Nicodemus, KK, Nikitina-Zake, L, Nisenbaum, L, Nordin, A, O'Callaghan, E, O'Dushlaine, C, O'Neill, FA, Oh, S-Y, Olincy, A, Olsen, L, Van Os, J, Endophenotypes International Consortium P, Pantelis, C, Papadimitriou, GN, Papiol, S, Parkhomenko, E, Pato, MT, Paunio, T, Pejovic-Milovancevic, M, Perkins, DO, Pietiläinen, O, Pimm, J, Pocklington, AJ, Powell, J, Price, A, Pulver, AE, Purcell, SM, Quested, D, Rasmussen, HB, Reichenberg, A, Reimers, Ma, Richards, AL, Roffman, JL, Roussos, P, Ruderfer, DM, Salomaa, V, Sanders, AR, Schall, U, Schubert, CR, Schulze, TG, Schwab, SG, Scolnick, EM, Scott, RJ, Seidman, LJ, Shi, J, Sigurdsson, E, Silagadze, T, Silverman, JM, Sim, K, Slominsky, P, Smoller, JW, So, H-C, Spencer, CCA, Stahl, EA, Stefansson, H, Steinberg, S, Stogmann, E, Straub, RE, Strengman, E, Strohmaier, J, Scott Stroup, T, Subramaniam, M, Suvisaari, J, Svrakic, DM, Szatkiewicz, JP, Söderman, E, Thirumalai, S, Toncheva, D, Tosato, S, Veijola, J, Waddington, J, Walsh, D, Wang, D, Wang, Q, Webb, BT, Weiser, M, Wildenauer, DB, Williams, NM, Williams, S, Witt, SH, Wolen, AR, Wong, EHM, Wormley, BK, Simon Xi, H, Zai, CC, Zheng, X, Zimprich, F, Wray, NR, Stefansson, K, Visscher, PM, Trust Case-Control Consortium, W, Adolfsson, R, Andreassen, Oa, Blackwood, DHR, Bramon, E, Buxbaum, JD, Børglum, AD, Cichon, S, Darvasi, A, Domenici, E, Ehrenreich, H, Esko, T, Gejman, PV, Gill, M, Gurling, H, Hultman, CM, Iwata, N, Jablensky, AV, Jönsson, EG, Kendler, KS, Kirov, G, Knight, J, Lencz, T, Levinson, DF, Li, QS, Liu, J, Malhotra, AK, McCarroll, Sa, McQuillin, A, Moran, JL, Mortensen, PB, Mowry, BJ, Nöthen, MM, Ophoff, Ra, Owen, MJ, Palotie, A, Pato, CN, Petryshen, TL, Posthuma, D, Rietschel, M, Riley, BP, Rujescu, D, Sham, PC, Sklar, P, St Clair, D, Weinberger, DR, Wendland, JR, Werge, T, Daly, MJ, Sullivan, PF and O'Donovan, MC (2014) ‘Biological insights from 108 schizophrenia-associated genetic loci’, Nature, 511, pp. 421–427.Google Scholar
Roweis, ST and Saul, LK (2000) ‘Nonlinear Dimensionality Reduction by Locally Linear Embedding’, Science, 290(5500), pp. 2323–2326.CrossRefGoogle ScholarPubMed
Schork, NJ (2015) ‘Time for one-person trials’, Nature, 520(7549), pp. 609–611.Google Scholar
Schroeder, K, Fisher, HL and Schäfer, I (2013) ‘Psychotic symptoms in patients with borderline personality disorder: prevalence and clinical management.’, Current opinion in psychiatry, 26, pp. 113–9.Google Scholar
Schumann, G, Binder, EB, Holte, A, de Kloet, E.R, Oedegaard, KJ, Robbins, TW, Walker-Tilley, TR, Bitter, I, Brown, VJ, Buitelaar, J, Ciccocioppo, R, Cools, R, Escera, C, Fleischhacker, W, Flor, H, Frith, CD, Heinz, A, Johnsen, E, Kirschbaum, C, Klingberg, T, Lesch, KP, Lewis, S, Maier, W, Mann, K, Martinot, JL, Meyer-Lindenberg, A, Müller, CP, Müller, WE, Nutt, DJ, Persico, A, Perugi, G, Pessiglione, M, Preuss, UW, Roiser, JP, Rossini, PM, Rybakowski, JK, Sandi, C, Stephan, KE, Undurraga, J, Vieta, E, van der Wee, N, Wykes, T, Haro, JM and Wittchen, HU (2014) ‘Stratified medicine for mental disorders’, European Neuropsychopharmacology, 24(1), pp. 5–50.CrossRefGoogle ScholarPubMed
Strang, G (2004) Linear Algebra and Its Applications. 4th edn. WB Saunders. doi: 10.2307/2003783.Google Scholar
Stroup, T.S, Mcevoy, JP, Swartz, MS, Byerly, MJ, Qlick, ID, Canive, JM, Mcqee, MF, Simpson, QM, Stevens, MC and Lieberman, JA (2003) ‘The National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project: schizophrenia trial design and protocol development.’, Schizophrenia bulletin, 29(11)CrossRefGoogle ScholarPubMed
Stroup, TS, Mcevoy, JP, Swartz, MS, et The National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project: schizophrenia trial design and protocol development. Schizophr Bull. 2003;29(1):15–31.Google Scholar
Tenenbaum, JB, de Silva, V and Langford, JC (2000) ‘A global geometric framework for nonlinear dimensionality reduction.’, Science, 290(5500), pp. 2319–23.Google Scholar
Toh, WL, Thomas, N and Rossell, SL (2015) ‘Auditory verbal hallucinations in bipolar disorder (BD) and major depressive disorder (MDD): A systematic review’, Journal of Affective Disorders, 184, pp. 18–28.Google Scholar
Walker, J, Curtis, V and Murray, RM (2002) ‘Schizophrenia and bipolar disorder: similarities in pathogenic mechanisms but differences in neurodevelopment.’, International clinical psychopharmacology, 17 Suppl 3, pp. S11–S19.Google Scholar
Wallwork, RS, Fortgang, R, Hashimoto, R, Weinberger, DR and Dickinson, D (2012) ‘Searching for a consensus five-factor model of the Positive and Negative Syndrome Scale for schizophrenia.’, Schizophrenia research. NIH Public Access, 137(1–3), pp. 246–50.Google Scholar
Supplementary material: File

Joyce supplementary material

Joyce supplementary material

Download Joyce supplementary material(File)
File 751.6 KB