Skip to main content Accessibility help

Exploring Heterogeneity on the Wisconsin Card Sorting Test in Schizophrenia Spectrum Disorders: A Cluster Analytical Investigation

  • Sean P. Carruthers (a1), Caroline T. Gurvich (a2), Denny Meyer (a1) (a3), Australian Schizophrenia Research Bank (a4), Chad Bousman (a5) (a6) (a7), Ian P. Everall (a5) (a7) (a8) (a9) (a10), Erica Neill (a1) (a7) (a8), Christos Pantelis (a5) (a7) (a8) (a10) (a11), Philip J. Sumner (a1), Eric J. Tan (a1) (a12), Elizabeth H.X. Thomas (a2), Tamsyn E. Van Rheenen (a1) (a11) and Susan L. Rossell (a1) (a12)...


Objectives: The Wisconsin Card Sorting Test (WCST) is a complex measure of executive function that is frequently employed to investigate the schizophrenia spectrum. The successful completion of the task requires the interaction of multiple intact executive processes, including attention, inhibition, cognitive flexibility, and concept formation. Considerable cognitive heterogeneity exists among the schizophrenia spectrum population, with substantive evidence to support the existence of distinct cognitive phenotypes. The within-group performance heterogeneity of individuals with schizophrenia spectrum disorder (SSD) on the WCST has yet to be investigated. A data-driven cluster analysis was performed to characterise WCST performance heterogeneity. Methods: Hierarchical cluster analysis with k-means optimisation was employed to identify homogenous subgroups in a sample of 210 schizophrenia spectrum participants. Emergent clusters were then compared to each other and a group of 194 healthy controls (HC) on WCST performance and demographic/clinical variables. Results: Three clusters emerged and were validated via altered design iterations. Clusters were deemed to reflect a relatively intact patient subgroup, a moderately impaired patient subgroup, and a severely impaired patient subgroup. Conclusions: Considerable within-group heterogeneity exists on the WCST. Identification of subgroups of patients who exhibit homogenous performance on measures of executive functioning may assist in optimising cognitive interventions. Previous associations found using the WCST among schizophrenia spectrum participants should be reappraised. (JINS, 2019, 25, 750–760)


Corresponding author

Correspondence and reprint requests to: Sean P. Carruthers, Centre for Mental Health, Level 10, ATC Building, Swinburne University of Technology, Hawthorn VIC 3122, Australia. E-mail:


Hide All
Allott, K.A., Killackey, E., Sun, P., Brewer, W.J., & Velligan, D.I. (2017). Improving vocational outcomes in first-episode psychosis by addressing cognitive impairments using cognitive adaptation training. Work, 56(4), 581589. doi: 10.3233/WOR-172517
Bellack, A.S., Blanchard, J.J., Murphy, P., & Podell, K. (1996). Generalization effects of training on the Wisconsin Card Sorting Test for schizophrenia patients. Schizophrenia Research, 19(2–3), 189194. doi: 10.1016/0920-9964(95)00067-4
Blair, J.R. & Spreen, O. (1989). Predicting premorbid IQ: A revision of the National Adult Reading Test. The Clinical Neuropsychologist, 3(2), 129136.
Bora, E. (2016). Differences in cognitive impairment between schizophrenia and bipolar disorder: Considering the role of heterogeneity. Psychiatry and Clinical Neurosciences, 70(10), 424433. doi: 10.1111/pcn.12410
Bowie, C.R., McGurk, S.R., Mausbach, B., Patterson, T.L., & Harvey, P.D. (2012). Combined cognitive remediation and functional skills training for schizophrenia: Effects on cognition, functional competence, and real-world behavior. American Journal of Psychiatry, 169(7), 710718. doi: 10.1176/appi.ajp.2012.11091337
Bryce, S.D., Rossell, S.L., Lee, S.J., Lawrence, R.J., Tan, E.J., Carruthers, S.P., & Ponsford, J.L. (2018). Neurocognitive and self-efficacy benefits of cognitive remediation in schizophrenia: A randomized controlled trial. Journal of the International Neuropsychological Society, 23, 114.
Bryson, G., Greig, T., Lysaker, P., & Bell, M. (2002). Longitudinal Wisconsin Card Sorting performance in schizophrenia patients in rehabilitation. Applied Neuropsychology, 9(4), 203209. doi: 10.1207/S15324826AN0904_2
Burdick, K.E., Russo, M., Frangou, S., Mahon, K., Braga, R.J., Shanahan, M., & Malhotra, A.K. (2014). Empirical evidence for discrete neurocognitive subgroups in bipolar disorder: Clinical implications. Psychological Medicine, 44(14), 30833096. doi: 10.1017/S0033291714000439
Cella, M., Bishara, A.J., Medin, E., Swan, S., Reeder, C., & Wykes, T. (2014). Identifying cognitive remediation change through computational modelling – Effects on reinforcement learning in schizophrenia. Schizophrenia Bulletin, 40(6), 14221432. doi: 10.1093/schbul/sbt152
Choi, J. & Kurtz, M.M. (2009). A comparison of remediation techniques on the Wisconsin Card Sorting Test in schizophrenia. Schizophrenia Research, 107(1), 7682. doi: 10.1016/j.schres.2008.09.017
Cobia, D.J., Csernansky, J.G., & Wang, L. (2011). Cortical thickness in neuropsychologically near-normal schizophrenia. Schizophrenia Research, 133(1–3), 6876. doi: 10.1016/j.schres.2011.08.017
Croca, M., Lagodka, A., Gadel, R., Bourdel, M.C., Bendjemaa, N., Gaillard, R., Olie, J.P., Champagne-Lavau, M., Krebs, M.O., & Amado, I. (2018). Theory of mind and schizophrenia in young and middle-aged patients: Influence of executive functions. Psychiatry Research, 259, 532537. doi: 10.1016/j.psychres.2017.10.041
Czepielewski, L.S., Wang, L., Gama, C.S., & Barch, D.M. (2017). The relationship of intellectual functioning and cognitive performance to brain structure in schizophrenia. Schizophrenia Bulletin, 43(2), 355364. doi: 10.1093/schbul/sbw090
Dawes, S.E., Jeste, D.V., & Palmer, B.W. (2011). Cognitive profiles in persons with chronic schizophrenia. Journal of Clinical and Experimental Neuropsychology, 33(8), 929936. doi: 10.1080/13803395.2011.578569
Donohoe, G. & Robertson, I.H. (2003). Can specific deficits in executive functioning explain the negative symptoms of schizophrenia? A review. Neurocase, 9(2), 97108. doi: 10.1076/neur.
Farreny, A., Aguado, J., Ochoa, S., Huerta-Ramos, E., Marsà, F., López-Carrilero, R., Carral, V., Haro, J.M., & Usall, J. (2012). REPYFLEC cognitive remediation group training in schizophrenia: Looking for an integrative approach. Schizophrenia Research, 142(1), 137144. doi: 10.1016/j.schres.2012.08.035
Fredrick, M.M., Mintz, J., Roberts, D.L., Maples, N.J., Sarkar, S., Li, X., & Velligan, D.I. (2015). Is cognitive adaptation training (CAT) compensatory, restorative, or both? Schizophrenia Research, 166(1–3), 290296. doi: 10.1016/j.schres.2015.06.003
Geisler, D., Walton, E., Naylor, M., Roessner, V., Lim, K.O., Schulz, S.C., Gollub, R.L., Calhoun, V.D., Sponheim, S.R., & Ehrlich, S. (2015). Brain structure and function correlates of cognitive subtypes in schizophrenia. Psychiatry Research-Neuroimaging, 234(1), 7483. doi: 10.1016/j.pscychresns.2015.08.008
Gilbert, E., Mérette, C., Jomphe, V., Émond, C., Rouleau, N., Bouchard, R.H., Roy, W.A., Paccalet, T., & Maziade, M. (2014). Cluster analysis of cognitive deficits may mark heterogeneity in schizophrenia in terms of outcome and response to treatment. European Archives of Psychiatry and Clinical Neuroscience, 264(4), 333343. doi: 10.1007/s00406-013-0463-7
Green, M.F., Kern, R.S., Braff, D.L., & Mintz, J. (2000). Neurocognitive deficits and functional outcome in schizophrenia: Are we measuring the ‘right stuff’? Schizophrenia Bulletin, 26(1), 119136.
Green, M.F., Kern, R.S., & Heaton, R.K. (2004). Longitudinal studies of cognition and functional outcome in schizophrenia: Implications for MATRICS. Schizophrenia Research, 72(1), 4151.
Heaton, R.K. (1993). Wisconsin Card Sorting Test Manual, Revised and Expanded. Psychological Assessment Services, Inc.
Hergueta, T., Baker, R., & Dunbar, G.C. (1998). The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IVand ICD-10. J clin Psychiatry, 59(Suppl 20), 2233.
Hill, S.K., Ragland, J.D., Gur, R.C., & Gur, R.E. (2002). Neuropsychological profiles delineate distinct profiles of schizophrenia, an interaction between memory and executive function, and uneven distribution of clinical subtypes. Journal of Clinical and Experimental Neuropsychology, 24(6), 765780. doi: 10.1076/jcen.24.6.765.8402
Hoti, F., Tuulio-Henriksson, A., Haukka, J., Partonen, T., Holmström, L., & Lönnqvist, J. (2004). Family-based clusters of cognitive test performance in familial schizophrenia. BMC Psychiatry, 4. doi: 10.1186/1471-244X-4-20
Joyce, E.M., & Roiser, J.P. (2007). Cognitive heterogeneity in schizophrenia. Current Opinion in Psychiatry, 20(3), 268272. doi: 10.1097/YCO.0b013e3280ba4975
Jurado, M.B., & Rosselli, M. (2007). The elusive nature of executive functions: A review of our current understanding. Neuropsychology Review, 17(3), 213233. doi: 10.1007/s11065-007-9040-z
Kay, S.R., Fiszbein, A., & Opler, L.A. (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13(2), 261276.
Kim, H.S., An, Y.M., Kwon, J.S., & Shin, M.S. (2014). A preliminary validity study of the Cambridge neuropsychological test automated battery for the assessment of executive function in schizophrenia and bipolar disorder. Psychiatry Investigation, 11(4), 394401. doi: 10.4306/pi.2014.11.4.394
Kurtz, M.M., Moberg, P.J., Gur, R.C., & Gur, R.E. (2001). Approaches to cognitive remediation of neuropsychological deficits in schizophrenia: A review and meta-analysis. Neuropsychology Review, 11(4), 197210. doi: 10.1023/A:1012953108158
Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159174.
Leucht, S., Kane, J.M., Etschel, E., Kissling, W., Hamann, J., & Engel, R.R. (2006). Linking the PANSS, BPRS, and CGI: Clinical implications. Neuropsychopharmacology, 31(10), 23182325. doi: 10.1038/sj.npp.1301147
Leucht, S., Rothe, P., Davis, J.M., & Engel, R.R. (2013). Equipercentile linking of the BPRS and the PANSS. European Neuropsychopharmacology, 23(8), 956959. doi: 10.1016/j.euroneuro.2012.11.004
Lewandowski, K.E., Baker, J.T., McCarthy, J.M., Norris, L.A., & Öngür, D. (2018). Reproducibility of cognitive profiles in psychosis using cluster analysis. Journal of the International Neuropsychological Society, 24(4), 382390. doi: 10.1017/S1355617717001047
Lewandowski, K.E., Sperry, S.H., Cohen, B.M., & Öngür, D. (2014). Cognitive variability in psychotic disorders: A cross-diagnostic cluster analysis. Psychological Medicine, 44(15), 32393248. doi: 10.1017/S0033291714000774
Li, C.S.R. (2004). Do schizophrenia patients make more perseverative than non-perseverative errors on the Wisconsin Card Sorting Test? A meta-analytic study. Psychiatry Research, 129(2), 179190. doi: 10.1016/j.psychres.2004.06.016
Light, G.A., Swerdlow, N.R., Rissling, A.J., Radant, A., Sugar, C.A., Sprock, J., Pela, M., Geyer, M.A., & Braff, D.L. (2012). Characterization of neurophysiologic and neurocognitive biomarkers for use in genomic and clinical outcome studies of schizophrenia. PLoS ONE, 7(7), e39434.
Liu, C.M., Fann, C.S., Chen, C.Y., Liu, Y.L., Oyang, Y.J., Yang, W.C., Chang, C.C., Wen, C.C., Chen, W.J., Hwang, T.J., Hsieh, M.H., Liu, C.C., Faraone, S.V., Tsuang, M.T., & Hwu, H.G. (2011). ANXA7, PPP3CB, DNAJC9, and ZMYND17 genes at chromosome 10q22 associated with the subgroup of schizophrenia with deficits in attention and executive function. Biological Psychiatry, 70(1), 5158. doi: 10.1016/j.biopsych.2011.02.033
Lysaker, P., Bell, M., & Beam-Goulet, J. (1995). Wisconsin card sorting test and work performance in schizophrenia. Psychiatry Research, 56(1), 4551. doi: 10.1016/0165-1781(94)02641-U
McDermid Vaz, S. & Heinrichs, R.W. (2006). Stability and validity of memory-based subtypes of schizophrenia. Journal of the International Neuropsychological Society, 12(6), 782791. doi: 10.1017/S1355617706060966
Miyake, A., & Friedman, N.P. (2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 814. doi: 10.1177/0963721411429458
Miyake, A., Friedman, N.P., Emerson, M.J., Witzki, A.H., Howerter, A., & Wager, T.D. (2000). The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49100. doi: 10.1006/cogp.1999.0734
Nieuwenstein, M.R., Aleman, A., & De Haan, E.H.F. (2001). Relationship between symptom dimensions and neurocognitive functioning in schizophrenia: A meta-analysis of WCST and CPT studies. Journal of Psychiatric Research, 35(2), 119125. doi: 10.1016/S0022-3956(01)00014-0
Overall, J.E., & Gorham, D.R. (1962). The brief psychiatric rating scale. Psychological Reports, 10(3), 799812.
Penadés, R., Catalán, R., Puig, O., Masana, G., Pujol, N., Navarro, V., Guarch, J., & Gastó, C. (2010). Executive function needs to be targeted to improve social functioning with Cognitive Remediation Therapy (CRT) in schizophrenia. Psychiatry Research, 177(1–2), 4145. doi: 10.1016/j.psychres.2009.01.032
Polgár, P., Réthelyi, J.M., Bálint, S., Komlósi, S., Czobor, P., & Bitter, I. (2010). Executive function in deficit schizophrenia: What do the dimensions of the Wisconsin Card Sorting Test tell us? Schizophrenia Research, 122(1–3), 8593. doi: 10.1016/j.schres.2010.06.007
Potter, A.I., & Nestor, P.G. (2010). IQ subtypes in schizophrenia: Distinct symptom and neuropsychological profiles. Journal of Nervous and Mental Disease, 198(8), 580585. doi: 10.1097/NMD.0b013e3181ea4e43
Rady, A., Elsheshai, A., el Wafa, H.A., & Elkholy, O. (2011). Wisconsin Card Sort Test (WCST) performance in schizophrenia and severe depression with psychotic features. German Journal of Psychiatry, 14(2), 9194.
Rempfer, M., Hamera, E., Brown, C., & Bothwell, R.J. (2006). Learning proficiency on the Wisconsin Card Sorting Test in people with serious mental illness: What are the cognitive characteristics of good learners? Schizophrenia Research, 87(1–3), 316322. doi: 10.1016/j.schres.2006.05.012
Rempfer, M.V., McDowd, J.M., & Brown, C.E. (2017). Measuring learning potential in people with schizophrenia: A comparison of two tasks. Psychiatry Research, 258, 316321. doi: 10.1016/j.psychres.2017.08.057
Reser, M.P., Allott, K.A., Killackey, E., Farhall, J., & Cotton, S.M. (2015). Exploring cognitive heterogeneity in first-episode psychosis: What cluster analysis can reveal. Psychiatry Research, 229(3), 819827. doi: 10.1016/j.psychres.2015.07.084
Rossell, S.L., Coakes, J., Shapleske, J., Woodruff, P.W.R., & David, A.S. (2003). Insight: Its relationship with cognitive function, brain volume and symptoms in schizophrenia. Psychological Medicine, 33(1), 111119. doi: 10.1017/S0033291702006803
Rossell, S.L., & David, A.S. (1997). Improving performance on the WCST: Variations on the original procedure. Schizophrenia Research, 28(1), 6376. doi: 10.1016/S0920-9964(97)00093-5
Rybakowski, J.K., Borkowska, A., Czerski, P.M., Dmitrzak-Weglarz, M., Skibinska, M., Kapelski, P., & Hauser, J. (2006). Performance on the Wisconsin Card Sorting Test in schizophrenia and genes of dopaminergic inactivation (COMT, DAT, NET). Psychiatry Research, 143(1), 1319. doi: 10.1016/j.psychres.2005.10.008
Sasabayashi, D., Takayanagi, Y., Nishiyama, S., Takahashi, T., Furuichi, A., Kido, M., Nishikawa, Y., Nakamura, M., Noguchi, K., & Suzuki, M. (2017). Increased frontal gyrification negatively correlates with executive function in patients with first-episode schizophrenia. Cerebral Cortex, 27(4), 26862694. doi: 10.1093/cercor/bhw101
Sauvé, G., Malla, A., Joober, R., Brodeur, M.B., & Lepage, M. (2018). Comparing cognitive clusters across first- and multiple-episode of psychosis. Psychiatry Research, 269, 707718. doi: 10.1016/j.psychres.2018.08.119
Scarr, E., Sundram, S., Deljo, A., Cowie, T.F., Gibbons, A.S., Juzva, S., Mackinnon, A., Wood, S.J., Testa, R., Pantelis, C., & Dean, B. (2012). Muscarinic M1 receptor sequence: Preliminary studies on its effects on cognition and expression. Schizophrenia Research, 138(1), 9498. doi: 10.1016/j.schres.2012.02.011
Seaton, B.E., Goldstein, G., & Allen, D.N. (2001). Sources of heterogeneity in schizophrenia: The role of neuropsychological functioning. Neuropsychology Review, 11(1), 4567. doi: 10.1023/A:1009013718684
Shafer, A. (2005). Meta-analysis of the brief psychiatric rating scale factor structure. Psychological Assessment, 17(3), 324335. doi: 10.1037/1040-3590.17.3.324
Uren, J., Cotton, S.M., Killackey, E., Saling, M.M., & Allott, K. (2017). Cognitive clusters in first-episode psychosis: Overlap with healthy controls and relationship to concurrent and prospective symptoms and functioning. Neuropsychology, 31(7), 787797. doi: 10.1037/neu0000367
Van der Does, A.J.W., & Van den Bosch, R.J. (1992). What determines Wisconsin Card Sorting performance in schizophrenia? Clinical Psychology Review, 12(6), 567583. doi: 10.1016/0272-7358(92)90132-R
Van Rheenen, T.E., Cropley, V., Zalesky, A., Bousman, C., Wells, R., Bruggemann, J., Sundram, S., Weinberg, D., Lenroot, R.K., Pereira, A., Weickert, C.S., Weickert, T.W., & Pantelis, C. (2018). Widespread volumetric reductions in Schizophrenia and Schizoaffective patients displaying compromised cognitive abilities. Schizophrenia Bulletin, 44(3), 560574. doi: 10.1093/schbul/sbx109
Van Rheenen, T.E., Lewandowski, K.E., Tan, E.J., Ospina, L.H., Ongur, D., Neill, E., Gurvich, C., Pantelis, C., Malhotra, A.K., Rossell, S.L., & Burdick, K. E. (2017). Characterizing cognitive heterogeneity on the schizophrenia-bipolar disorder spectrum. Psychological Medicine, 47(10), 18481864. doi: 10.1017/S0033291717000307
Velligan, D.I., Bow-Thomas, C.C., Mahurin, R.K., Miller, A.L., & Halgunseth, L.C. (2000). Do specific neurocognitive deficits predict specific domains of community function in schizophrenia? Journal of Nervous and Mental Disease, 188(8), 518524. doi: 10.1097/00005053-200008000-00007
Waford, R.N., & Lewine, R. (2010). Is perseveration uniquely characteristic of schizophrenia? Schizophrenia Research, 118(1–3), 128133. doi: 10.1016/j.schres.2010.01.031
Wechsler, D. (2001). Wechsler Test of Adult Reading: WTAR. San Antonio TX: Psychological Corporation.
Weickert, T.W., Goldberg, T.E., Gold, J.M., Bigelow, L.B., Egan, M.F., & Weinberger, D.R. (2000). Cognitive impairments in patients with schizophrenia displaying preserved and compromised intellect. Archives of General Psychiatry, 57(9), 907913. doi: 10.1001/archpsyc.57.9.907
Weinberg, D., Lenroot, R., Jacomb, I., Allen, K., Bruggemann, J., Wells, R., Balzan, R., Liu, D., Galletly, C., Catts, S.V., Weickert, C.S., & Weickert, T.W. (2016). Cognitive subtypes of schizophrenia characterized by differential brain volumetric reductions and cognitive decline. JAMA Psychiatry, 73(12), 12511259. doi: 10.1001/jamapsychiatry.2016.2925
Wells, R., Swaminathan, V., Sundram, S., Weinberg, D., Bruggemann, J., Jacomb, I., Cropley, V., Lenroot, R., Pereira, A.M., & Zalesky, A. (2015). The impact of premorbid and current intellect in schizophrenia: Cognitive, symptom, and functional outcomes. npj Schizophrenia, 1, 15043.
Wilmsmeier, A., Ohrmann, P., Suslow, T., Siegmund, A., Koelkebeck, K., Rothermundt, M., Kugel, H., Arolt, V., Vauer, J., & Pedersen, A. (2010). Neural correlates of set-shifting: Decomposing executive functions in schizophrenia. Journal of Psychiatry and Neuroscience, 35(5), 321329. doi: 10.1503/jpn.090181
Woodward, N.D., & Heckers, S. (2015). Brain structure in neuropsychologically defined subgroups of schizophrenia and psychotic bipolar disorder. Schizophrenia Bulletin, 41(6), 13491359. doi: 10.1093/schbul/sbv048
Wykes, T., Reeder, C., Landau, S., Everitt, B., Knapp, M., Patel, A., & Romeo, R. (2007). Cognitive remediation therapy in schizophrenia: Randomised controlled trial. British Journal of Psychiatry, 190(May), 421427. doi: 10.1192/bjp.bp.106.026575


Type Description Title
Supplementary materials

Carruthers et al. supplementary material
Tables S1-S4 and Figures S1-S2

 Word (6.9 MB)
6.9 MB

Exploring Heterogeneity on the Wisconsin Card Sorting Test in Schizophrenia Spectrum Disorders: A Cluster Analytical Investigation

  • Sean P. Carruthers (a1), Caroline T. Gurvich (a2), Denny Meyer (a1) (a3), Australian Schizophrenia Research Bank (a4), Chad Bousman (a5) (a6) (a7), Ian P. Everall (a5) (a7) (a8) (a9) (a10), Erica Neill (a1) (a7) (a8), Christos Pantelis (a5) (a7) (a8) (a10) (a11), Philip J. Sumner (a1), Eric J. Tan (a1) (a12), Elizabeth H.X. Thomas (a2), Tamsyn E. Van Rheenen (a1) (a11) and Susan L. Rossell (a1) (a12)...


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed