Hostname: page-component-848d4c4894-r5zm4 Total loading time: 0 Render date: 2024-06-16T09:36:28.445Z Has data issue: false hasContentIssue false

Aberrant processing of deviant stimuli in schizophrenia revealed by fusion of fMRI and EEG data

Published online by Cambridge University Press:  24 June 2014

Vince D. Calhoun*
The Mind Research Network, Albuquerque, NM 87131, USA Department of ECE, University of New Mexico, Albuquerque, NM 87131, USA Olin Neuropsychiatry Research Center, Institute of Living, Hartford, CT 06106, USA Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA
Lei Wu
The Mind Research Network, Albuquerque, NM 87131, USA Department of ECE, University of New Mexico, Albuquerque, NM 87131, USA
Kent A. Kiehl
The Mind Research Network, Albuquerque, NM 87131, USA Olin Neuropsychiatry Research Center, Institute of Living, Hartford, CT 06106, USA Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA
Tom Eichele
Department of Biological and Medical Psychology, University of Bergen, 5009 Bergen, Norway
Godfrey D. Pearlson
Olin Neuropsychiatry Research Center, Institute of Living, Hartford, CT 06106, USA Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA
Vince D. Calhoun, The Mind Research Network, 1101 Yale Blvd NE, Albuquerque, NM 87106, USA. Tel: +1 860 833 5511; Fax: +1 505 272 8002; E-mail:


Calhoun VD, Wu L, Kiehl KA, Eichele T, Pearlson GD. Aberrant processing of deviant stimuli in schizophrenia revealed by fusion of fMRI and EEG data.


Aberrant electrophysiological and haemodynamic processing of auditory oddball stimuli is among the most robustly documented findings in patients with schizophrenia. However, no study to date has directly examined linked patterns of electrical and haemodynamic differences in patients and controls.


In a recent paper we demonstrated a data-driven approach, joint independent component analysis (jICA) to fuse together functional magnetic resonance imaging (fMRI) and event-related potential (ERP) data and elucidated the chronometry of auditory oddball target detection in healthy control subjects. In this paper we extend our fusion method to identify specific differences in the neuronal chronometry of target detection for chronic schizophrenia patients compared to healthy controls.


We found one linked source, consistent with the N2 response, known to be related to cognitive processing of deviant stimuli, spatially localized to bilateral fronto-temporal regions. This source showed significant between-group differences both in amplitude response and in the fMRI/ERP distribution pattern. These findings are consistent with previous work showing N2 amplitude and latency abnormalities in schizophrenia, and provide new information about the linkage between the two.


In summary, we use a novel approach to isolate and identify a linked fMRI/ERP component which shows marked differences in chronic schizophrenia patients. We also show that jointly using both fMRI and ERP measures provides a fully picture of the underlying haemodynamic and electrical changes which are present in patients. Our approach also has broad applicability to other diseases such as autism, Alzheimer's disease, or bipolar disorder.

Research Article
Copyright © 2010 John Wiley & Sons A/S

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Calhoun, VD, Maciejewski, PK, Pearlson, GD, Kiehl, KA.Temporal lobe and “Default” hemodynamic brain modes discriminate between schizophrenia and bipolar disorder. Hum Brain Mapp 2008;29:12651275. CrossRefGoogle ScholarPubMed
Calhoun, VD, Adali, T, Giuliani, N, Pekar, JJ, Pearlson, GD, Kiehl, KA.A method for multimodal analysis of independent source differences in schizophrenia: combining gray matter structural and auditory oddball functional data. Hum Brain Mapp 2006;27:4762. CrossRefGoogle ScholarPubMed
Kiehl, KA, Stevens, MC, Celone, K, Kurtz, M, Krystal, JH.Abnormal hemodynamics in schizophrenia during an auditory oddball task. Biol Psychiatry 2005;57: 10291040. CrossRefGoogle ScholarPubMed
Kiehl, KA, Liddle, PF.An event-related functional magnetic resonance imaging study of an auditory oddball task in schizophrenia. Schizophr Res 2001;48:159171. CrossRefGoogle ScholarPubMed
Sutton, S, Tueting, P, Zubin, J, John, ER.Information delivery and the sensory evoked potential. Science 1967;155:14361439. CrossRefGoogle ScholarPubMed
Polich, J, Kok, A.Cognitive and biological determinants of P300: an integrative review. Biol Psychol 1995;41:103146. CrossRefGoogle ScholarPubMed
Naatanen, R, Alho, K.Mismatch negativity—a unique measure of sensory processing in audition. Int J Neurosci 1995;80:317337. CrossRefGoogle ScholarPubMed
Gehring, WJ, Gratton, G, Coles, MG, Donchin, E.Probability effects on stimulus evaluation and response processes. J Exp Psychol Hum Percept Perform 1992;18: 198216. CrossRefGoogle ScholarPubMed
Folstein, JR, Van Petten, C.Influence of cognitive control and mismatch on the N2 component of the ERP: a review. Psychophysiology 2008;45:152170. CrossRefGoogle ScholarPubMed
Knight, RT, Nakada, T.Cortico-limbic circuits and novelty: a review of EEG and blood flow data. Rev Neurosci 1998;9:5770. CrossRefGoogle ScholarPubMed
Donchin, E, Coles, MGH.Is the P300 component a manifestation of context updating? Behav Brain Sci 1988;11:357374. CrossRefGoogle Scholar
Michie, PT.What has MMN revealed about the auditory system in schizophrenia? International Journal of Psychophysiology 2001;42:177194. CrossRefGoogle ScholarPubMed
Näätänen, R, Kähkönen, S.Central auditory dysfunction in schizophrenia as revealed by the mismatch negativity (MMN) and its magnetic equivalent MMNm: a review. Int J Neuropsychopharmacol 2009;12:125135. CrossRefGoogle ScholarPubMed
Ebmeier, KP, Potter, DD, Cochrane, RHet al. P300 and smooth eye pursuit: concordance of abnormalities and relation to clinical features in DSM-III schizophrenia. Acta Psychiatr Scand 1990;82:283288. CrossRefGoogle ScholarPubMed
McCarley, RW, Shenton, ME, O’Donnell, BFet al. Auditory P300 abnormalities and left posterior superior temporal gyrus volume reduction in schizophrenia. Arch Gen Psychiatry 1993;50:190197. CrossRefGoogle ScholarPubMed
Ford, JM, White, PM, Csernansky, JG, Faustman, WO, Roth, WT, Pfefferbaum, A.ERPs in schizophrenia: effects of antipsychotic medication. Biol Psychol 1994;36:153170. CrossRefGoogle ScholarPubMed
Hirayasu, Y, Shenton, ME, Salisbury, DF, McCarley, RW.Lower left temporal lobe MRI volumes in patients with first-episode schizophrenia compared with psychotic patients with first-episode affective disorder and normal subjects. Am J Psychiatry 1998;155:13841391. CrossRefGoogle ScholarPubMed
Blackwood, DH, Whalley, LJ, Christie, JE, Blackburn, IM, St Clair, DM, McInnes, A.Changes in auditory P3 event-related potential in schizophrenia and depression. The British Journal of Psychiatry 1987;150:154160. Google ScholarPubMed
Polich, J, Ladish, C, Bloom, FE.P300 assessment of early Alzheimer's disease. Electroencephalogr Clin Neurophysiol 1990;77:179189. CrossRefGoogle ScholarPubMed
St Clair, DM, Blackwood, DH, Christie, JE.P3 and other long latency auditory evoked potentials in presenile dementia Alzheimer type and alcoholic Korsakoff syndrome. The British Journal of Psychiatry 1985;147:702706. Google ScholarPubMed
O’Donnell, BF, Shenton, ME, McCarley, RWet al. The auditory N2 component in schizophrenia: relationship to MRI temporal lobe gray matter and to other ERP abnormalities. Biol Psychiatry 1993;34:2640. CrossRefGoogle ScholarPubMed
Laurent, A, Garcia-Larréa, L, D’Amato, Tet al. Auditory event-related potentials and clinical scores in unmedicated schizophrenic patients. Psychiatry Res 1999;86:229238. CrossRefGoogle ScholarPubMed
Demiralp, T, Uçok, A, Devrim, M, Isoglu-Alkaç, U, Tecer, A, Polich, J.N2 and P3 components of event-related potential in first-episode schizophrenic patients: scalp topography, medication, and latency effects. Psychiatry Res 2002;111:167179. CrossRefGoogle ScholarPubMed
Frodl, T, Meisenzahl, EM, Gallinat, J, Hegerl, U, Möller, HJ.Markers from event-related potential subcomponents and reaction time for information processing dysfunction in schizophrenia. Eur Arch Psychiatry Clin Neurosci 1998;248:307313. CrossRefGoogle ScholarPubMed
Bruder, GE, Kayser, J, Tenke, CE, Friedman, M, Malaspina, D, Gorman, JM.Event-related potentials in schizophrenia during tonal and phonetic oddball tasks: relations to diagnostic subtype, symptom features and verbal memory. Biol Psychiatry 2001;50:447452. CrossRefGoogle ScholarPubMed
Kayser, J, Bruder, GE, Tenke, CE, Stuart, BK, Amador, XF, Gorman, JM.Event-related brain potentials (ERPs) in schizophrenia for tonal and phonetic oddball tasks. Biol Psychiatry 2001;49:832847. CrossRefGoogle ScholarPubMed
Näätänen, R, Paavilainen, P, Rinne, T, Alho, K.The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clinical Neurophysiology 2007;118:25442590. Google ScholarPubMed
Jessen, F, Fries, T, Kucharski, Cet al. Amplitude reduction of the mismatch negativity in first-degree relatives of patients with schizophrenia. Neurosci Lett 2001;309:185188. CrossRefGoogle ScholarPubMed
Salisbury, DF, Shenton, ME, Griggs, CB, Bonner-Jackson, A, McCarley, RW.Mismatch negativity in chronic schizophrenia and first-episode schizophrenia. Arch Gen Psychiatry 2002;59:686694. CrossRefGoogle ScholarPubMed
Kasai, K, Okazawa, K, Nakagome, Ket al. Mismatch negativity and N2b attenuation as an indicator for dysfunction of the preattentive and controlled processing for deviance detection in schizophrenia: a topographic event-related potential study, vol. 35. Elsevier, 1999;141156. Google ScholarPubMed
Kiehl, KA, Laurens, KR, Duty, TL, Forster, BB, Liddle, PF.Neural sources involved in auditory target detection and novelty processing: an event-related fMRI study. Psychophysiology 2001;38:133142. Google ScholarPubMed
McCarthy, G, Luby, M, Gore, J, Goldman-Rakic, P.Infrequent events transiently activate human prefrontal and parietal cortex as measured by functional MRI. J Neurophysiol 1997;77:16301634. CrossRefGoogle ScholarPubMed
Clark, VP, Fannon, S, Lai, S, Benson, R.Paradigm-dependent modulation of event-related fMRI activity evoked by the oddball task. Hum Brain Mapp 2001;14:116127. CrossRefGoogle ScholarPubMed
Kiehl, KA, Stevens, M, Laurens, KR, Pearlson, GD, Calhoun, VD, Liddle, PF.An adaptive reflexive processing model of neurocognitive function: supporting evidence from a large scale (n = 100) fMRI study of an auditory oddball task. NeuroImage 2005;25:899915. CrossRefGoogle ScholarPubMed
Calhoun, VD, Adali, T, Kraut, M, Pearlson, GD.A weighted-least squares algorithm for estimation and visualization of relative latencies in event-related functional MRI. Magn Reson Med 2000;44:947954. 3.0.CO;2-5>CrossRefGoogle ScholarPubMed
Saad, ZS, Ropella, KM, Cox, RW, DeYoe, EA.Analysis and use of FMRI response delays. Hum Brain Mapp 2001;13:7493. CrossRefGoogle ScholarPubMed
Moosmann, M, Eichele, T, Nordby, H, Hugdahl, K, Calhoun, VD.Joint independent component analysis for simultaneous EEG-fMRI: principle and simulation. Int J Psychiatry 2008;67:212221. Google ScholarPubMed
Calhoun, VD, Pearlson, GD, Kiehl, KA.Neuronal chronometry of target detection: fusion of hemodynamic and event-related potential data. NeuroImage 2006;30:544553. CrossRefGoogle ScholarPubMed
Eichele, T, Calhoun, VD, Moosmann, Met al. Unmixing concurrent EEG-fMRI with parallel independent component analysis. Int J Psychiatry 2008;67:222234. Google ScholarPubMed
Wu, L, Calhoun, VD.An approach for fusion between EEG and FMRI data. Toronto, Canada: ISMRM, 2008. Google Scholar
McKeown, MJ, Makeig, S, Brown, GGet al. Analysis of fMRI data by blind separation into independent spatial components. Hum Brain Mapp 1998;6:160188. 3.0.CO;2-1>CrossRefGoogle ScholarPubMed
Calhoun, VD, Adali, T, Pearlson, GD, Pekar, JJ.Spatial and temporal independent component analysis of functional MRI data containing a pair of task-related waveforms. Hum Brain Mapp 2001;13:4353. CrossRefGoogle ScholarPubMed
Kayser, J, Tenke, CE.Optimizing PCA methodology for ERP component identification and measurement: theoretical rationale and empirical evaluation. Clin Neurophysiol 2003;114:23072325. CrossRefGoogle ScholarPubMed
Debener, S, Makeig, S, Delorme, A, Engel, AK.What is novel in the novelty oddball paradigm? Functional significance of the novelty P3 event-related potential as revealed by independent component analysis. Brain Res Cogn Brain Res 2005;22:309321. CrossRefGoogle ScholarPubMed
Martinez-Montes, E, Valdes-Sosa, PA, Miwakeichi, F, Goldman, RI, Cohen, MS.Concurrent EEG/fMRI analysis by multiway Partial Least Squares. NeuroImage 2004;22:10231034. CrossRefGoogle ScholarPubMed
Calhoun, VD, Adali, T, Kiehl, KA, Astur, R, Pekar, JJ, Pearlson, GD.A method for multitask fMRI data fusion applied to schizophrenia. Hum Brain Mapp 2006;27:598610. CrossRefGoogle ScholarPubMed
Calhoun, VD, Kiehl, KA, Liddle, PF, Pearlson, GD.Aberrant localization of synchronous hemodynamic activity in auditory cortex reliably characterizes schizophrenia. Biol Psychiatry 2004;55:842849. CrossRefGoogle ScholarPubMed
Spitzer, RL, Williams, JB, Gibbon, M.Structured Clinical interview for DSM-IV: Non-patient edition (SCID-NP). New York: Biometrics Research Department, New York State Psychiatric Institute, 1996. Google Scholar
Kiehl, KA, Liddle, PF.Reproducibility of the hemodynamic response to auditory oddball stimuli: a six-week test-retest study, vol 18. New York: Wiley Subscription Services, Inc., A Wiley Company, 2003;4252. Google Scholar
Freire, L, Roche, A, Mangin, JF.What is the best similarity measure for motion correction in fMRI time series? IEEE Trans Med Imaging 2002;21:470484. CrossRefGoogle ScholarPubMed
Friston, K, Ashburner, J, Frith, CD, Poline, JP, Heather, JD, Frackowiak, RS.Spatial registration and normalization of images. Hum Brain Mapp 1995;2:165189. CrossRefGoogle Scholar
Jung, TP, Makeig, S, Humphries, Cet al. Removing electroencephalographic artifacts by blind source separation. Psychophysiology 2000;37:163178. CrossRefGoogle ScholarPubMed
Bell, AJ, Sejnowski, TJ.An information maximisation approach to blind separation and blind deconvolution. Neural Comput 1995;7:11291159. CrossRefGoogle Scholar
Lee, TW, Girolami, M, Sejnowski, TJ.Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Comput 1999;11:417441. CrossRefGoogle ScholarPubMed
Wax, M, Kailath, T.Detection of signals by information theoretic criteria. IEEE Trans Acous Speech, Sig Proc 1985;33:387392. CrossRefGoogle Scholar
Li, Y-O, Adali, T, Calhoun, VD.Estimating the number of independent components for functional magnetic resonance imaging data. Hum Brain Mapp 2007;28:12511266. CrossRefGoogle ScholarPubMed
Hastie, T, Tibshirani, R, Friedman, J, Franklin, J.The elements of statistical learning: data mining, inference and prediction. New York: Springer-Verlag New York, Inc., 2001;214217CrossRefGoogle Scholar
Holmes, AP, Friston, KJ.Generalisability, random effects and population inference. Neuroimage: Abstracts of the Fourth International Conference on Functional Mapping of the Human Brain. Vol 7; 1998:S574.Google Scholar
Woods, RP.Modeling for intergroup comparisons of imaging data. NeuroImage 1996;4:8494. CrossRefGoogle ScholarPubMed
Genovese, CR, Lazar, NA, Nichols, T.Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage 2002;15:870878. CrossRefGoogle ScholarPubMed
Yoshiura, T, Zhong, J, Shibata, DK, Kwok, WE, Shrier, DA, Numaguchi, Y.Functional MRI study of auditory and visual oddball tasks. Neuroreport 1999;10:16831688. CrossRefGoogle ScholarPubMed
Friston, KJ.Schizophrenia and the disconnection hypothesis. Acta Psychiatr Scand 1999;395(Suppl.): 6879. CrossRefGoogle ScholarPubMed
Andreasen, NC, Paradiso, S, O’Leary, DS.“Cognitive dysmetria” as an integrative theory of schizophrenia: a dysfunction in cortical-subcortical-cerebellar circuitry? Schizophr Bull 1998;24:203218. CrossRefGoogle ScholarPubMed
Braver, TS, Barch, DM, Cohen, JD.Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function. Biol Psychiatry 1999;46:312328. CrossRefGoogle ScholarPubMed
McCarley, RW, Faux, SF, Shenton, ME, Nestor, PG, Adams, J.Event-related potentials in schizophrenia: their biological and clinical correlates and a new model of schizophrenic pathophysiology. Schizophr Res 1991;4:209231. CrossRefGoogle Scholar
Weinberger, DR.Implications of normal brain development for the pathogenesis of schizophrenia. Arch Gen Psychiatry 1987;44:660669. CrossRefGoogle ScholarPubMed
Ross, CA, Pearlson, GD.Schizophrenia, the heteromodal association neocortex and development: potential for a neurogenetic approach. Trends Neurosci 1996;19: 171176. CrossRefGoogle ScholarPubMed
Pearlson, GD, Petty, RG, Ross, CA, Tien, AY.Schizophrenia: a disease of heteromodal association cortex? Neuropsychopharmacology 1996;14:117. CrossRefGoogle ScholarPubMed
Liddle, PF, Friston, KJ, Frith, CD, Hirsch, SR, Jones, T, Frackowiak, RS.Patterns of cerebral blood flow in schizophrenia. Br J Psychiatry 1992;160:179186. CrossRefGoogle ScholarPubMed
Logothetis, NK, Pauls, J, Augath, M, Trinath, T, Oeltermann, A.Neurophysiological investigation of the basis of the fMRI signal. Nature 2001;412:150157. CrossRefGoogle ScholarPubMed
Vitacco, D, Brandeis, D, Pascual-Marqui, R, Martin, E.Correspondence of event-related potential tomography and functional magnetic resonance imaging during language processing. Hum Brain Mapp 2002;17:412. CrossRefGoogle ScholarPubMed
Horovitz, SG, Skudlarski, P, Gore, JC.Correlations and dissociations between BOLD signal and P300 amplitude in an auditory oddball task: a parametric approach to combining fMRI and ERP. Magn Reson Imaging 2002;20:319325. CrossRefGoogle Scholar
Mathalon, DH, Whitfield, SL, Ford, JM.Anatomy of an error: ERP and fMRI. Biol Psychol 2003;64:119141. CrossRefGoogle ScholarPubMed
Debener, S, Ullsperger, M, Siegel, M, Fiehler, K, Von Cramon, DY, Engel, AK.Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci 2005;25:1173011737. CrossRefGoogle ScholarPubMed
Eichele, T, Specht, K, Moosmann, M, Jongsma, ML, Quiroga, RQ, Nordby, H.Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc Natl Acad Sci U S A 2005;102:1779817803. CrossRefGoogle ScholarPubMed
Schellart, NAM, Reits, D.Influences of perinatal dioxin load to visual motion and oddball stimuli examined with an EEG and MEG analysis. Clinical Neurophysiology 2008;119:14861495. Google ScholarPubMed
Lim, CL, Gordon, E, Rennie, C, Wright, JJ, Bahramali, H, Li, WM, Clouston, P, Morris, JGL.Dynamics of SCR, EEG, and ERP activity in an oddball paradigm with short interstimulus intervals. Psychophysiology 1999;36:543551. CrossRefGoogle Scholar
D’Arcy, RCN, Connolly, JF, Crocker, SF.Latency shifts in the N2b component track phonological deviations in spoken words. Clinical Neurophysiology 2000;111:4044. Google ScholarPubMed
Salisbury, DF, O’Donnell, BF, McCarley, RW, Shenton, ME, Benavage, A.The N2 event-related potential reflects attention deficit in schizophrenia. Biol Psychol 1994;39:113. CrossRefGoogle ScholarPubMed
Kirino, E, Inoue, R.Relationship of mismatch negativity to background EEG and morphological findings in schizophrenia. Neuropsychobiology 1999;40:1420. CrossRefGoogle ScholarPubMed
Ford, JM, Mathalon, DH, Kalba, S, Marsh, L, Pfefferbaum, A.N1 and P300 abnormalities in patients with schizophrenia, epilepsy, and epilepsy with schizophrenialike features. Biol Psychiatry 2001;49:848860. CrossRefGoogle ScholarPubMed
Pearlson, GD, Barta, PE, Powers, REet al. Ziskind-Somerfeld Research Award. Medial and superior temporal gyral volumes and cerebral asymmetry in schizophrenia versus bipolar disorder. Biol Psychiatry 1996;41:114. CrossRefGoogle Scholar
Barta, PE, Pearlson, GD, Brill, LBet al. Planum temporale asymmetry reversal in schizophrenia: replication and relationship to gray matter abnormalities. Am J Psychiatry 1997;154:661667. Google ScholarPubMed
Allen, AJ, Griss, ME, Folley, BS, Hawkins, KA, Pearlson, GDIP.Endophenotypes in schizophrenia: A selective review. Schizophr Res 2009;109:2437. CrossRefGoogle ScholarPubMed
Li, Y-O, Wang, W, Adali, T, Calhoun, VD. CCA for joint blind source separation of multiple datasets with application to group FMRI analysis. Paper presented at International Conference on Acoustics, Speech, Signal Processing (ICASSP). Las Vegas, Nevada, 2008. CrossRefGoogle Scholar
Correa, N, Li, Y-O, Adali, T, Calhoun, VD. Examining associations between fMRI and EEG data using canonical correlation analysis. Paper presented at: IEEE International Symposium on Biomedical Imaging. Paris, France, 2008. CrossRefGoogle Scholar