Hostname: page-component-76fb5796d-skm99 Total loading time: 0 Render date: 2024-04-27T05:23:15.980Z Has data issue: false hasContentIssue false
  • English
  • Français

Linking brain imaging signals to visual perception

Published online by Cambridge University Press:  29 October 2013

ANDREW E. WELCHMAN  and
ZOE KOURTZI
ANDREW E. WELCHMAN*
Affiliation:
School of Psychology, University of Birmingham, Birmingham, UK Laboratory for Neuro- and Psychophysiology, K.U. Leuven, Leuven, Belgium
ZOE KOURTZI*
Affiliation:
School of Psychology, University of Birmingham, Birmingham, UK Laboratory for Neuro- and Psychophysiology, K.U. Leuven, Leuven, Belgium
*
*Address correspondence to: Andrew E. Welchman, University of Birmingham, School of Psychology, Edgbaston, Birmingham B15 2TT, UK. E-mail: a.e.welchman@bham.ac.uk and Zoe Kourtzi. E-mail: z.kourtzi@bham.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

The rapid advances in brain imaging technology over the past 20 years are affording new insights into cortical processing hierarchies in the human brain. These new data provide a complementary front in seeking to understand the links between perceptual and physiological states. Here we review some of the challenges associated with incorporating brain imaging data into such “linking hypotheses,” highlighting some of the considerations needed in brain imaging data acquisition and analysis. We discuss work that has sought to link human brain imaging signals to existing electrophysiological data and opened up new opportunities in studying the neural basis of complex perceptual judgments. We consider a range of approaches when using human functional magnetic resonance imaging to identify brain circuits whose activity changes in a similar manner to perceptual judgments and illustrate these approaches by discussing work that has studied the neural basis of 3D perception and perceptual learning. Finally, we describe approaches that have sought to understand the information content of brain imaging data using machine learning and work that has integrated multimodal data to overcome the limitations associated with individual brain imaging approaches. Together these approaches provide an important route in seeking to understand the links between physiological and psychological states.

Keywords

fMRIAdaptation, pattern classification3D shape perceptionPerceptual learning
Type
Retrospective and prospective analyses of linking propositions
Information
Visual Neuroscience , Volume 30 , Special Issue 5-6: Linking Hypotheses in Visual Neuroscience: A Tribute to Davida Teller , November 2013 , pp. 229 - 241
Copyright
Copyright © Cambridge University Press 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adini, Y., Sagi, D. & Tsodyks, M. (2002). Context-enabled learning in the human visual system. Nature 415, 790–793.CrossRefGoogle ScholarPubMed
Ban, H., Preston, T.J., Meeson, A. & Welchman, A.E. (2012). The integration of motion and disparity cues to depth in dorsal visual cortex. Nature Neuroscience 15, 636–643.CrossRefGoogle ScholarPubMed
Beckett, A., Peirce, J.W., Sanchez-Panchuelo, R.M., Francis, S. & Schluppeck, D. (2012). Contribution of large scale biases in decoding of direction-of-motion from high-resolution fMRI data in human early visual cortex. Neuroimage 63, 1623–1632.CrossRefGoogle ScholarPubMed
Boynton, G.M. (2011). Spikes, BOLD, attention, and awareness: A comparison of electrophysiological and fMRI signals in V1. Journal of Vision 11, 12.CrossRefGoogle ScholarPubMed
Boynton, G.M., Demb, J.B., Glover, G.H. & Heeger, D.J. (1999). Neuronal basis of contrast discrimination. Vision Research 39, 257–269.CrossRefGoogle ScholarPubMed
Boynton, G.M., Engel, S.A., Glover, G.H. & Heeger, D.J. (1996). Linear systems analysis of functional magnetic resonance imaging in human V1. The Journal of Neuroscience 16, 4207–4221.CrossRefGoogle ScholarPubMed
Bradley, D.C., Chang, G.C. & Andersen, R.A. (1998). Encoding of three-dimensional structure-from-motion by primate area MT neurons. Nature 392, 714–717.CrossRefGoogle ScholarPubMed
Bridge, H. & Parker, A.J. (2007). Topographical representation of binocular depth in the human visual cortex using fMRI. Journal of Vision 7, 1–14.CrossRefGoogle ScholarPubMed
Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S. & Movshon, J.A. (1996). A relationship between behavioral choice and the visual responses of neurons in macaque MT. Visual Neuroscience 13, 87–100.CrossRefGoogle ScholarPubMed
Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. (1992). The analysis of visual motion: A comparison of neuronal and psychophysical performance. The Journal of Neuroscience 12, 4745–4765.CrossRefGoogle Scholar
Cardoso, M.M., Sirotin, Y.B., Lima, B., Glushenkova, E. & Das, A. (2012). The neuroimaging signal is a linear sum of neurally distinct stimulus- and task-related components. Nature Neuroscience 15, 1298–1306.CrossRefGoogle ScholarPubMed
Chandrasekaran, C., Canon, V., Dahmen, J.C., Kourtzi, Z. & Welchman, A.E. (2007). Neural correlates of disparity-defined shape discrimination in the human brain. Journal of Neurophysiology 97, 1553–1565.CrossRefGoogle ScholarPubMed
Cogan, A.I., Kontsevich, L.L., Lomakin, A.J., Halpern, D.L. & Blake, R. (1995). Binocular disparity processing with opposite-contrast stimuli. Perception 24, 33–47.CrossRefGoogle ScholarPubMed
Cottereau, B.R., Mckee, S.P., Ales, J.M. & Norcia, A.M. (2011). Disparity-tuned population responses from human visual cortex. The Journal of Neuroscience 31, 954–965.CrossRefGoogle ScholarPubMed
Cottereau, B.R., McKee, S.P., Ales, J.M. & Norcia, A.M. (2012). Disparity-specific spatial interactions: Evidence from EEG source imaging. The Journal of Neuroscience 32, 826–840.CrossRefGoogle ScholarPubMed
Cox, D.D. & Savoy, R.L. (2003). Functional magnetic resonance imaging (fMRI) “brain reading”: Detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270.CrossRefGoogle ScholarPubMed
Cumming, B.G. & DeAngelis, G.C. (2001). The physiology of stereopsis. Annual Review of Neuroscience 24, 203–238.CrossRefGoogle ScholarPubMed
Cumming, B.G. & Parker, A.J. (1997). Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature 389, 280–283.CrossRefGoogle ScholarPubMed
Cumming, B.G., Shapiro, S.E. & Parker, A.J. (1998). Disparity detection in anticorrelated stereograms. Perception 27, 1367–1377.CrossRefGoogle ScholarPubMed
Das, K., Giesbrecht, B. & Eckstein, M.P. (2010). Predicting variations of perceptual performance across individuals from neural activity using pattern classifiers. Neuroimage 51, 1425–1437.CrossRefGoogle ScholarPubMed
DeAngelis, G.C. & Newsome, W.T. (1999). Organization of disparity-selective neurons in macaque area MT. The Journal of Neuroscience 19, 1398–1415.CrossRefGoogle ScholarPubMed
Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D.Y. & Engel, A.K. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. The Journal of Neuroscience 25, 11730–11737.CrossRefGoogle ScholarPubMed
Dosher, B.A. & Lu, Z.L. (1999). Mechanisms of perceptual learning. Vision Research 39, 3197–3221.CrossRefGoogle ScholarPubMed
Eichele, T., Specht, K., Moosmann, M., Jongsma, M.L., Quiroga, R.Q., Nordby, H. & Hugdahl, K. (2005). Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proceedings of the National Academy of Sciences of the United States of America 102, 17798–17803.CrossRefGoogle ScholarPubMed
Fine, I. & Jacobs, R.A. (2002). Comparing perceptual learning tasks: A review. Journal of Vision 2, 190–203.CrossRefGoogle ScholarPubMed
Freeman, J., Brouwer, G.J., Heeger, D.J. & Merriam, E.P. (2011). Orientation decoding depends on maps, not columns. The Journal of Neuroscience 31, 4792–4804.CrossRefGoogle Scholar
Georgieva, S.S., Peeters, R., Kolster, H., Todd, J.T. & Orban, G.A. (2009). The processing of 3D shape from disparity in the human brain. The Journal of Neuroscience 29, 727–742.CrossRefGoogle ScholarPubMed
Georgieva, S.S., Todd, J.T., Peeters, R. & Orban, G.A. (2008). The extraction of 3D shape from texture and shading in the human brain. Cerebral Cortex 18, 2416–2438.CrossRefGoogle ScholarPubMed
Gilbert, C.D., Sigman, M. & Crist, R.E. (2001). The neural basis of perceptual learning. Neuron 31, 681–697.CrossRefGoogle ScholarPubMed
Goldstone, R.L. (1998). Perceptual learning. Annual Review of Psychology 49, 585–612.CrossRefGoogle ScholarPubMed
Goldstone, R.L., Lippa, Y. & Shiffrin, R.M. (2001). Altering object representations through category learning. Cognition 78, 27–43.CrossRefGoogle ScholarPubMed
Grill-Spector, K., Henson, R. & Martin, A. (2006). Repetition and the brain: Neural models of stimulus-specific effects. Trends in Cognitive Sciences 10, 14–23.CrossRefGoogle ScholarPubMed
Grill-Spector, K. & Malach, R. (2001). fMR-adaptation: A tool for studying the functional properties of human cortical neurons. Acta Psychologica 107, 293–321.CrossRefGoogle ScholarPubMed
Haynes, J.D. & Rees, G. (2005). Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nature Neuroscience 8, 686–691.CrossRefGoogle ScholarPubMed
Haynes, J.D. & Rees, G. (2006). Decoding mental states from brain activity in humans. Nature Reviews. Neuroscience 7, 523–534.CrossRefGoogle ScholarPubMed
Heeger, D.J. & Ress, D. (2002). What does fMRI tell us about neuronal activity? Nature Reviews. Neuroscience 3, 142–151.CrossRefGoogle ScholarPubMed
Jacobs, R.A. (2009). Adaptive precision pooling of model neuron activities predicts the efficiency of human visual learning. Journal of Vision 9, 22, 1–15.CrossRefGoogle ScholarPubMed
Janssen, P., Vogels, R., Liu, Y. & Orban, G.A. (2003). At least at the level of inferior temporal cortex, the stereo correspondence problem is solved. Neuron 37, 693–701.CrossRefGoogle ScholarPubMed
Jehee, J.F.M., Ling, S., Swisher, J.D., van Bergen, R.S. & Tong, F. (2012). Perceptual learning selectively refines orientation representations in early visual cortex. The Journal of Neuroscience 32, 16747–16753.CrossRefGoogle ScholarPubMed
Johnson, J.S. & Olshausen, B.A. (2003). Timecourse of neural signatures of object recognition. Journal of Vision 3, 499–512.CrossRefGoogle ScholarPubMed
Kamitani, Y. & Tong, F. (2005). Decoding the visual and subjective contents of the human brain. Nature Neuroscience 8, 679–685.CrossRefGoogle ScholarPubMed
Kanwisher, N., Chun, M.M., McDermott, J. & Ledden, P.J. (1996). Functional imagining of human visual recognition. Brain Research. Cognitive Brain Research 5, 55–67.CrossRefGoogle Scholar
Kourtzi, Z., Betts, L.R., Sarkheil, P. & Welchman, A.E. (2005). Distributed neural plasticity for shape learning in the human visual cortex. PLoS Biology 3, e204.CrossRefGoogle ScholarPubMed
Kourtzi, Z. & Kanwisher, N. (2001). Representation of perceived object shape by the human lateral occipital complex. Science 293, 1506–1509.CrossRefGoogle ScholarPubMed
Krekelberg, B., Boynton, G.M. & van Wezel, R.J.A. (2006). Adaptation: From single cells to BOLD signals. Trends in Neurosciences 29, 250–256.CrossRefGoogle ScholarPubMed
Kriegeskorte, N., Cusack, R. & Bandettini, P. (2010). How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? Neuroimage 49, 1965–1976.CrossRefGoogle ScholarPubMed
Law, C.T. & Gold, J.I. (2008). Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area. Nature Neuroscience 11, 505–513.CrossRefGoogle Scholar
Li, S., Mayhew, S.D. & Kourtzi, Z. (2009). Learning shapes the representation of behavioral choice in the human brain. Neuron 62, 441–452.CrossRefGoogle ScholarPubMed
Li, S., Mayhew, SD & Kourtzi, Z. (2012). Learning shapes spatiotemporal brain patterns for flexible categorical decisions. Cerebral Cortex 22, 2322–2335.CrossRefGoogle ScholarPubMed
Li, W., Piech, V. & Gilbert, C.D. (2004). Perceptual learning and top-down influences in primary visual cortex. Nature Neuroscience 7, 651–657.CrossRefGoogle ScholarPubMed
Logothetis, N.K., Pauls, J., Augath, M., Trinath, T. & Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–170.CrossRefGoogle ScholarPubMed
Mayhew, S.D., Li, S. & Kourtzi, Z. (2012). Learning acts on distinct processes for visual form perception in the human brain. The Journal of Neuroscience 32, 775–786.CrossRefGoogle ScholarPubMed
Miller, K.L., Luh, W.M., Liu, T.T., Martinez, A., Obata, T., Wong, E.C., Frank, L.R. & Buxton, R.B. (2001). Nonlinear temporal dynamics of the cerebral blood flow response. Human Brain Mapping 13, 1–12.CrossRefGoogle ScholarPubMed
Mukai, I., Kim, D., Fukunaga, M., Japee, S., Marrett, S. & Ungerleider, L.G. (2007). Activations in visual and attention-related areas predict and correlate with the degree of perceptual learning. The Journal of Neuroscience 27, 11401–11411.CrossRefGoogle ScholarPubMed
Mukamel, R., Gelbard, H., Arieli, A., Hasson, U., Fried, I. & Malach, R. (2005). Coupling between neuronal firing, field potentials, and fMRI in human auditory cortex. Science 309, 951–954.CrossRefGoogle ScholarPubMed
Norman, K.A., Polyn, S.M., Detre, G.J. & Haxby, J.V. (2006). Beyond mind-reading: Multi-voxel pattern analysis of fMRI data. Trends in Cognitive Sciences 10, 424–430.CrossRefGoogle ScholarPubMed
Ohla, K., Busch, N.A., Dahlem, M.A. & Herrmann, C.S. (2005). Circles are different: The perception of glass patterns modulates early event-related potentials. Vision Research 45, 2668–2676.CrossRefGoogle ScholarPubMed
Op de Beeck, H.P., Baker, C.I., DiCarlo, J.J. & Kanwisher, N.G. (2006). Discrimination training alters object representations in human extrastriate cortex. The Journal of Neuroscience 26, 13025–13036.CrossRefGoogle ScholarPubMed
Orban, G.A., Janssen, P. & Vogels, R. (2006). Extracting 3D structure from disparity. Trends in Neurosciences 29, 466–473.CrossRefGoogle ScholarPubMed
Parker, A.J. (2007). Binocular depth perception and the cerebral cortex. Nature Reviews. Neuroscience 8, 379–391.CrossRefGoogle ScholarPubMed
Pei, F., Pettet, M.W., Vildavski, V.Y. & Norcia, A.M. (2005). Event-related potentials show configural specificity of global form processing. Neuroreport 16, 1427–1430.CrossRefGoogle ScholarPubMed
Philiastides, M.G. & Sajda, P. (2006). Temporal characterization of the neural correlates of perceptual decision making in the human brain. Cerebral Cortex 16, 509–518.CrossRefGoogle ScholarPubMed
Philiastides, M.G. & Sajda, P. (2007). EEG-informed fMRI reveals spatiotemporal characteristics of perceptual decision making. The Journal of Neuroscience 27, 13082–13091.CrossRefGoogle ScholarPubMed
Ponce, C.R., Lomber, S.G. & Born, R.T. (2008). Integrating motion and depth via parallel pathways. Nature Neuroscience 11, 216–223.CrossRefGoogle ScholarPubMed
Preston, T.J., Kourtzi, Z. & Welchman, A.E. (2009). Adaptive estimation of three-dimensional structure in the human brain. The Journal of Neuroscience 29, 1688–1698.CrossRefGoogle ScholarPubMed
Preston, T.J., Li, S., Kourtzi, Z. & Welchman, A.E. (2008). Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain. The Journal of Neuroscience 28, 11315–11327.CrossRefGoogle ScholarPubMed
Raemaekers, M., Lankheet, M.J.M., Moorman, S., Kourtzi, Z. & van Wezel, R.J.A. (2009). Directional anisotropy of motion responses in retinotopic cortex. Human Brain Mapping 30, 3970–3980.CrossRefGoogle ScholarPubMed
Ress, D. & Heeger, D.J. (2003). Neuronal correlates of perception in early visual cortex. Nature Neuroscience 6, 414–420.CrossRefGoogle ScholarPubMed
Sawamura, H., Orban, G.A. & Vogels, R. (2006). Selectivity of neuronal adaptation does not match response selectivity: A single-cell study of the fMRI adaptation paradigm. Neuron 49, 307–318.CrossRefGoogle Scholar
Scannell, J.W. & Young, M.P. (1999). Neuronal population activity and functional imaging. Proceedings of the Royal Society of London. Series B, Biological Sciences 266, 875–881.CrossRefGoogle ScholarPubMed
Sigman, M., Pan, H., Yang, Y., Stern, E., Silbersweig, D. & Gilbert, C.D. (2005). Top-down reorganization of activity in the visual pathway after learning a shape identification task. Neuron 46, 823–835.CrossRefGoogle ScholarPubMed
Swisher, J.D., Gatenby, J.C., Gore, J.C., Wolfe, B.A., Moon, C.H., Kim, S.G. & Tong, F. (2010). Multiscale pattern analysis of orientation-selective activity in the primary visual cortex. The Journal of Neuroscience 30, 325–330.CrossRefGoogle ScholarPubMed
Tanskanen, T., Saarinen, J., Parkkonen, L. & Hari, R. (2008). From local to global: Cortical dynamics of contour integration. Journal of Vision 8, 11–12.CrossRefGoogle ScholarPubMed
Teich, A.F. & Qian, N. (2003). Learning and adaptation in a recurrent model of V1 orientation selectivity. Journal of Neurophysiology 89, 2086–2100.CrossRefGoogle Scholar
Teller, D.Y. (1984). Linking propositions. Vision Research 24, 1233–1246.CrossRefGoogle ScholarPubMed
Teller, D.Y. & Pugh, E.N. Jr. (1983). Linking propositions in color vision. In Colour Vision: Physiology and Psychophysics, ed. Mollon, J.D. & Sharpe, L.T., London, UK: Academic Press.Google Scholar
Uka, T. & DeAngelis, G.C. (2003). Contribution of middle temporal area to coarse depth discrimination: Comparison of neuronal and psychophysical sensitivity. The Journal of Neuroscience 23, 3515–3530.CrossRefGoogle ScholarPubMed
Viswanathan, A. & Freeman, R.D. (2007). Neurometabolic coupling in cerebral cortex reflects synaptic more than spiking activity. Nature Neuroscience 10, 1308–1312.CrossRefGoogle ScholarPubMed
Vul, E. & Pashler, H. (2012). Voodoo and circularity errors. Neuroimage 62, 945–948.CrossRefGoogle ScholarPubMed
Welchman, A.E., Deubelius, A., Conrad, V., Bülthoff, H.H. & Kourtzi, Z. (2005). 3D shape perception from combined depth cues in human visual cortex. Nature Neuroscience 8, 820–827.CrossRefGoogle ScholarPubMed
Yotsumoto, Y., Watanabe, T. & Sasaki, Y. (2008). Different dynamics of performance and brain activation in the time course of perceptual learning. Neuron 57, 827–833.CrossRefGoogle ScholarPubMed
Zatorre, R., Fields, R. & Johansen-Berg, H. (2012). Plasticity in gray and white: Neuroimaging changes in brain structure during learning. Nature Neuroscience 15, 528–536.CrossRefGoogle ScholarPubMed
Zhang, J., Meeson, A., Welchman, A.E. & Kourtzi, Z. (2010). Learning alters the tuning of fMRI multi-voxel patterns for visual forms. The Journal of Neuroscience 30, 14127–14133.CrossRefGoogle Scholar