Skip to main content Accessibility help
×
Home

Sounds can affect visual perception mediated primarily by the parvocellular pathway

  • PHILIP M. JAEKL (a1) and LAURENCE R. HARRIS (a1)

Abstract

We investigated the effect of auditory–visual sensory integration on visual tasks that were predominantly dependent on parvocellular processing. These tasks were (i) detecting metacontrast-masked targets and (ii) discriminating orientation differences between high spatial frequency Gabor patch stimuli. Sounds that contained no information relevant to either task were presented before, synchronized with, or after the visual targets, and the results were compared to conditions with no sound. Both tasks used a two-alternative forced choice technique. For detecting metacontrast-masked targets, one interval contained the visual target and both (or neither) intervals contained a sound. Sound–target synchrony within 50 ms lowered luminance thresholds for detecting the presence of a target compared to when no sound occurred or when sound onset preceded target onset. Threshold angles for discriminating the orientation of a Gabor patch consistently increased in the presence of a sound. These results are compatible with sound-induced activity in the parvocellular visual pathway increasing the visibility of flashed targets and hindering orientation discrimination.

Copyright

Corresponding author

Address correspondence and reprint requests to: Philip Jaekl, Centre for Brain and Cognition, Universitat Pompeu Fabra, C/Roc Boronat, 138, 08018 Barcelona, Spain. E-mail: phil.jaekl@upf.edu

References

Hide All
Alpern, M. (1953). Metacontrast. Journal of the Optical Society of America 43, 648–657.
Ansorge, U., Klotz, W. & Neumann, O. (1998). Manual and verbal responses to completely masked (unreportable) stimuli: Exploring some conditions for the metacontrast dissociation. Perception 27, 1177–1189.
Berger, T.D., Martelli, M. & Pelli, D.G. (2003). Flicker flutter: Is an illusory event as good as the real thing? Journal of Vision 3, 406–412.
Bertelson, P. & Aschersleben, G. (2003). Temporal ventriloquism: Crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. International Journal of Psychophysiology 50, 147–155.
Bolognini, N., Frassinetti, F., Serino, A. & Ladavas, E. (2005). “Acoustical vision” of below threshold stimuli: Interaction among spatially converging audiovisual inputs. Experimental Brain Research 160, 273–282.
Bookheimer, S.Y., Zeffiro, T.A., Blaxton, T.A., Gaillard, W.D., Malow, B. & Theodore, W.H. (1998). Regional cerebral blood flow during auditory responsive naming: Evidence for cross-modality neural activation. Neuroreport 9, 2409–2413.
Brainard, D.H. (1997). The psychophysics toolbox. Spatial Vision 10, 433–436.
Breitmeyer, B. (1984). Visual Masking: An Integrative Approach. Oxford: Oxford University Press.
Breitmeyer, B.G. & Ganz, L. (1976). Implications of sustained and transient channels for theories of visual pattern masking, saccadic suppression, and information processing. Psychological Review 83, 1–36.
Breitmeyer, B.G. & Ogmen, H. (2006). Visual Masking: Time Slices through Conscious and Unconscious Vision (2nd ed.). New York: Oxford University Press.
Breitmeyer, B.G., Ro, T. & Singhal, N.S. (2004). Unconscious color priming occurs at stimulus- not percept-dependent levels of processing. Psychological Science 15, 198–202.
Bridgeman, B. (1980). Temporal response characteristics of cells in monkey striate cortex measured with metacontrast masking and brightness discrimination. Brain Research 196, 347–364.
Burr, D.C., Ross, J. & Morrone, M.C. (1986). Seeing objects in motion. Proceedings of the Royal Society of London. Series B 227, 249–265.
Busse, L., Roberts, K.C., Crist, R.E., Weissman, D.H. & Woldorff, M.G. (2005). The spread of attention across modalities and space in a multisensory object. Proceedings of the National Academy of Sciences of the United States of America 102, 18751–18756.
Callaway, E.M. (2005). Neural substrates within primary visual cortex for interactions between parallel visual pathways. Progress in Brain Research 149, 59–64.
Calvert, G.A., Spence, C. & Stein, B.E. (2004). The Handbook of Multisensory Processes. Cambridge, MA: MIT.
Calvert, G.A. & Thesen, T. (2004). Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology, Paris 98, 191–205.
Celesia, G.G. (1976). Organization of auditory cortical areas in man. Brain 99, 403–414.
Clark, V.P. & Hillyard, S.A. (1996). Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential. Journal of Cognitive Neuroscience 8, 387–402.
Clavagnier, S., Falchier, A. & Kennedy, H. (2004). Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective & Behavioral Neuroscience 4, 117–126.
Colombo, M. & Gross, C.G. (1994). Responses of inferior temporal cortex and hippocampal neurons during delayed matching to sample in monkeys (Macaca fascicularis). Behavioral Neuroscience 108, 443–455.
Cornsweet, T.N. & Pinsker, H.M. (1965). Luminance discrimination of brief flashes under various conditions of adaptation. The Journal of Physiology 176, 294–310.
De Gelder, B. & Bertelson, P. (2003). Multisensory integration, perception and ecological validity. Trends in Cognitive Sciences 7, 460–467.
Dehaene, S. (2003). The neural basis of the Weber-Fechner law: A logarithmic mental number line. Trends in Cognitive Sciences 7, 145–147.
Desimone, R., Albright, T.D., Gross, C.G. & Bruce, C. (1984). Stimulus-selective properties of inferior temporal neurons in the macaque. The Journal of Neuroscience 4, 2051–2062.
Desimone, R., Fleming, J. & Gross, C.G. (1980). Prestriate afferents to inferior temporal cortex: An HRP study. Brain Research 184, 41–55.
Distler, C. & Hoffmann, K.P. (1993). Visual receptive field properties in kitten pretectal nucleus of the optic tract and dorsal terminal nucleus of the accessory optic tract. Journal of Neurophysiology 70, 814–827.
Driver, J. & Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57, 11–23.
Driver, J. & Spence, C. (2000). Multisensory perception: Beyond modularity and convergence. Current Biology 10, R731–R735.
Enns, J.T. & Di Lollo, V. (2000). What’s new in visual masking? Trends in Cognitive Sciences 4, 345–352.
Falchier, A., Clavagnier, S., Barone, P. & Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. The Journal of Neuroscience 22, 5749–2759.
Fendrich, R. & Corballis, P.M. (2001). The temporal cross-capture of audition and vision. Perception & Psychophysics 63, 719–725.
Foxe, J.J. & Schroeder, C.E. (2005). The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16, 419–423.
Foxe, J.J. & Simpson, G.V. (2002). Flow of activation from V1 to frontal cortex in humans. A framework for defining “early” visual processing. Experimental Brain Research 142, 139–150.
Frassinetti, F., Bolognini, N. & Ladavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research 147, 332–343.
Gattass, C.R., Ghobrial, I. & Bunn-Moreno, M.M. (1988). Specific inhibition of OKT8 binding to peripheral blood mononuclear cells by jacalin. Immunology Letters 17, 133–138.
Gescheider, G.A. (1997). Psychophysics: The Fundamentals (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Gibson, J.R. & Maunsell, J.H. (1997). Sensory modality specificity of neural activity related to memory in visual cortex. Journal of Neurophysiology 78, 1263–1275.
Goodale, M.A. & Milner, A.D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences 15, 20–25.
Griebel, U. & Schmid, A. (1997). Brightness discrimination ability in the West Indian manatee (Trichechus manatus). The Journal of Experimental Biology 200, 1587–1592.
Iwai, E., Aihara, T. & Hikosaka, K. (1987). Inferotemporal neurons of the monkey responsive to auditory signal. Brain Research 410, 121–124.
Klotz, W. & Wolff, P. (1995). The effect of a masked stimulus on the response to the masking stimulus. Psychological Research 58, 92–101.
Kondo, H. & Komatsu, H. (2000). Suppression on neuronal responses by a metacontrast masking stimulus in monkey V4. Neuroscience Research 36, 27–33.
Kopinska, A. & Harris, L.R. (2004). Simultaneity constancy. Perception 33, 1049–1060.
Leo, F., Bertini, C., Di Pelligrino, G. & Ladavas, E. (2008). Multisensory integration for orienting responses in humans requires the activation of the superior colliculus. Experimental Brain Research 186, 67–77.
Livingstone, M. & Hubel, D.H. (1988). Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science 240, 740–749.
Manjarrez, E., Mendez, I., Martinez, L., Flores, A. & Mirasso, C.R. (2007). Effects of auditory noise on the psychophysical detection of visual signals: Cross-modal stochastic resonance. Neuroscience Letters 415, 231–236.
Miller, G.A. (1947). Sensitivity to changes in the intensity of white and it’s relation to loudness and masking. The Journal of the Acoustical Society of America 19, 609–619.
Mishkin, M. & Ungerleider, L.G. (1982). Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behavioural Brain Research 6, 57–77.
Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E. & Foxe, J.J. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Research Cognitive Brain Research 14, 115–128.
Morein-Zamir, S., Soto-Faraco, S. & Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Brain Research. Cognitive Brain Research 17, 154–163.
Mullen, K.T., Dumoulin, S.O., Mcmahon, K.L., de Zbicaray, G.I. & Hess, R.F. (2007). Selectivity of human retinotopic visual cortex to S-cone-opponent, L/M-cone-opponent and achromatic stimulation. The European Journal of Neuroscience 25, 491–502.
Neumann, O. & Klotz, W. (1994). Motor responses to non-reportable, masked stimuli: Where is the limit of direct parameter specification? In Attention and Performance XV: Conscious and Nonconscious Information Processing, ed. Umiltà, C. & Moskovitch, M. Cambridge, MA: MIT Press.
Perry, V.H., Oehler, R. & Cowey, A. (1984). Retinal ganglion cells that project to the dorsal lateral geniculate nucleus in the macaque monkey. Neuroscience 12, 1101–1123.
Recanzone, G.H. (2003). Auditory influences on visual temporal rate perception. Journal of Neurophysiology 89, 1078–1093.
Ringo, J.L. & O’Neill, S.G. (1993). Indirect inputs to ventral temporal cortex of monkey: The influence of unit activity of alerting auditory input, interhemispheric subcortical visual input, reward, and the behavioral response. Journal of Neurophysiology 70, 2215–2225.
Ro, T., Singhal, N.S., Breitmeyer, B.G. & Garcia, J.O. (2009). Unconscious processing of color and form in metacontrast masking. Attention, Perception & Psychophysics 71, 95–103.
Rockland, K.S. & Van Hoesen, G.W. (1994). Direct temporal-occipital feedback connections to striate cortex (V1) in the macaque monkey. Cerebral Cortex 4, 300–313.
Rogowitz, B.E. (1983). Spatial/temporal interactions: Backward and forward metacontrast masking with sine-wave gratings. Vision Research 23, 1057–1073.
Romanski, L.M. (2007). Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cerebral Cortex 17(Suppl. 1), i61–i69.
O’Scalaidhe, S.P., Rodman, H.R., Albright, T.D. & Gross, C.G. (1997). The effects of combined superior temporal polysensory area and frontal eye field lesions on eye movements in the macaque monkey. Behavioural Brain Research 84, 31–46.
Schneider, T.R., Engel, A.K. & Debener, S. (2008). Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Experimental Psychology 55, 121–132.
Shipp, S. & Zeki, S. (1985). Segregation of pathways leading from area V2 to areas V4 and V5 of macaque monkey visual cortex. Nature 315, 322–325.
Slutsky, D.A. & Recanzone, G.H. (2001). Temporal and spatial dependency of the ventriloquism effect. Neuroreport 12, 7–10.
Smiley, J.F. & Falchier, A. (2009). Multisensory connections of monkey auditory cerebral cortex. Hearing Research 258, 37–46.
Stein, B.E., London, N., Wilkinson, L.K. & Price, D.D. (1996). Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. Journal of Cognitive Neuroscience 8, 497–506.
Stein, B.E. & Stanford, T.R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews. Neuroscience 9, 255–266.
Stewart, A.L. & Purcell, D.G. (1974). Visual backward masking by a flash of light: A study of u-shaped detection functions. Journal of Experimental Psychology 103, 553–566.
Suied, C., Bonneel, N. & Viaud-Delmon, I. (2009). Integration of auditory and visual information in the recognition of realistic objects. Experimental Brain Research 194, 91–102.
Tranel, D., Damasio, H., Eichhorn, G.R., Grabowski, T., Ponto, L.L. & Hichwa, R.D. (2003). Neural correlates of naming animals from their characteristic sounds. Neuropsychologia 41, 847–854.
Tranel, D., Grabowski, T.J., Lyon, J. & Damasio, H. (2005). Naming the same entities from visual or from auditory stimulation engages similar regions of left inferotemporal cortices. Journal of Cognitive Neuroscience 17, 1293–1305.
Vroomen, J. & de Gelder, B. (2004). Temporal ventriloquism: Sound modulates the flash-lag effect. Journal of Experimental Psychology. Human Perception and Performance 30, 513–518.
Watkins, S., Shams, L., Josephs, O. & Rees, G. (2007). Activity in human V1 follows multisensory perception. NeuroImage 37, 572–578.
Watkins, S., Shams, L., Tanaka, S., Haynes, J.D. & Rees, G. (2006). Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 31, 1247–1256.
Watson, A.B. & Pelli, D.G. (1983). QUEST: a Bayesian adaptive psychometric method. Perception & Psychophysics 33, 113–120.

Keywords

Related content

Powered by UNSILO

Sounds can affect visual perception mediated primarily by the parvocellular pathway

  • PHILIP M. JAEKL (a1) and LAURENCE R. HARRIS (a1)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.