Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-76fb5796d-2lccl Total loading time: 0 Render date: 2024-04-26T05:07:08.100Z Has data issue: false hasContentIssue false

17 - The time marker account of cross-channel temporal judgments

from Part III - Temporal phenomena: binding and asynchrony

Published online by Cambridge University Press:  05 October 2010

By
Shin'ya Nishida  and
Alan Johnston
Edited by
Romi Nijhawan  and
Beena Khurana
Romi Nijhawan
Affiliation:
University of Sussex
Beena Khurana
Affiliation:
University of Sussex
Get access

Summary

Summary

The human sensory system, at least in its early stages, consists of multiple channels for different modalities (e.g., vision, audition) and for different attributes in each modality (color, motion). Temporal congruency is a critical factor in the binding of signals across channels, but little is known about what representations and algorithms are used for matching. We first analyze this mechanism from a general theoretical point of view and then address the specific mechanisms underlying the perception of color–motion synchrony and audiovisual simultaneity. We hypothesize that judgments about cross-channel temporal relations are based on the comparison of time markers by a mid-level perceptual process. The time markers are amodal tokens that reference salient, figural features extracted from early-level sensory signals. A temporal marker should reference the time a specific event occurs in the world rather than the time the processing of the event completes in the brain.

Introduction

The human sensory system has a complex architecture. It consists of multiple parallel channels for different sensory modalities (e.g., vision, audition). The channel for each sensory modality is subdivided into multiple parallel channels, each specialized for processing of different attributes (e.g., color, motion). Furthermore each channel consists of multiple serial processing stages. The transmission and processing of sensory information by neural mechanisms takes time, and the amount of time taken varies significantly across channels. For example, in monkey visual cortex, the latency of stimulus onset evoked response is about 40–100 msec in V1, 50–100 msec in MT, 70–160 msec in V4, and 90–180 msec in IT (Bullier 2001).

Type
Chapter
Information
Space and Time in Perception and Action , pp. 278 - 300
Publisher: Cambridge University Press
Print publication year: 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adams, W. J., & Mamassian, P. (2004). The effects of task and saliency on latencies for colour and motion processing. Proc R Soc Lond B Biol Sci 271: 139–146.CrossRefGoogle ScholarPubMed
Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. J Opt Soc Am A 2: 284–299.CrossRefGoogle ScholarPubMed
Amano, K., Goda, N., Nishida, S., Ejima, Y., Takeda, T., & Ohtani, Y. (2006). Estimation of the timing of human visual perception from magnetoencephalography. J Neurosci 26: 3981–3991.CrossRefGoogle ScholarPubMed
Amano, K., Johnston, A., & Nishida, S. (2007). Two mechanisms underlying the effect of angle of motion direction change on colour-motion asynchrony. Vision Res 47: 687–705.CrossRefGoogle ScholarPubMed
Amano, K., Nishida, S., & Takeda, T. (2004a). Enhanced neural responses correlated with perceptual binding of color and motion. Neurol Clin Neurophysiol 2004: 48.Google Scholar
Amano, K., Nishida, S., & Takeda, T. (2004b). MEG responses for color-motion asynchrony [Abstract]. J Vis 4: 554a.CrossRefGoogle Scholar
Arnold, D. H. (2005). Perceptual pairing of colour and motion. Vision Res 45: 3015–3026.CrossRefGoogle Scholar
Arnold, D. H., & Clifford, C. W. (2002). Determinants of asynchronous processing in vision. Proc R Soc Lond B Biol Sci 269: 579–583.CrossRefGoogle ScholarPubMed
Arnold, D. H., Clifford, C. W., & Wenderoth, P. (2001). Asynchronous processing in vision: color leads motion. Curr Biol 11: 596–600.CrossRefGoogle ScholarPubMed
Arrighi, R., Alais, D., & Burr, D. (2005). Perceived timing of first- and second-order changes in vision and hearing. Exp Brain Res 166: 445–454.CrossRefGoogle ScholarPubMed
Ashida, H., Seiffert, A. E., & Osaka, N. (2001). Inefficient visual search for second-order motion. J Opt Soc Am A Opt Image Sci Vis 18: 2255–2266.CrossRefGoogle ScholarPubMed
Aymoz, C., & Viviani, P. (2004). Perceptual asynchronies for biological and non-biological visual events. Vision Res 44: 1547–1563.CrossRefGoogle ScholarPubMed
Bachmann, T., Poder, E., & Luiga, I. (2004). Illusory reversal of temporal order: the bias to report a dimmer stimulus as the first. Vision Res 44: 241–246.CrossRefGoogle ScholarPubMed
Bartels, A., & Zeki, S. (1998). The theory of multistage integration in the visual brain. Proc R Soc Lond B Biol Sci 265: 2327–2332.CrossRefGoogle ScholarPubMed
Bedell, H. E., Chung, S. T., Ogmen, H., & Patel, S. S. (2003). Color and motion: which is the tortoise and which is the hare? Vision Res 43: 2403–2412.CrossRefGoogle ScholarPubMed
Braddick, O. (1974). A short-range process in apparent motion. Vision Res 14: 519–527.CrossRefGoogle ScholarPubMed
Bullier, J. (2001). Integrated model of visual processing. Brain Res Brain Res Rev 36: 96–107.CrossRefGoogle ScholarPubMed
Burr, D. C., & Ross, J. (2002). Direct evidence that “speedlines” influence motion mechanisms. J Neurosci 22: 8661–8664.CrossRefGoogle ScholarPubMed
Castelo-Branco, M., Goebel, R., Neuenschwander, S., & Singer, W. (2000). Neural synchrony correlates with surface segregation rules. Nature 405: 685–689.CrossRefGoogle ScholarPubMed
Cavanagh, P., Arguin, M., & von Grünau, M. (1989). Interattribute apparent motion. Vision Res 29: 1197–1204.CrossRefGoogle ScholarPubMed
Cavanagh, P., Holcombe, A. O., & Chou, W. (2008). Mobile computation: spatiotemporal integration of the properties of objects in motion. J Vis 8(12): 1, 1–23.CrossRefGoogle ScholarPubMed
Churchland, P. S. (1981). On the alleged backwards referral of experiences and its relevance to the mind-body problem. Philos Sci 48: 165–181.CrossRefGoogle Scholar
Clifford, C. W., Arnold, D. H., & Pearson, J. (2003). A paradox of temporal perception revealed by a stimulus oscillating in colour and orientation. Vision Res 43: 2245–2253.CrossRefGoogle ScholarPubMed
Clifford, C. W., Spehar, B., & Pearson, J. (2004). Motion transparency promotes synchronous perceptual binding. Vision Res 44: 3073–3080.CrossRefGoogle ScholarPubMed
Cook, E. P., & Maunsell, J. H. (2002). Dynamics of neuronal responses in macaque MT and VIP during motion detection. Nat Neurosci 5: 985–994.CrossRefGoogle ScholarPubMed
Dennett, D. C. (1991). Consciousness Explained. Boston: Little Brown & Co.Google Scholar
Dennett, D. C., & Kinsbourne, M. (1992). Time and the observer: the where and when of consciousness in the brain. Behav Brain Sci 15: 183–247.CrossRefGoogle Scholar
Eagleman, D. M., & Sejnowski, T. J. (2000). Motion integration and postdiction in visual awareness. Science 287: 2036–2038.CrossRefGoogle ScholarPubMed
Engel, A. K., Fries, P., & Singer, W. (2001). Dynamic predictions: oscillations and synchrony in top-down processing. Nat Rev Neurosci 2: 704–716.CrossRefGoogle ScholarPubMed
Fujisaki, W., Koene, A., Arnold, D., Johnston, A., & Nishida, S. (2006). Visual search for a target changing in synchrony with an auditory signal. Proc R Soc Lond B Biol Sci 273: 865–874.CrossRefGoogle ScholarPubMed
Fujisaki, W., & Nishida, S. (2005). Temporal frequency characteristics of synchrony-asynchrony discrimination of audio-visual signals. Exp Brain Res 166: 455–464.CrossRefGoogle ScholarPubMed
Fujisaki, W., & Nishida, S. (2007). Feature-based processing of audio-visual synchrony perception revealed by random pulse trains. Vision Res 47: 1075–1093.CrossRefGoogle ScholarPubMed
Fujisaki, W., & Nishida, S. (2008). Top-down feature-based selection of matching features for audio-visual synchrony discrimination. Neurosci Lett 433: 225–230.CrossRefGoogle ScholarPubMed
Fujisaki, W., & Nishida, S. (2009). Audio–tactile superiority over visuo–tactile and audio–visual combinations in the temporal resolution of synchrony perception. Exp Brain Res 198: 245–259.CrossRefGoogle ScholarPubMed
Fujisaki, W., Shimojo, S., Kashino, M., & Nishida, S. (2004). Recalibration of audiovisual simultaneity. Nat Neurosci 7: 773–778.CrossRefGoogle ScholarPubMed
Geisler, W. S. (1999). Motion streaks provide a spatial code for motion direction. Nature 400: 65–69.CrossRefGoogle ScholarPubMed
Gottsdanker, R. (1956). The ability of human operators to detect acceleration of target motion. Psychol Bull 53: 477–487.CrossRefGoogle ScholarPubMed
He, S., Cavanagh, P., & Intriligator, J. (1996). Attentional resolution and the locus of visual awareness. Nature 383: 334–337.CrossRefGoogle ScholarPubMed
Holcombe, A. O., & Cavanagh, P. (2001). Early binding of feature pairs for visual perception. Nat Neurosci 4: 127–128.CrossRefGoogle ScholarPubMed
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nat Rev Neurosci 2: 194–203.CrossRefGoogle ScholarPubMed
Jaskowski, P. (1996). Simple reaction time and perception of temporal order: dissociations and hypotheses. Percept Mot Skills 82: 707–730.CrossRefGoogle ScholarPubMed
Johnston, A., McCowan, P. W., & Buxton, H. (1992). A computational model of the analysis of some first-order and second-order motion patterns by simple and complex cells. Proc R Soc Lond B Biol Sci 250: 297–306.CrossRefGoogle ScholarPubMed
Johnston, A., & Nishida, S. (2001). Time perception: brain time or event time? Curr Biol 11: R427–430.CrossRefGoogle ScholarPubMed
Kanai, R., Paffen, C. L., Gerbino, W., & Verstraten, F. A. (2004). Blindness to inconsistent local signals in motion transparency from oscillating dots. Vision Res 44: 2207–2212.CrossRefGoogle ScholarPubMed
Kawano, K., Shidara, M., Watanabe, Y., & Yamane, S. (1994). Neural activity in cortical area MST of alert monkey during ocular following responses. J Neurophysiol 71: 2305–2324.CrossRefGoogle ScholarPubMed
Keeble, D. R., & Nishida, S. (2001). Micropattern orientation and spatial localization. Vision Res 41: 3719–3733.CrossRefGoogle ScholarPubMed
Kolers, P. A. (1972). Aspects of Motion Perception. New York: Pergamon Press.Google Scholar
Kopinska, A., & Harris, L. R. (2004). Simultaneity constancy. Perception 33: 1049–1060.CrossRefGoogle ScholarPubMed
Libet, B. (1981). The experimental evidence for subjective referral of a sensory experience backwards in time: Reply to P.S. Churchland. Philos Sci 48: 182–197.CrossRefGoogle Scholar
Libet, B., Wright, E. W. Jr., Feinstein, B., & Pearl, D. K. (1979). Subjective referral of the timing for a conscious sensory experience: a functional role for the somatosensory specific projection system in man. Brain 102: 193–224.CrossRefGoogle ScholarPubMed
Linares, D., & Lopez-Moliner, J. (2006). Perceptual asynchrony between color and motion with a single direction change. J Vis 6: 974–981.CrossRefGoogle ScholarPubMed
Lu, Z. L., & Sperling, G. (1995). Attention-generated apparent motion. Nature 377: 237–239.CrossRefGoogle ScholarPubMed
Lu, Z. L., & Sperling, G. (2001). Three-systems theory of human visual motion perception: review and update. J Opt Soc Am A Opt Image Sci Vis 18: 2331–2370.CrossRefGoogle ScholarPubMed
Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: Freeman.Google Scholar
McDonald, J. J., Teder-Salejarvi, W. A., Russo, F. D., & Hillyard, S. A. (2005). Neural basis of auditory-induced shifts in visual time-order perception. Nat Neurosci 8: 1197–1202.CrossRefGoogle ScholarPubMed
Moradi, F., & Shimojo, S. (2004). Perceptual-binding and persistent surface segregation. Vision Res 44: 2885–2899.CrossRefGoogle ScholarPubMed
Morrone, M. C., Ross, J., & Burr, D. (2005). Saccadic eye movements cause compression of time as well as space. Nat Neurosci 8: 950–954.CrossRefGoogle ScholarPubMed
Moutoussis, K., & Zeki, S. (1997a). A direct demonstration of perceptual asynchrony in vision. Proc R Soc Lond B Biol Sci 264: 393–399.CrossRefGoogle ScholarPubMed
Moutoussis, K., & Zeki, S. (1997b). Functional segregation and temporal hierarchy of the visual perceptive systems. Proc R Soc Lond B Biol Sci 264: 1407–1414.CrossRefGoogle ScholarPubMed
Neumann, O., Esselmann, U., & Klotz, W. (1993). Differential effects of visual-spatial attention on response latency and temporal-order judgment. Psychol Res 56: 26–34.CrossRefGoogle ScholarPubMed
Nishida, S., & Johnston, A. (2002). Marker correspondence, not processing latency, determines temporal binding of visual attributes. Curr Biol 12: 359–368.CrossRefGoogle Scholar
Paul, L., & Schyns, P. G. (2003). Attention enhances feature integration. Vision Res 43: 1793–1798.CrossRefGoogle ScholarPubMed
Pelli, D. G., Palomares, M., & Majaj, N. J. (2004). Crowding is unlike ordinary masking: distinguishing feature integration from detection. J Vis 4: 1136–1169.CrossRefGoogle ScholarPubMed
Pöppel, E. (1997). A hierarchical model of temporal perception. Trends Cogn Sci 1: 56–61.CrossRefGoogle ScholarPubMed
Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression of visual processing in an RSVP task: an attentional blink? J Exp Psychol Hum Percept Perform 18: 849–860.CrossRefGoogle Scholar
Reeves, A., & Sperling, G. (1986). Attention gating in short-term visual memory. Psychol Rev 93: 180–206.CrossRefGoogle ScholarPubMed
Reichardt, W. (1961). Autocorrelation, a principle for the evaluation of sensory information by the central nervous system. In W. A., Rosenblith (ed.), Sensory Communication. Cambridge, MA: MIT Press.Google Scholar
Roufs, J. A. (1963). Perception lag as a function of stimulus luminance. Vision Res 3: 81–91.CrossRefGoogle Scholar
Simpson, W. A. (1994). Temporal summation of visual motion. Vision Res 34: 2547–2559.CrossRefGoogle ScholarPubMed
Stelmach, L. B., & Herdman, C. M. (1991). Directed attention and perception of temporal order. J Exp Psychol Hum Percept Perform 17: 539–550.CrossRefGoogle ScholarPubMed
Tallon-Baudry, C., Bertrand, O., Delpuech, C., & Pernier, J. (1996). Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J Neurosci 16: 4240–4249.CrossRefGoogle ScholarPubMed
Tappe, T., Niepel, M., & Neumann, O. (1994). A dissociation between reaction time to sinusoidal gratings and temporal-order judgment. Perception 23: 335–347.CrossRefGoogle ScholarPubMed
Treisman, A. M. (1999). Solutions to the binding problem: progress through controversy and convergence. Neuron 24: 105–110, 111–125.CrossRefGoogle ScholarPubMed
Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognit Psychol 12: 97–136.CrossRefGoogle Scholar
Ullman, S. (1984). Visual routines. Cognition 18: 97–159.CrossRefGoogle ScholarPubMed
Victor, J. D., & Conte, M. M. (2002). Temporal phase discrimination depends critically on separation. Vision Res 42: 2063–2071.CrossRefGoogle ScholarPubMed
Viviani, P., & Aymoz, C. (2001). Colour, form, and movement are not perceived simultaneously. Vision Res 41: 2909–2918.CrossRefGoogle Scholar
Vroomen, J., Keetels, M., de Gelder, B., & Bertelson, P. (2004). Recalibration of temporal order perception by exposure to audio-visual asynchrony. Brain Res Cogn Brain Res 22: 32–35.CrossRefGoogle ScholarPubMed
Werkhoven, P., Snippe, H. P., & Toet, A. (1992). Visual processing of optic acceleration. Vision Res 32: 2313–2329.CrossRefGoogle ScholarPubMed
Whitney, D., & Murakami, I. (1998). Latency difference, not spatial extrapolation. Nat Neurosci 1: 656–657.CrossRefGoogle Scholar
Yamamoto, S., & Kitazawa, S. (2001). Reversal of subjective temporal order due to arm crossing. Nat Neurosci 4: 759–765.CrossRefGoogle ScholarPubMed
Zeki, S. (2003). The disunity of consciousness. Trends Cogn Sci 7: 214–218.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×