Hostname: page-component-76fb5796d-2lccl Total loading time: 0 Render date: 2024-04-26T23:00:02.885Z Has data issue: false hasContentIssue false
  • English
  • Français

Visual speech fills in both discrimination and identification of non-intact auditory speech in children*

Published online by Cambridge University Press:  20 July 2017

SUSAN JERGER ,
MARKUS F. DAMIAN ,
RACHEL P. MCALPINE  and
HERVÉ ABDI
SUSAN JERGER*
Affiliation:
School of Behavioral and Brain Sciences, University of Texas at Dallas, and Callier Center for Communication Disorders
MARKUS F. DAMIAN
Affiliation:
University of Bristol, School of Experimental Psychology
RACHEL P. MCALPINE
Affiliation:
School of Behavioral and Brain Sciences, University of Texas at Dallas, and Callier Center for Communication Disorders
HERVÉ ABDI
Affiliation:
School of Behavioral and Brain Sciences, University of Texas at Dallas
*
Address for correspondence: Susan Jerger, School of Behavioral and Brain Sciences, GR4.1, University of Texas at Dallas, 800 W. Campbell Rd, Richardson, TX 75080, USA. tel: 512-216-2961; e-mail: sjerger@utdallas.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

To communicate, children must discriminate and identify speech sounds. Because visual speech plays an important role in this process, we explored how visual speech influences phoneme discrimination and identification by children. Critical items had intact visual speech (e.g. bæz) coupled to non-intact (excised onsets) auditory speech (signified by /–b/æz). Children discriminated syllable pairs that differed in intactness (i.e. bæz:/–b/æz) and identified non-intact nonwords (/–b/æz). We predicted that visual speech would cause children to perceive the non-intact onsets as intact, resulting in more same responses for discrimination and more intact (i.e. bæz) responses for identification in the audiovisual than auditory mode. Visual speech for the easy-to-speechread /b/ but not for the difficult-to-speechread /g/ boosted discrimination and identification (about 35–45%) in children from four to fourteen years. The influence of visual speech on discrimination was uniquely associated with the influence of visual speech on identification and receptive vocabulary skills.

Type
Articles
Information
Journal of Child Language , Volume 45 , Issue 2 , March 2018 , pp. 392 - 414
Check for updates
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

This research was supported by the National Institute on Deafness and Other Communication Disorders, grant DC-00421 to the University of Texas at Dallas. Dr Abdi would like to acknowledge the support of an EURIAS fellowship at the Paris Institute for Advanced Studies (France), with the support of the European Union's 7th Framework Program for research, and from funding from the French State managed by the “Agence Nationale de la Recherche (program: Investissements d'avenir, ANR-11-LABX-0027-01 Labex RFIEA+).” Sincere appreciation to speech science colleagues for their guidance and advice to adopt a perceptual criterion for editing the non-intact stimuli. We appreciate Dr Nancy Tye-Murray's comments on an earlier version of this paper. We thank the children and parents who participated and the research staff who assisted, namely Aisha Aguilera, Carissa Dees, Nina Dinh, Nadia Dunkerton, Alycia Elkins, Brittany Hernandez, Cassandra Karl, Demi Krieger, Michelle McNeal, Jeffrey Okonye, and Kimberly Periman of the University of Texas at Dallas (data collection, analysis, presentation), and Derek Hammons and Scott Hawkins of the University of Texas at Dallas and Dr Brent Spehar and Dr Nancy Tye-Murray of Washington University School of Medicine (stimuli recording and editing, computer programming).

References

REFERENCES

Abdi, H., Edelman, B., Valentin, D. & Dowling, W. (2009). Experimental design and analysis for psychology. New York: Oxford University Press.Google Scholar
Aslin, R. & Smith, L. (1988). Perceptual development. Annual Review of Psychology 39, 435–73.Google Scholar
Boothroyd, A. (1988). Linguistic factors in speechreading. Volta Review 90, 77–87.Google Scholar
Boothroyd, A., Eisenberg, L. S. & Martinez, A. S. (2010). An on-line imitative test of speech-pattern contrast perception (OlimSpac): developmental effects in normally hearing children. Journal of Speech, Language, and Hearing Research 53, 531–42.Google Scholar
Bouton, S., Cole, P. & Serniclaes, W. (2012). The influence of lexical knowledge on phoneme discrimination in deaf children with cochlear implants. Speech Communication 54, 189–98.Google Scholar
Briscoe, J., Bishop, D. & Norbury, C. (2001). Phonological processing, language, and literacy: a comparison of children with mild-to-moderate sensorineural hearing loss and those with specific language impairment. Journal of Child Psychology and Psychiatry 42, 329–40.CrossRefGoogle ScholarPubMed
Burnham, D. & Dodd, B. (2004). Auditory–visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. Developmental Psychobiology 44, 209–20.Google Scholar
Calvert, G., Spence, C. & Stein, B. (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Campbell, R. (1988). Tracing lip movements: making speech visible. Visible Language 22, 32–57.Google Scholar
Carney, A. (1996). Audition and the development of oral communication competency. In Bess, F., Gravel, J. & Tharpe, A. (eds), Amplication for children with auditory deficits, 29–54. Nashville, TN: Bill Wilkerson Center Press.Google Scholar
Clopper, C., Pisoni, D. & Tierney, A. (2006). Effects of open-set and closed-set task demands on spoken word recognition. Journal of the American Academy of Audiology 17, 331–49.Google Scholar
Conway, C. & Pisoni, D. (2008). Neurocognitive basis of implicit learning of sequential structure and its relation to language processing. Annals of the New York Academy of Sciences 1145, 113–31.Google Scholar
Desjardins, R., Rogers, J. & Werker, J. (1997). An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks. Journal of Experimental Child Psychology 66, 85–110.Google Scholar
Dodd, B. & Campbell, R. (Eds.) (1987). Hearing by eye: the psychology of lip-reading. London: Lawrence Erlbaum.Google Scholar
Dunn, L. & Dunn, D. (2007). The Peabody Picture Vocabulary Test-IV, 4th ed. Minneapolis, MN: NCS Pearson.Google Scholar
Dupont, S., Aubin, J. & Menard, L. (2005). A study of the McGurk effect in 4- and 5-year-old French Canadian children. ZAS Papers in Linguistics 40, 1–17.CrossRefGoogle Scholar
Edwards, J. & Lahey, M. (1998). Nonword repetitions of children with specific language impairment: exploration of some explanations for their inaccuracies. Applied Psycholinguistics 19, 279–309.CrossRefGoogle Scholar
Edwards, J., Munson, B. & Beckman, M. (2011). Lexicon–phonology relationships and dynamics of early language development—a commentary on Stoel-Gammon's ‘Relationships between lexical and phonological development in young children'. Journal of Child Language 38, 35–40.CrossRefGoogle ScholarPubMed
Erdener, D. & Burnham, D. (2013). The relationship between auditory-visual speech perception and language-specific speech perception at the onset of reading instruction in English-speaking children. Journal of Experimental Child Psychology 114, 120–38.Google Scholar
Files, B., Tjan, B., Jiang, J. & Bernstein, L. (2015). Visual speech discrimination and identification of natural and synthetic consonant stimuli. Frontiers in Psychology 6, 878. Online: <doi: 10.3389/fpsyg.2015.00878>.Google Scholar
Fort, M., Spinelli, E., Savariaux, C. & Kandel, S. (2010). The word superiority effect in audiovisual speech perception. Speech Communication 52, 525–32.Google Scholar
Fort, M., Spinelli, E., Savariaux, C. & Kandel, S. (2012). Audiovisual vowel monitoring and the word superiority effect in children. International Journal of Behavioral Development 36, 457–67.CrossRefGoogle Scholar
Gathercole, S. (2006). Nonword repetition and word learning: the nature of the relationship. Applied Psycholinguistics 27, 513–43.Google Scholar
Gogate, L., Walker-Andrews, A. & Bahrick, L.. (2001). The intersensory origins of word comprehension: an ecological-dynamic systems view. Developmental Science 4, 1–37.Google Scholar
Hnath-Chisolm, T., Laipply, E. & Boothroyd, A. (1998). Age-related changes on a children's test of sensory-level speech perception capacity. Journal of Speech, Language, and Hearing Research 41, 94–106.Google Scholar
Holt, R., Kirk, K. & Hay-McCutcheon, M. (2011). Assessing multimodal spoken word-in-sentence recognition in children with normal hearing and children with cochlear implants. Journal of Speech, Language, and Hearing Research 54, 632–57.Google Scholar
Hornickel, J., Skoe, E., Nicol, T., Zecker, S. & Kraus, N. (2009). Subcortical differentiation of stop consonants relates to reading and speech-in-noise perception. Proceedings of the National Academy of Sciences of the United States of America 106, 13022–27.Google Scholar
Jerger, S., Damian, M. F., Spence, M. J., Tye-Murray, N. & Abdi, H. (2009). Developmental shifts in children's sensitivity to visual speech: a new multimodal picture-word task. Journal of Experimental Child Psychology 102, 40–59.Google Scholar
Jerger, S., Damian, M., Tye-Murray, N. & Abdi, H. (2014). Children use visual speech to compensate for non-intact auditory speech. Journal of Experimental Child Psychology 126, 295–312.Google Scholar
Jerger, S., Damian, M., Tye-Murray, N. & Abdi, H. (2017). Children perceive speech onsets by ear and eye. Journal of Child Language 44, 185–215.Google Scholar
Jerger, S., Martin, R. & Jerger, J. (1987). Specific auditory perceptual dysfunction in a learning disabled child. Ear and Hearing 8, 78–86.CrossRefGoogle Scholar
Kushnerenko, E., Tomalski, P., Ballieux, H., Potton, A., Birtles, D., Frostick, C. & Moore, D. (2013). Brain responses and looking behavior during audiovisual speech integration in infants predict auditory speech comprehension in the second year of life. Frontiers in Psychology 4, 432. Online: <doi: 10.3389/fpsyg.2013.00432. eCollection 2013>.Google Scholar
Lalonde, K. & Holt, R. (2014). Cognitive and linguistic sources of variance in 2-year-olds’ speech-sound discrimination: a preliminary investigation. Journal of Speech, Language, and Hearing Research 57, 308–26.CrossRefGoogle ScholarPubMed
Lalonde, K. & Holt, R. (2015). Preschoolers benefit from visually salient speech cues. Journal of Speech, Language, and Hearing Research 58, 135–50.Google Scholar
Lalonde, K. & Holt, R. (2016). Audiovisual speech perception development at varying levels of perceptual processing. Journal of the Acoustical Society of America 139, 1713–23.Google Scholar
Lewkowicz, D. & Hansen-Tift, A. (2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceedings of the National Academy of Sciences of the United States of America 109, 1431–6.Google Scholar
Martin, R., Breedin, S. & Damian, M. (1999). The relation of phoneme discrimination, lexical access, and short-term memory: a case study and interactive activation account. Brain and Language 70, 437–82.Google Scholar
Massaro, D., Thompson, L., Barron, B. & Laren, E. (1986). Developmental changes in visual and auditory contributions to speech perception. Journal of Experimental Child Psychology 41, 93–113.Google Scholar
McConachie, H. & Moore, V. (1994). Early expressive language of severely visually impaired children. Developmental Medicine & Child Neurology 36, 230–40.Google Scholar
McGurk, H. & MacDonald, M. (1976). Hearing lips and seeing voices. Nature 264, 746–8.Google Scholar
Menn, L. & Stoel-Gammon, C. (2009). Phonological development. In Gleason, J. & Ratner, N. (eds), The development of language, 58–103. Boston, MA: Pearson.Google Scholar
Mills, A. (1987). The development of phonology in the blind child. In Dodd, B. & Campbell, R. (eds), Hearing by eye: the psychology of lipreading, 145–61. London: Erlbaum.Google Scholar
Rato, A. (2014). Effects of perceptual training on the identification of English vowels by native speakers of European Portuguese. Concordia Working Papers in Applied Linguistics 5, 529–46.Google Scholar
Rosenblum, L., Schmuckler, M. & Johnson, J. (1997). The McGurk effect in infants. Perception & Psychophysics 59, 347–57.Google Scholar
Ross, L., Molholm, S., Blanco, D., Gomez-Ramirez, M., Saint-Amour, D. & Foxe, J. (2011). The development of multisensory speech perception continues into the late childhood years. European Journal of Neuroscience 33, 2329–37.Google Scholar
Rubin, P., Turvey, M. & van Gelder, P. (1976). Initial phonemes are detected faster in spoken words than in spoken nonwords. Perception & Psychophysics 19, 394–8.Google Scholar
Snowling, M. & Hulme, C. (1994). The development of phonological skills. Philosophical Transactions of the Royal Society of London B 346, 21–7.Google Scholar
Sommers, M., Kirk, K. & Pisoni, D. (1997). Some considerations in evaluating spoken word recognition by normal-hearing, noise-masked normal-hearing, and cochlear implant listeners I: the effects of response format. Ear & Hearing 18, 89–99.Google Scholar
Teinonen, T., Aslin, R., Alku, P. & Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition 108, 850–5.Google Scholar
Tremblay, C., Champoux, R., Voss, P., Bacon, B., Lepore, F. & Theoret, H. (2007). Speech and non-speech audio-visual illusions: a developmental study. PLoS One 2(8), online: <e742.DOI:10.1371/journal.pone.0000742>.Google Scholar
Tsao, F., Liu, H. & Kuhl, P. (2004). Speech perception in infancy predicts language development in the second year of life: a longitudinal study. Child Development 75, 1067–84.Google Scholar
Tye-Murray, N. (2014). Foundations of aural rehabilitation: children, adults, and their family members, 4th ed. Boston, MA: Cengage Learning.Google Scholar