Hostname: page-component-8448b6f56d-c4f8m Total loading time: 0 Render date: 2024-04-24T17:11:32.877Z Has data issue: false hasContentIssue false
  • English
  • Français

An ERP study on novel word learning in an immersive virtual reality context

Published online by Cambridge University Press:  07 June 2023

Lu Jiao ,
Mengrui Zhu ,
Zijie Xu ,
Guanzhu Zhou ,
John W. Schwieter  and
Cong Liu Open the ORCID record for Cong Liu [Opens in a new window]
Lu Jiao
Affiliation:
Department of Psychology, Normal College & School of Teacher Education, Qingdao University, Qingdao, China
Mengrui Zhu
Affiliation:
Department of Psychology, Normal College & School of Teacher Education, Qingdao University, Qingdao, China
Zijie Xu
Affiliation:
School of Foreign Languages, Qingdao University, Qingdao, China
Guanzhu Zhou
Affiliation:
Department of Psychology, Normal College & School of Teacher Education, Qingdao University, Qingdao, China
John W. Schwieter
Affiliation:
Language Acquisition, Multilingualism, and Cognition Laboratory / Bilingualism Matters @ Wilfrid Laurier University, Waterloo, Canada
Cong Liu*
Affiliation:
Department of Psychology, Normal College & School of Teacher Education, Qingdao University, Qingdao, China
*
Corresponding author: Cong Liu Department of Psychology Qingdao University, No.308, NingXia R. Qingdao, 266071, China Email: congu902@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

In this behavioral and electrophysiological study, we compare novel word learning, particularly lexical form acquisition, in an immersive virtual reality (VR) context with a picture-word (PW) association context. We also test whether inhibitory control and age of second language acquisition (L2 AoA) have modulating effects. Chinese speakers of L2 English learned two sets of German words, one set in each of the contexts. Behavioral performance from a subsequent recognition task indicated that responses to VR-learned words were faster than PW-leaned words. ERPs revealed that VR-learned words elicited more negative N100 and N200 waveforms than PW-learned words. Moreover, a significant relationship between L2 AoA and N200 amplitude was observed for VR-learned words. Taken together, the results suggest that the multi-sensory, interactive experience simulated by an immersive VR context has a positive effect on early lexical form acquisition of novel words.

Keywords

Novel word learningbilingualismimmersive virtual realityERPs
Type
Research Article
Information
Bilingualism: Language and Cognition , Volume 27 , Issue 1 , January 2024 , pp. 128 - 136
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278.CrossRefGoogle ScholarPubMed
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.CrossRefGoogle ScholarPubMed
Basirat, A., Brunellière, A., & Hartsuiker, R. (2018). The role of audiovisual speech in the early stages of lexical processing as revealed by the ERP word repetition effect. Language Learning, 68, 80–101.CrossRefGoogle Scholar
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4. R package version, 1(7), 1–23.Google Scholar
Biau, E., Fromont, L. A., & Soto-Faraco, S. (2018). Beat gestures and syntactic parsing: An ERP study. Language Learning, 68, 102–126.CrossRefGoogle Scholar
Bogulski, C. A., Bice, K., & Kroll, J. F. (2019). Bilingualism as a desirable difficulty: Advantages in word learning depend on regulation of the dominant language. Bilingualism: Language and Cognition, 22(5), 1052–1067.CrossRefGoogle ScholarPubMed
Connolly, J. F., & Phillips, N. A. (1994). Event-related potential components reflect phonological and semantic processing of the terminal word of spoken sentences. Journal of Cognitive Neuroscience, 6(3), 256–266.CrossRefGoogle ScholarPubMed
Fan, J., McCandliss, B. D., Sommer, T., Raz, A., & Posner, M. I. (2002). Testing the efficiency and independence of attentional networks. Journal of Cognitive Neuroscience, 14(3), 340–347.CrossRefGoogle ScholarPubMed
Friedrich, M., & Friederici, A. D. (2008). Neurophysiological correlates of online word learning in 14-month-old infants. NeuroReport: For Rapid Communication of Neuroscience Research, 19(18), 1757–1761. https://doi.org/10.1097/WNR.0b013e328318f014CrossRefGoogle Scholar
Fuhrman, O., Eckerling, A., Friedmann, N., Tarrasch, R., & Raz, G. (2021). The moving learner: Object manipulation in virtual reality improves vocabulary learning. Journal of Computer Assisted Learning, 37(3), 672–683.CrossRefGoogle Scholar
Hirosh, Z., & Degani, T. (2018). Direct and indirect effects of multilingualism on novel language learning: An integrative review. Psychonomic Bulletin & Review, 25(3), 892–916.CrossRefGoogle ScholarPubMed
Ibáñez, M. B., García, J. J., Galán, S., Maroto, D., Morillo, D., & Kloos, C. D. (2011). Design and implementation of a 3D multi-user virtual world for language learning. Journal of Educational Technology & Society, 14(4), 2–10.Google Scholar
Jackson, J., & Schwieter, J. W. (2019). Study abroad and immersion. In Schwieter, J. W & Benati, A. (Eds.), The Cambridge handbook of language learning (pp. 727–750). Cambridge University Press.CrossRefGoogle Scholar
Jeong, H., Sugiura, M., Sassa, Y., Wakusawa, K., Horie, K., Sato, S., & Kawashima, R. (2010). Learning second language vocabulary: Neural dissociation of situation-based learning and text-based learning. Neuroimage, 50(2), 802–809.CrossRefGoogle ScholarPubMed
Jeong, H., Li, P., Suzuki, W., Sugiura, M., & Kawashima, R. (2021). Neural mechanisms of language learning from social contexts. Brain and Language, 212, 104874.CrossRefGoogle ScholarPubMed
Jiao, L., Zhang, Y., Plummer, P., Liu, C., & Chen, B. (2019). The influence of bilingual language experience on executive control: An ERPs study. Journal of Neurolinguistics, 51, 42–52.CrossRefGoogle Scholar
Jiao, L., Liu, C., Schwieter, J. W., & Chen, B. (2021). Switching between newly learned languages impacts executive control. Psychophysiology, 58(10), Article e13888. https://doi.org/10.1111/psyp.13888CrossRefGoogle ScholarPubMed
Jiao, L., Duan, X., Liu, C., & Chen, B. (2022). Comprehension-based language switching between newly learned languages: The role of individual differences. Journal of Neurolinguistics, 61, 101036.CrossRefGoogle Scholar
Johnson-Glenberg, M. C., Birchfield, D. A., Tolentino, L., & Koziupa, T. (2014). Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. Journal of Educational Psychology, 106(1), 86–104.CrossRefGoogle Scholar
Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 21(1), 60–99.CrossRefGoogle ScholarPubMed
Kapa, L. L., & Colombo, J. (2014). Executive function predicts artificial language learning. Journal of Memory and Language, 76, 237–252.CrossRefGoogle ScholarPubMed
Klassen, G., Ferreira, A., & Schwieter, J. W. (2021). The role of study abroad in the acquisition and processing of L2 gender agreement. Applied Linguistics Review [Ahead-of-Print], 1–23.Google Scholar
Kroll, J. F., & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between bilingual memory representations. Journal of Memory and Language, 33(2), 149–174.CrossRefGoogle Scholar
Kuhl, P. K., Tsao, F. M., & Liu, H. M. (2003). Foreign-language experience in infancy: Effects of short-term exposure and social interaction on phonetic learning. Proceedings of the National Academy of Sciences, 100(15), 9096–9101.CrossRefGoogle ScholarPubMed
Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62, 621–647.CrossRefGoogle ScholarPubMed
Lan, Y. J., Fang, S. Y., Legault, J., & Li, P. (2015). Second language acquisition of Mandarin Chinese vocabulary: Context of learning effects. Educational Technology Research and Development, 63(5), 671–690.CrossRefGoogle Scholar
Laufer, B., & Hulstijn, J. (2001). Incidental vocabulary acquisition in a second language: The construct of task-induced involvement. Applied Linguistics, 22(1), 1–26.CrossRefGoogle Scholar
Legault, J., Fang, S. Y., Lan, Y. J., & Li, P. (2019a). Structural brain changes as a function of second language vocabulary training: Effects of learning context. Brain and Cognition, 134, 90–102.CrossRefGoogle ScholarPubMed
Legault, J., Zhao, J., Chi, Y. A., Chen, W., Klippel, A., & Li, P. (2019b). Immersive virtual reality as an effective tool for second language vocabulary learning. Languages, 4(1), 1–32.CrossRefGoogle Scholar
Li, P., & Jeong, H. (2020). The social brain of language: Grounding second language learning in social interaction. npj Science of Learning, 5(1), 1–9.CrossRefGoogle ScholarPubMed
Li, P., & Lana, Y. J. (2021). Digital language learning (DLL): Insights from behavior, cognition, and the brain. Bilingualism: Language and Cognition, 1–18.Google Scholar
Linck, J. A., Kroll, J. F., & Sunderman, G. (2009). Losing access to the native language while immersed in a second language: Evidence for the role of inhibition in second-language learning. Psychological Science, 20(12), 1507–1515.CrossRefGoogle Scholar
Liu, Y., & van Hell, J. G. (2020). Learning novel word meanings: an ERP study on lexical consolidation in monolingual, inexperienced foreign language learners. Language Learning, 70(S2), 45–74.CrossRefGoogle Scholar
Liu, C., Wang, H., Timmer, K., & Jiao, L. (2022). The foreign language effect on altruistic decision making: Insights from the framing effect. Bilingualism: Language and Cognition, 25(5), 890–898.CrossRefGoogle Scholar
Makransky, G., & Petersen, G. B. (2021). The cognitive affective model of immersive learning (CAMIL): a theoretical research-based model of learning in immersive virtual reality. Educational Psychology Review, 33(3), 937–958.CrossRefGoogle Scholar
Makransky, G., Andreasen, N. K., Baceviciute, S., & Mayer, R. E. (2021). Immersive virtual reality increases liking but not learning with a science simulation and generative learning strategies promote learning in immersive virtual reality. Journal of Educational Psychology, 113(4), 719–735.CrossRefGoogle Scholar
Malt, B. C., Li, P., Pavlenko, A., Zhu, H., & Ameel, E. (2015). Bidirectional lexical interaction in late immersed Mandarin-English bilinguals. Journal of Memory and Language, 82, 86–104.CrossRefGoogle Scholar
Mathalon, D. H., Whitfield, S. L., & Ford, J. M. (2003). Anatomy of an error: ERP and fMRI. Biological Psychology, 64(1–2), 119–141.CrossRefGoogle ScholarPubMed
Mayer, K. M., Yildiz, I. B., Macedonia, M., & von Kriegstein, K. (2015). Visual and motor cortices differentially support the translation of foreign language words. Current Biology, 25(4), 530–535.CrossRefGoogle ScholarPubMed
Nair, K. K., Biedermann, B., & Nickels, L. (2016). Consequences of late bilingualism for novel word learning: Evidence from Tamil–English bilingual speakers. International Journal of Bilingualism, 20, 473–487. https://doi.org/10.1177/1367006914567005CrossRefGoogle Scholar
Papin, K., & Kaplan-Rakowski, R. (2022). A study of vocabulary learning using annotated 360° pictures. Computer Assisted Language Learning, 1–28.CrossRefGoogle Scholar
Peeters, D. (2018). A standardized set of 3-D objects for virtual reality research and applications. Behavior Research Methods, 50(3), 1047–1054.CrossRefGoogle ScholarPubMed
Snodgrass, J. G., & Vanderwart, M. (1980). A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6(2), 174–215.Google ScholarPubMed
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer.CrossRefGoogle Scholar
van den Brink, D., Brown, C. M., & Hagoort, P. (2001). Electrophysiological evidence for early contextual influences during spoken-word recognition: N200 versus N400 effects. Journal of Cognitive Neuroscience, 13(7), 967–985.CrossRefGoogle ScholarPubMed
van Veen, V., & Carter, C. S. (2002). The anterior cingulate as a conflict monitor: fMRI and ERP studies. Physiology & Behavior, 77(4–5), 477–482.CrossRefGoogle ScholarPubMed
Vogel, E. K., & Luck, S. J. (2000). The visual N1 component as an index of a discrimination process. Psychophysiology, 37(2), 190–203.CrossRefGoogle ScholarPubMed
von Kriegstein, K., & Giraud, A. L. (2006). Implicit multisensory associations influence voice recognition. PLoS biology, 4(10), e326. https://doi.org/10.1371/journal.pbio.0040326CrossRefGoogle ScholarPubMed
Zhang, Q., & Yang, Y. (2003). The determiners of picture naming latency. Acta Psychologica Sinica, 35(4), 447–454.Google Scholar
Zinszer, B. D., Malt, B. C., Ameel, E., & Li, P. (2014). Native-likeness in second language lexical categorization reflects individual language history and linguistic community norms. Frontiers in Psychology, 5, 1203.CrossRefGoogle ScholarPubMed