Hostname: page-component-8448b6f56d-qsmjn Total loading time: 0 Render date: 2024-04-16T12:21:10.179Z Has data issue: false hasContentIssue false
  • English
  • Français

An event-related potential study of cross-modal translation recognition in Chinese–English bilinguals: the role of cross-linguistic orthography and phonology

Published online by Cambridge University Press:  12 December 2022

Er-Hu Zhang ,
Jiaxin Li ,
Defeng Li ,
Yiqiang Chen ,
Xin-Dong Zhang ,
Xinyi Wang  and
Hong-Wen Cao
Er-Hu Zhang
Affiliation:
Research Center for Language, Cognition and Language Application, Chongqing University, Chongqing, China
Jiaxin Li
Affiliation:
Research Center for Language, Cognition and Language Application, Chongqing University, Chongqing, China
Defeng Li
Affiliation:
Centre for Studies of Translation, Interpreting and Cognition, University of Macau, Taipa, Macau SAR, China
Yiqiang Chen
Affiliation:
Centre for Studies of Translation, Interpreting and Cognition, University of Macau, Taipa, Macau SAR, China
Xin-Dong Zhang
Affiliation:
Research Center for Language, Cognition and Language Application, Chongqing University, Chongqing, China
Xinyi Wang
Affiliation:
Research Center for Language, Cognition and Language Application, Chongqing University, Chongqing, China
Hong-Wen Cao*
Affiliation:
Research Center for Language, Cognition and Language Application, Chongqing University, Chongqing, China Centre for Studies of Translation, Interpreting and Cognition, University of Macau, Taipa, Macau SAR, China
*
*Corresponding author. Email: caohwen@cqu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Extensive evidence has demonstrated that bilinguals non-selectively activate lexicons of both languages when reading or hearing words in one language. Here, we further investigated the electrophysiological roles of cross-linguistic orthography and phonology in the processing of L2 spoken words in unbalanced Chinese (L1)–English (L2) bilinguals in a cross-modal situation. Relative to unrelated control, the recognition of auditory L2 words showed behavioral interference effects when paired with orthographic or phonological neighbors of the correct translations of L2 words. Moreover, the lexical effects were also exhibited in the electrophysiological data, as reflected by marginally less positive late positive component (500–800 ms) amplitudes in the frontal region. Importantly, the orthographic rather than phonological translation neighbor condition elicited less negative N400 (300–500 ms) amplitudes in the parietal–occipital regions, suggesting that this orthographic translation neighbor condition facilitated the co-activation of spoken L2 words. Taken together, these findings indicate that cross-linguistic orthographic and phonological activation have different temporal dynamics with both bottom-up parallel cross-linguistic activation and the top-down inhibitory control mechanism governing the two-language lexical organization in L2 spoken word recognition.

Keywords

bilingualscross-modalorthography and phonologyL2 spoken word recognition
Type
Article
Information
Language and Cognition , Volume 15 , Issue 2 , June 2023 , pp. 292 - 313
Check for updates
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alvarez, J. A., & Emory, E. (2006). Executive function and the frontal lobes: A meta-analytic review. Neuropsychology Review, 16(1), 17–42. https://doi.org/10.1007/s11065-006-9002-xCrossRefGoogle ScholarPubMed
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/j.jml.2007.12.005CrossRefGoogle Scholar
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://EconPapers.repec.org/RePEc:jss:jstsof:v:067:i01CrossRefGoogle Scholar
Berghoff, R., McLoughlin, J., & Bylund, E. (2021). L1 activation during L2 processing is modulated by both age of acquisition and proficiency. Journal of Neurolinguistics, 58, 100979. https://doi.org/10.1016/j.jneuroling.2020.100979CrossRefGoogle Scholar
Bijeljac-Babic, R., Biardeau, A., & Grainger, J. (1997). Masked orthographic priming in bilingual word recognition. Memory & Cognition, 25(4), 447–457. https://doi.org/10.3758/bf03201121CrossRefGoogle ScholarPubMed
Blumenfeld, H. K., & Marian, V. (2013). Parallel language activation and cognitive control during spoken word recognition in bilinguals. Journal of Cognitive Psychology, 25(5), 547–567. https://doi.org/10.1080/20445911.2013.812093CrossRefGoogle ScholarPubMed
Brouwer, H., Crocker, M. W., Venhuizen, N. J., & Hoeks, J. C. J. (2017). A neurocomputational model of the N400 and the P600 in language processing. Cognitive Science, 41(Suppl 6), 1318–1352. https://doi.org/10.1111/cogs.12461CrossRefGoogle ScholarPubMed
Brouwer, H., Fitz, H., & Hoeks, J. (2012). Getting real about semantic illusions: Rethinking the functional role of the P600 in language comprehension. Brain Research, 1446, 127–143. https://doi.org/10.1016/j.brainres.2012.01.055CrossRefGoogle ScholarPubMed
Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977–990. https://doi.org/10.3758/BRM.41.4.977CrossRefGoogle Scholar
Brysbaert, M., Van Dyck, G., & Van de Poel, M. (1999). Visual word recognition in bilinguals: Evidence from masked phonological priming. Journal of Experimental Psychology: Human Perception and Performance, 25(1), 137–148. https://doi.org/10.1037//0096-1523.25.1.137Google ScholarPubMed
Cai, Q., & Brysbaert, M. (2010). SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. PLoS One, 5(6), e10729. https://doi.org/10.1371/journal.pone.0010729CrossRefGoogle ScholarPubMed
Carrasco-Ortiz, H., Amengual, M., & Gries, S. T. (2021). Cross-language effects of phonological and orthographic similarity in cognate word recognition. Linguistic Approaches to Bilingualism, 11(3), 389–417. https://doi.org/10.1075/lab.18095.carCrossRefGoogle Scholar
Chen, P., Bobb, S. C., Hoshino, N., & Marian, V. (2017). Neural signatures of language co-activation and control in bilingual spoken word comprehension. Brain Research, 1665, 50–64. https://doi.org/10.1016/j.brainres.2017.03.023CrossRefGoogle ScholarPubMed
de Groot, A. M. B. (1992). Determinants of word translation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(5), 1001–1018. https://doi.org/10.1037/0278-7393.18.5.1001Google Scholar
Diependaele, K., Ziegler, J. C., & Grainger, J. (2010). Fast phonology and the bimodal interactive activation model. European Journal of Cognitive Psychology, 22(5), 764–778. https://doi.org/10.1080/09541440902834782CrossRefGoogle Scholar
Dijkstra, T., & Van Heuven, W. J. B. (1998). The BIA model and bilingual word recognition. In Grainger, J., & Jacobs, A. M. (Eds.), Scientific psychology series. Localist connectionist approaches to human cognition (pp. 189–225). Lawrence Erlbaum Associates Publishers.Google Scholar
Dijkstra, T., & van Heuven, W. J. B. (2002). The architecture of the bilingual word recognition system: From identification to decision. Bilingualism: Language and Cognition, 5(3), 175–197. https://doi.org/10.1017/s1366728902003012CrossRefGoogle Scholar
Dimitropoulou, M., Duñabeitia, J. A., & Carreiras, M. (2011). Phonology by itself: Masked phonological priming effects with and without orthographic overlap. Journal of Cognitive Psychology, 23(2), 185–203. https://doi.org/10.1080/20445911.2011.477811CrossRefGoogle Scholar
Duyck, W., Diependaele, K., Drieghe, D., & Brysbaert, M. (2004). The size of the cross-lingual masked phonological priming effect does not depend on second language proficiency. Experimental Psychology, 51(2), 116–124. https://doi.org/10.1027/1618-3169.51.2.116CrossRefGoogle Scholar
Emmorey, K., Giezen, M. R., & Gollan, T. H. (2016). Psycholinguistic, cognitive, and neural implications of bimodal bilingualism. Bilingualism: Language and Cognition, 19(2), 223–242. https://doi.org/10.1017/S1366728915000085CrossRefGoogle ScholarPubMed
Emmorey, K., Luk, G., Pyers, J. E., & Bialystok, E. (2008). The source of enhanced cognitive control in bilinguals: Evidence from bimodal bilinguals. Psychological Science, 19(12), 1201–1206. https://doi.org/10.1111/j.1467-9280.2008.02224.xCrossRefGoogle ScholarPubMed
Escudero, P., & Wanrooij, K. (2010). The effect of L1 orthography on non-native vowel perception. Language & Speech, 53(Pt 3), 343–365. https://doi.org/10.1177/0023830910371447CrossRefGoogle ScholarPubMed
Frances, C., Navarra-Barindelli, E., & Martin, C. D. (2021). Inhibitory and facilitatory effects of phonological and orthographic similarity on L2 word recognition across modalities in bilinguals. Scientific Report, 11(1), 12812. https://doi.org/10.1038/s41598-021-92259-zCrossRefGoogle ScholarPubMed
Giezen, M. R., Blumenfeld, H. K., Shook, A., Marian, V., & Emmorey, K. (2015). Parallel language activation and inhibitory control in bimodal bilinguals. Cognition, 141, 9–25. https://doi.org/10.1016/j.cognition.2015.04.009CrossRefGoogle ScholarPubMed
Giezen, M. R., & Emmorey, K. (2016). Language co-activation and lexical selection in bimodal bilinguals: Evidence from picture-word interference. Bilingualism: Language and Cognition, 19(2), 264–276. https://doi.org/10.1017/S1366728915000097CrossRefGoogle ScholarPubMed
Green, D. (1998). Mental control of the bilingual lexico-semantic system. Bilingualism: Language and Cognition, 1(2), 67–81. https://doi.org/10.1017/S1366728998000133CrossRefGoogle Scholar
Greenhouse, S. W., & Geisser, S. (1959). On methods in the analysis of profile data. Psychometrika, 24(2), 95–112. https://doi.org/10.1007/BF02289823CrossRefGoogle Scholar
Guo, T., Misra, M., Tam, J. W., & Kroll, J. F. (2012). On the time course of accessing meaning in a second language: An electrophysiological and behavioral investigation of translation recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(5), 1165–1186. https://doi.org/10.1037/a0028076Google Scholar
Holcomb, P. J., Grainger, J., & O’Rourke, T. (2002). An electrophysiological study of the effects of orthographic neighborhood size on printed word perception. Journal of Cognitive Neuroscience, 14(6), 938–950. https://doi.org/10.1162/089892902760191153CrossRefGoogle ScholarPubMed
Hoshino, N., & Thierry, G. (2012). Do Spanish-English bilinguals have their fingers in two pies – or is it their toes? An electrophysiological investigation of semantic access in bilinguals. Frontiers in Psychology, 3, 9. https://doi.org/10.3389/fpsyg.2012.00009CrossRefGoogle ScholarPubMed
Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434–446. https://doi.org/10.1016/j.jml.2007.11.007CrossRefGoogle ScholarPubMed
Jankowiak, K., & Rataj, K. (2017). The N400 as a window into lexico-semantic processing in bilingualism. Poznan Studies in Contemporary Linguistics, 53, 119–156. https://doi.org/10.1515/psicl-2017-0006CrossRefGoogle Scholar
Jiang, N. (1999). Testing processing explanations for the asymmetry in masked cross-language priming. Bilingualism: Language and Cognition, 2, 59–75. https://doi.org/10.1017/S1366728999000152CrossRefGoogle Scholar
Jouravlev, O., Lupker, S. J., & Jared, D. (2014). Cross-language phonological activation: Evidence from masked onset priming and ERPs. Brain and Language, 134, 11–22. https://doi.org/10.1016/j.bandl.2014.04.003CrossRefGoogle ScholarPubMed
Jung, T. P., Makeig, S., Humphries, C., Lee, T. W., McKeown, M. J., Iragui, V., & Sejnowski, T. J. (2000). Removing electroencephalographic artifacts by blind source separation. Psychophysiology, 37(2), 163–178. https://doi.org/10.1111/1469-8986.3720163CrossRefGoogle ScholarPubMed
Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62, 621–647. https://doi.org/10.1146/annurev.psych.093008.131123CrossRefGoogle ScholarPubMed
Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307(5947), 161–163. https://doi.org/10.1038/307161a0CrossRefGoogle ScholarPubMed
Lemhofer, K., & Broersma, M. (2012). Introducing LexTALE: A quick and valid Lexical Test for Advanced Learners of English. Behavior Research Methods, 44(2), 325–343. https://doi.org/10.3758/s13428-011-0146-0CrossRefGoogle ScholarPubMed
Ma, F., & Ai, H. (2018). Chinese learners of English see Chinese words when reading English words. Journal of Psycholinguist Research, 47(3), 505–521. https://doi.org/10.1007/s10936-017-9533-8CrossRefGoogle ScholarPubMed
Ma, F., Chen, P., Guo, T., & Kroll, J. F. (2017). When late second language learners access the meaning of L2 words: Using ERPs to investigate the role of the L1 translation equivalent. Journal of Neurolinguistics, 41, 50–69. https://doi.org/10.1016/j.jneuroling.2016.09.006CrossRefGoogle Scholar
Marian, V., Bartolotti, J., Daniel, N. L., & Hayakawa, S. (2021). Spoken words activate native and non-native letter-to-sound mappings: Evidence from eye tracking. Brain and Language, 223, 105045. https://doi.org/10.1016/j.bandl.2021.105045CrossRefGoogle ScholarPubMed
Marian, V., Blumenfeld, H. K., & Boukrina, O. V. (2008). Sensitivity to phonological similarity within and across languages. Journal of Psycholinguistic Research, 37(3), 141–170. https://doi.org/10.1007/s10936-007-9064-9CrossRefGoogle ScholarPubMed
Marian, V., & Spivey, M. (2003). Competing activation in bilingual language processing: Within- and between-language competition. Bilingualism: Language and Cognition, 6(2), 97–115. https://doi.org/10.1017/S1366728903001068CrossRefGoogle Scholar
McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychological Review, 88(5), 375–407. https://doi.org/10.1037/0033-295X.88.5.375CrossRefGoogle Scholar
Meade, G., Midgley, K. J., Sevcikova Sehyr, Z., Holcomb, P. J., & Emmorey, K. (2017). Implicit co-activation of American Sign Language in deaf readers: An ERP study. Brain and Language, 170, 50–61. https://doi.org/10.1016/j.bandl.2017.03.004CrossRefGoogle ScholarPubMed
Midgley, K. J., Holcomb, P. J., & Grainger, J. (2011). Effects of cognate status on word comprehension in second language learners: An ERP investigation. Journal of Cognitive Neuroscience, 23(7), 1634–1647. https://doi.org/10.1162/jocn.2010.21463CrossRefGoogle ScholarPubMed
Midgley, K. J., Holcomb, P. J., vanHeuven, W. J. B., & Grainger, J. (2008). An electrophysiological investigation of cross-language effects of orthographic neighborhood. Brain Research, 1246, 123–135. https://doi.org/10.1016/j.brainres.2008.09.078CrossRefGoogle ScholarPubMed
Mishra, R. K., & Singh, N. (2014). Language non-selective activation of orthography during spoken word processing in Hindi–English sequential bilinguals: An eye tracking visual world study. Reading and Writing, 27(1), 129–151. https://doi.org/10.1007/s11145-013-9436-5CrossRefGoogle Scholar
Mishra, R. K., & Singh, N. (2016). The influence of second language proficiency on bilingual parallel language activation in Hindi–English bilinguals. Journal of Cognitive Psychology, 28(4), 396–411. https://doi.org/10.1080/20445911.2016.1146725CrossRefGoogle Scholar
Moldovan, C. D., Demestre, J., Ferré, P., & Sánchez-Casas, R. (2016). The role of meaning and form similarity in translation recognition in highly proficient balanced bilinguals: A behavioral and ERP study. Journal of Neurolinguistics, 37, 1–11. https://doi.org/10.1016/j.jneuroling.2015.07.002CrossRefGoogle Scholar
Müller, O., Duñabeitia, J. A., & Carreiras, M. (2010). Orthographic and associative neighborhood density effects: What is shared, what is different? Psychophysiology, 47(3), 455–466. https://doi.org/10.1111/j.1469-8986.2009.00960.xCrossRefGoogle ScholarPubMed
Peeters, D., Dijkstra, T., & Grainger, J. (2013). The representation and processing of identical cognates by late bilinguals: RT and ERP effects. Journal of Memory and Language, 68(4), 315–332. https://doi.org/10.1016/j.jml.2012.12.003CrossRefGoogle Scholar
Perre, L., & Ziegler, J. C. (2008). On-line activation of orthography in spoken word recognition. Brain Research, 1188, 132–138. https://doi.org/10.1016/j.brainres.2007.10.084CrossRefGoogle ScholarPubMed
Qu, Q., Cui, Z., & Damian, M. F. (2018). Orthographic effects in second-language spoken-word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(8), 1325–1332. https://doi.org/10.1037/xlm0000520Google ScholarPubMed
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.orgGoogle Scholar
Salverda, A. P., & Tanenhaus, M. K. (2010). Tracking the time course of orthographic information in spoken-word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(5), 1108. https://doi.org/10.1037/a0019901Google ScholarPubMed
Sauval, K., Perre, L., Duncan, L. G., Marinus, E., & Casalis, S. (2017). Automatic phonological activation during visual word recognition in bilingual children: A cross-language masked priming study in grades 3 and 5. Journal of Experimental Child Psychology, 154, 64–77. https://doi.org/10.1016/j.jecp.2016.10.003CrossRefGoogle ScholarPubMed
Shook, A., & Marian, V. (2012). Bimodal bilinguals co-activate both languages during spoken comprehension. Cognition, 124(3), 314–324. https://doi.org/10.1016/j.cognition.2012.05.014CrossRefGoogle ScholarPubMed
Shook, A., & Marian, V. (2013). The bilingual language interaction network for comprehension of speech. Bilingualism: Language and Cognition, 16(2), 304–324. https://doi.org/10.1017/s1366728912000466CrossRefGoogle Scholar
Shook, A., & Marian, V. (2019). Covert co-activation of bilinguals’ non-target language: Phonological competition from translations. Linguistic Approaches to Bilingualism, 9(2), 228–252. https://doi.org/10.1075/lab.17022.shoCrossRefGoogle ScholarPubMed
Stites, M. C., Federmeier, K. D., & Christianson, K. (2016). Do morphemes matter when reading compound words with transposed letters? Evidence from eye-tracking and event-related potentials. Language, Cognition and Neuroscience, 31(10), 1299–1319. https://doi.org/10.1080/23273798.2016.1212082CrossRefGoogle ScholarPubMed
Sunderman, G. L., & Priya, K. (2012). Translation recognition in highly proficient Hindi–English bilinguals: The influence of different scripts but connectable phonologies. Language and Cognitive Processes, 27(9), 1265–1285. https://doi.org/10.1080/01690965.2011.596420CrossRefGoogle Scholar
Thierry, G., & Wu, Y. J. (2007). Brain potentials reveal unconscious translation during foreign-language comprehension. Proceedings of the National Academy of Sciences of the United States of America, 104(30), 12530–12535. https://doi.org/10.1038/nrn2241CrossRefGoogle ScholarPubMed
van de Meerendonk, N., Indefrey, P., Chwilla, D. J., & Kolk, H. H. (2011). Monitoring in language perception: Electrophysiological and hemodynamic responses to spelling violations. Neuroimage, 54(3), 2350–2363. https://doi.org/10.1016/j.neuroimage.2010.10.022CrossRefGoogle ScholarPubMed
van Heuven, W., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effects in bilingual word recognition. Journal of Memory and Language, 39, 458–483. https://doi.org/10.1006/jmla.1998.2584CrossRefGoogle Scholar
Veivo, O., Järvikivi, J., Porretta, V., & Hyönä, J. (2016). Orthographic activation in L2 spoken word recognition depends on proficiency: Evidence from eye-tracking. Frontiers in Psychology, 7, 1120. https://doi.org/10.3389/fpsyg.2016.01120CrossRefGoogle ScholarPubMed
Veivo, O., Porretta, V., HyÖNÄ, J., & JÄRvikivi, J. (2018). Spoken second language words activate native language orthographic information in late second language learners. Applied Psycholinguistics, 39(5), 1011–1032. https://doi.org/10.1017/s0142716418000103CrossRefGoogle Scholar
Villameriel, S., Dias, P., Costello, B., & Carreiras, M. (2016). Cross-language and cross-modal activation in hearing bimodal bilinguals. Journal of Memory and Language, 87, 59–70. https://doi.org/10.1016/j.jml.2015.11.005CrossRefGoogle Scholar
Weber, A., & Cutler, A. (2004). Lexical competition in non-native spoken-word recognition. Journal of Memory and Language, 50(1), 1–25. https://doi.org/10.1016/S0749-596X(03)00105-0CrossRefGoogle Scholar
Wen, Y., Filik, R., & van Heuven, W. J. B. (2018). Electrophysiological dynamics of Chinese phonology during visual word recognition in Chinese–English bilinguals. Scientific Reports, 8(1), 6869. https://doi.org/10.1038/s41598-018-25072-wCrossRefGoogle ScholarPubMed
Wen, Y., & van Heuven, W. J. B. (2018). Limitations of translation activation in masked priming: Behavioural evidence from Chinese–English bilinguals and computational modelling. Journal of Memory and Language, 101, 84–96. https://doi.org/10.1016/j.jml.2018.03.004CrossRefGoogle Scholar
Winter, B. (2019). Statistics for linguists: An introduction using R. Routledge.CrossRefGoogle Scholar
Wu, C. Y., Ho, M. H., & Chen, S. H. (2012). A meta-analysis of fMRI studies on Chinese orthographic, phonological, and semantic processing. Neuroimage, 63(1), 381–391. https://doi.org/10.1016/j.neuroimage.2012.06.047CrossRefGoogle ScholarPubMed
Wu, Y. J., & Thierry, G. (2010). Chinese–English bilinguals reading English hear Chinese. Journal of Neuroscience, 30(22), 7646–7651. https://doi.org/10.1523/jneurosci.1602-10.2010CrossRefGoogle ScholarPubMed
Xia, V., & Andrews, S. (2015). Masked translation priming asymmetry in Chinese–English bilinguals: Making sense of the Sense Model. The Quarterly Journal of Experimental Psychology, 68(2), 294–325. https://doi.org/10.1080/17470218.2014.944195CrossRefGoogle ScholarPubMed
Zhang, T., van Heuven, W. J. B., & Conklin, K. (2011). Fast automatic translation and morphological decomposition in Chinese–English bilinguals. Psychological Science, 22(10), 1237–1242. https://doi.org/10.1177/0956797611421492CrossRefGoogle ScholarPubMed
Zhou, H., Chen, B., Yang, M., & Dunlap, S. (2010). Language nonselective access to phonological representations: Evidence from Chinese–English bilinguals. The Quarterly Journal of Experimental Psychology, 63(10), 2051–2066. https://doi.org/10.1080/17470211003718705CrossRefGoogle ScholarPubMed