Hostname: page-component-76fb5796d-r6qrq Total loading time: 0 Render date: 2024-04-25T11:41:58.271Z Has data issue: false hasContentIssue false

A META-ANALYSIS OF SENSITIVITY TO GRAMMATICAL INFORMATION DURING SELF-PACED READING: TOWARDS A FRAMEWORK OF REFERENCE FOR READING TIME EFFECT SIZES

Published online by Cambridge University Press:  07 May 2019

Nick Avery*
Affiliation:
University of York
Emma Marsden
Affiliation:
University of York
*
*Correspondence concerning this article should be addressed to Nicholas Avery, Department of Education, University of York, Heslington, York, United Kingdom, YO10 5DD. E-mail: nicholas.avery@york.ac.uk

Abstract

Despite extensive theoretical and empirical research, we do not have estimations of the magnitude of sensitivity to grammatical information during L2 online processing. This is largely due to reliance on null hypothesis significance testing (Plonsky, 2015). The current meta-analysis draws on data from one elicitation technique, self-paced reading, across 57 studies (N = 3,052), to estimate sensitivity to L2 morphosyntax and how far L1 background moderates this. Overall, we found a reliable sensitivity to L2 morphosyntax at advanced proficiencies (d = .20, 95% CIs .15, .25), with some evidence that this was reliably lower than for native speakers (NSs). These patterns were not generally moderated by linguistic feature or sentence region. However, effects for anomaly detection were larger among NSs than L2 learners and the effects among L2 learners appeared to show a trend toward L1 influence. Finding smaller effects than in other subdomains, we provide an initial framework of reference for L2 reading time effect sizes.

Type
Research Article
Copyright
Copyright © Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Aspects of this article were presented at the 2017 Second Language Research Forum, Ohio State University, and at the 2nd Biennial Conference of the French Second Language Acquisition Network: Research Methodology in the Field of Second Language Acquisition and Learning, Université Paul-Valéry, Montpellier 3, 2018. We are grateful to the editors and three anonymous SSLA reviewers for their valuable comments. We thank Luke Plonsky for useful statistical advice and the authors who shared their data with us: Filiz Çele, Masahiro Hara, Alan Juffs (NSF grant SBR-9709152), Sehoon Jung, and Natasha Tokowicz.

The experiment in this article earned an Open Data badge for transparent practices. The materials are available at https://www.iris-database.org/iris/app/home/detail?id=york%3a936220&ref=search/.

References

REFERENCES

Adesope, O., Lavin, T., Thompson, T., & Ungerleider, C. (2010). A systematic review and meta-analysis of the cognitive correlates of bilingualism. Review of Educational Research, 80, 207245.CrossRefGoogle Scholar
Aldawayan, S., Fiorentino, R., & Gabriele, A. (2010). Evidence of syntactic constraints in the processing of wh-movement: A study of Nadji Arabic learners of English. In VanPatten, B. & Jegerski, J. (Eds.), Research in second language processing and parsing (pp. 6586). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.Google Scholar
Bel, A., Sagarra, N., Comínguez, J., & García-Alcaraz, E. (2016). Transfer and proficiency effects in L2 processing of subject anaphora. Lingua, 184, 134159.CrossRefGoogle Scholar
Bley-Vroman, R. (1990). The logical problem of foreign language learning. Linguistic Analysis, 20, 349.Google Scholar
Bley-Vroman, R. (2009). The evolving context of the fundamental difference hypothesis. Studies in Second Language Acquisition, 31, 175198.CrossRefGoogle Scholar
Borenstein, M., Hedges, L., & Rothstein, H. (2007). Meta-analysis: Fixed effect vs. random effects. Retrieved from www.meta-analysis.com.Google Scholar
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.CrossRefGoogle Scholar
Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1, 120.CrossRefGoogle ScholarPubMed
Bultena, S., Dijkstra, T., & van Hell, J. (2014). Cognate effects in sentence context depend on word class, L2 proficiency, and task. The Quarterly Journal of Experimental Psychology, 67, 12141241.CrossRefGoogle ScholarPubMed
Chan, H. (2012). Tense-aspect processing in second language learners (Unpublished doctoral dissertation). University of Pittsburgh.Google Scholar
Clahsen, H., & Felser, C. (2006a). Grammatical processing in language learners. Applied Psycholinguistics, 27, 342.CrossRefGoogle Scholar
Clahsen, H., & Felser, C. (2006b). How native-like is non-native language processing? Trends in Cognitive Sciences, 10, 564570.CrossRefGoogle Scholar
Clahsen, H., & Felser, C. (2018). Some notes on the shallow structure hypothesis. Studies in Second Language Acquisition, 40, 693706.CrossRefGoogle Scholar
Clahsen, H., Felser, C., Neubauer, C., Sato, M., & Silva, R. (2010). Morphological structure in native and nonnative language processing. Language Learning, 60, 2143CrossRefGoogle Scholar
Coe, R. (2002). It’s the effect size, stupid: What the effect size is and why it is important. Paper presented at the British Educational Research Association annual conference, Exeter, 12–14 September 2002.Google Scholar
Cohen, J. (1969). Statistical power for the behavioural sciences (2nd ed.). Hove, UK: Laurence Erlbaum.Google Scholar
Cumming, G. (2009). Inference by eye: Reading the overlap of independent confidence intervals. Statistics in Medicine, 28, 205220.CrossRefGoogle ScholarPubMed
Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: RoutledgeGoogle Scholar
Dekydtspotter, L., Edmonds, A., Fultz, A., & Renaud, C. (2010). Modularity of L2 sentence processing: Prosody, context and morphology in relative clause ambiguity in English-French interlanguage. In Iverson, M., Ivanov, I., Judy, T., Rothman, J., Slabakova, R., & Tryzna, M. (Eds.), Proceedings of the 2009 mind/context divide workshop (pp. 1327). Somerville, MA: Cascadilla Proceedings Project.Google Scholar
Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology, 5, 781.CrossRefGoogle ScholarPubMed
Dussias, P. (2003). Syntactic ambiguity resolution in L2 learners: Some effects of bilinguality on L1 and L2 processing strategies. Studies in Second Language Acquisition, 25, 529557.CrossRefGoogle Scholar
Ellis, N. (2006). Meta-analysis, human cognition and language learning. In Norris, J. & Ortega, L. (Eds.), Synthesizing research on language learning and teaching (pp. 301322). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
Felser, C., Roberts, L., Marinis, T., & Gross, R. (2003). The processing of ambiguous sentences by first and second language learners of English. Applied Psycholinguistics, 24, 453489.CrossRefGoogle Scholar
Fender, M. (2003). English word recognition and word integration skills of native Arabic and Japanese-speaking learners of English as a second language. Applied Psycholinguistics, 24, 289315.CrossRefGoogle Scholar
Foote, R. (2011). Integrated knowledge of agreement in early and late English-Spanish bilinguals. Applied Psycholinguistics, 32, 187220.CrossRefGoogle Scholar
Gerth, S., Otto, C., Felser, C., & Nam, Y. (2017). Strength of garden path effects in native and non-native speakers’ processing of subject-object ambiguities. International Journal of Bilingualism, 21, 125144.CrossRefGoogle Scholar
Greenland, S., Senn, S., Rothman, K., Carlin, J., Poole, C., Goodman, S., & Altmann, D. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31, 337350.CrossRefGoogle ScholarPubMed
Havik, E., Roberts, L., van Hout, R., Schreuder, R., & Haverkort, M. (2009). Processing subject-object ambiguities in the L2: A self-paced reading study with German L2 learners of Dutch. Language Learning, 59, 73112.CrossRefGoogle Scholar
Hedge, C., Powell, G., Bompas, A., Vivian-Griffiths, S., & Sumner, P. (2018). Low and variable correlation between reaction time costs and accuracy costs explained by two accumulation models: Meta-analysis and simulations. Psychological Bulletin, 144, 12001227.CrossRefGoogle Scholar
Hopp, H. (2006). Syntactic features and re-analysis in near-native processing. Second Language Research, 22, 369397.CrossRefGoogle Scholar
Hopp, H. (2009). The syntax-discourse interface in near-native L2 acquisition: Off-line and on-line performance. Bilingualism: Language and Cognition, 12, 463483.CrossRefGoogle Scholar
Hopp, H. (2010). Ultimate attainment in L2 inflection: Performance similarities between non-native and native speakers. Lingua, 120, 901931.CrossRefGoogle Scholar
Howell, D. (2013). Statistical methods for psychology (8th ed.). Belmont, MA: Wadsworth Cengage Learning.Google Scholar
Hulstijn, J. (2019). An individual differences framework for comparing non-native with native speakers: Perspectives from basic language cognition theory. Language Learning, 69, 157183.CrossRefGoogle Scholar
Jackson, C. (2008). Proficiency level and the interaction of lexical and morphosyntactic information during L2 sentence processing. Language Learning, 58, 875909.CrossRefGoogle Scholar
Jackson, C. (2010). The processing of subject-object ambiguities by English and Dutch L2 learners of German. In VanPatten, B. & Jegerski, J. (Eds.), Research in second language processing and parsing (pp. 207230). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
Jackson, C., & Dussias, P. (2009). Cross-linguistic differences and their impact on L2 sentence processing. Bilingualism: Language and Cognition, 12, 6582.CrossRefGoogle Scholar
Jackson, C., & van Hell, J. (2011). The effects of L2 proficiency level on the processing of wh-questions among Dutch second language speakers of English. International Review of Applied Linguistics, 49, 195219.CrossRefGoogle ScholarPubMed
Jarvis, S., & Pavlenko, A. (2008). Cross-linguistic influence in language and cognition. London, UK: Routledge.CrossRefGoogle Scholar
Jegerski, J. (2012). The processing of subject-object ambiguities in native and near-native Mexican Spanish. Bilingualism: Language and Cognition, 15, 721735.CrossRefGoogle Scholar
Jegerski, J. (2014). Self-paced reading. In Jegerski, J. & VanPatten, B. (Eds.), Research methods in second language psycholinguistics (pp. 2049). New York, NY: Routledge.Google Scholar
Jegerski, J. (2016). Number attraction effects in near-native Spanish comprehension. Studies in Second Language Acquisition, 38, 533.CrossRefGoogle Scholar
Jegerski, J. (2018). Sentence processing in Spanish as a heritage language: A self-paced reading study of relative clause attachment. Language Learning, 68, 598634.CrossRefGoogle Scholar
Jiang, N. (2004). Morphological insensitivity in second language processing. Applied Psycholinguistics, 25, 603634.CrossRefGoogle Scholar
Jiang, N. (2007). Selective integration of linguistic knowledge in adult second language learning. Language Learning, 57, 133.CrossRefGoogle Scholar
Jiang, N., Novokshanova, E., Masuda, K., & Wang, X. (2011). Morphological congruency and the acquisition of L2 morphemes. Language Learning, 61, 940967.CrossRefGoogle Scholar
Juffs, A. (1998a). Main verb versus reduced relative clause ambiguity resolution in L2 sentence processing. Language Learning, 48, 107147.CrossRefGoogle Scholar
Juffs, A. (1998b). Some effects of first language argument structure and morphosyntax on second language sentence processing. Second Language Research, 14, 406424.CrossRefGoogle Scholar
Juffs, A. (2005). The influence of first language on the processing of wh-movement in English as a second language. Second Language Research, 21, 121151.CrossRefGoogle Scholar
Juffs, A. (2006). Processing reduced relative vs. main verb ambiguity in English as a Second Language: A replication study with working memory. In Slabakova, R., Montrul, S., & Prevost, P. (Eds.), Inquiries in linguistic development in honor of Lydia White (pp. 213232). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
Juffs, A., & Harrington, M. (1995). Parsing effects in second language sentence processing: Subject and object asymmetries in wh-extraction. Studies in Second Language Acquisition, 17, 483516.CrossRefGoogle Scholar
Juffs, A., & Rodriguez, G. (2015). Second language sentence processing. New York, NY: Routledge.Google Scholar
Jung, S. (2010). Second language processing of wh-movement in English: The effects of first language and learning environment (Unpublished MA dissertation). Michigan State University.Google Scholar
Just, M., Carpenter, P., & Woolley, J. (1982). Paradigms and processes in reading comprehension. Journal of Experimental Psychology: General, 111, 228238.CrossRefGoogle ScholarPubMed
Kaan, E., Ballantyne, J., & Wijnen, F. (2015). Effects of reading speed on second-language sentence processing. Applied Psycholinguistics, 36, 799830.CrossRefGoogle Scholar
Keating, G., & Jegerski, J. (2015). Experimental designs in sentence processing research: A methodological review and user’s guide. Studies in Second Language Acquisition, 37, 132.CrossRefGoogle Scholar
Keating, G., Jegerski, J., & VanPatten, B. (2016). Online processing of subject pronouns in monolingual and heritage bilingual speakers of Mexican Spanish. Bilingualism: Language and Cognition, 19, 3649.CrossRefGoogle Scholar
Lakshmanan, U., Kim, K., McCreary, R., Park, K., Suen, U., & Lee, S. (2009). L2 learners’ sensitivity to strong and weak subjacency-violations in online processing. In Bowles, M., Ionin, T., Montrul, S., & Tremblay, A. (Eds.), Proceedings of the 10th Generative Approaches to Second Language Acquisition Conference (GASLA 2009) (pp. 136143). Somerville, MA: Cascadilla Proceedings Project.Google Scholar
Larson-Hall, J. (2016). A guide to doing statistics in second language research using SPSS and R. New York, NY: Routledge.Google Scholar
Lauro, J., & Schwartz, A. (2017). Bilingual non-selective lexical access in sentence contexts: A meta-analytic review. Journal of Memory and Language, 92, 217233.CrossRefGoogle Scholar
Lee, E., Lu, D., & Garnsey, S. (2013). L1 word order and sensitivity to verb bias in L2 processing. Bilingualism: Language and Cognition, 16, 761775.CrossRefGoogle Scholar
Lee, J., Jang, J., & Plonsky, L. (2015). The effectiveness of second language pronunciation instruction: A meta-analysis. Applied Linguistics, 36, 345366.CrossRefGoogle Scholar
Lee, S., & Huang, H. (2008). Visual input enhancement and grammar learning: A meta-analytic review. Studies in Second Language Acquisition, 30, 307331.CrossRefGoogle Scholar
Lipsey, M., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.Google Scholar
MacWhinney, B. (2005). Extending the competition model. International Journal of Bilingualism, 9, 6984.CrossRefGoogle Scholar
Marinis, T. (2007). On-line processing of passives in L1 and L2 children. In Belikova, A., Meroni, L., & Umeda, M. (Eds.), Proceedings of the 2nd Conference on Generative Approaches to Language Acquisition North America (GALANA) (pp. 265276). Somerville, MA: Cascadilla Proceedings Project.Google Scholar
Marinis, T. (2010). Using on-line processing methods in language acquisition research. In Blunsworth, S. & Blom, E. (Eds.), Experimental methods in language acquisition research (pp. 139162). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
Marinis, T., Roberts, L., Felser, C., & Clahsen, H. (2005). Gaps in second language processing. Studies in Second Language Acquisition, 27, 5378.CrossRefGoogle Scholar
Marsden, E., Thompson, S., & Plonsky, L. (2018). A methodological synthesis of self-paced reading in second language research. Applied Psycholinguistics, 39, 861904.CrossRefGoogle Scholar
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68, 321–291.CrossRefGoogle Scholar
Morgan-Short, K., Marsden, E., Heil, J., Issa, B. II, Leow, R., Mikhaylova, A., Mikołajczak, S., Moreno, N., Slabakova, R., & Szudarski, P. (2018). Multi-site replication in second language acquisition research: Attention to form during listening and reading comprehension. Language Learning, 68, 392437.CrossRefGoogle Scholar
Morris, S., & DeShon, R. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105125.CrossRefGoogle ScholarPubMed
Mueller, J., & Jiang, N. (2013). The acquisition of the Korean honorific affix (u)si by advanced L2 learners. The Modern Language Journal, 97, 318339.CrossRefGoogle Scholar
Norouzian, R., de Miranda, M., & Plonsky, L. (2018). The Bayesian revolution in second language research. Language Learning, 68, 10321075.CrossRefGoogle Scholar
Norris, J., & Ortega, L. (Eds.), (2006). Synthesizing research on language learning and teaching. Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
Odlin, T. (2005). Cross-linguistic influence. In Doughty, C., & Long, M. (Eds.), The handbook of second language acquisition (pp. 436486). Oxford, UK: Blackwell.Google Scholar
Odlin, T. (2012). Cross-linguistic influence in second language acquisition. In Chappelle, C. (Ed.), The encyclopaedia of applied linguistics (pp. 15621568). Malden, MA: Blackwell.Google Scholar
O’Grady, W. (2005). Syntactic carpentry: An emergentist approach to syntax. Mahwah, NJ: Lawrence ErlbaumCrossRefGoogle Scholar
Oswald, F., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85110.CrossRefGoogle Scholar
Pan, H., Schimke, S., & Felser, C. (2015). Referential context effects in non-native relative clause ambiguity resolution. International Journal of Bilingualism, 19, 298313.CrossRefGoogle Scholar
Papadopoulou, D. (2005). Reading time studies of second language ambiguity resolution. Second Language Research, 21, 98120.CrossRefGoogle Scholar
Papadopoulou, D., & Clahsen, H. (2003). Parsing strategies in L1 and L2 sentence processing: A study of relative clause attachment in Greek. Studies in Second Language Acquisition, 25, 501528.CrossRefGoogle Scholar
Perpiñán, S. (2014). L2 grammar and L2 processing in the acquisition of Spanish prepositional relative clauses. Bilingualism: Language and Cognition, 18, 577596.CrossRefGoogle Scholar
Plonsky, L. (2011). The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 61, 9931038.CrossRefGoogle Scholar
Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655687.CrossRefGoogle Scholar
Plonsky, L. (Ed.), (2015). Advancing quantitative methods in second language research. New York, NY: Routledge.CrossRefGoogle Scholar
Plonsky, L., & Brown, D. (2015). Domain definition and search techniques in meta-analyses of L2 research (Or why 18 meta-analyses of feedback have different results). Second Language Research, 31, 267278.CrossRefGoogle Scholar
Plonsky, L., & Oswald, F. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878911.CrossRefGoogle Scholar
Rah, A., & Adone, D. (2010). Processing of the reduced relative clause versus main verb ambiguity in L2 learners at different proficiency levels. Studies in Second Language Acquisition, 32, 79109.CrossRefGoogle Scholar
Rayner, K., Schotter, E., Masson, M., Potter, M., & Treiman, R. (2016). So much to read, so little time: How do we read, and can speed reading help? Psychological Science in the Public Interest, 17, 434.CrossRefGoogle ScholarPubMed
Renaud, C. (2014). A processing investigation of the accessibility of the uninterpretable gender feature in L2 French and L2 Spanish adjective agreement. Linguistic Approaches to Bilingualism, 4, 222255.CrossRefGoogle Scholar
Roberts, L. (2007). Investigating real-time sentence processing in the second language. Stem-, Spraak- en Taalpathologie, 15, 115127.Google Scholar
Roberts, L. (2012). Psycholinguistic techniques and resources in second language acquisition research. Second Language Research, 28, 113127.CrossRefGoogle Scholar
Roberts, L. (2013). Sentence processing in bilinguals. In van Gompel, R. (Ed.), Sentence processing (pp. 221246). Hove, UK: Psychology Press.Google Scholar
Roberts, L. (2016). Self-paced reading and L2 grammatical processing. In Mackey, A. & Marsden, E. (Eds.), Advancing methodology and practice: The IRIS repository for instruments for research into second languages (pp. 5872). New York, NY: Routledge.Google Scholar
Roberts, L., & Felser, C. (2011). Plausibility and recovery from garden paths in second language sentence processing. Applied Psycholinguistics, 32, 299331.CrossRefGoogle Scholar
Roberts, L., & Liszka, S. (2013). Processing tense/aspect-agreement violations on-line in the second language: A self-paced reading study with French and German L2 learners of English. Second Language Research, 29, 413439.CrossRefGoogle Scholar
Rosenthal, R. (1979). The “file-drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–461.CrossRefGoogle Scholar
Sagarra, N., & Herschensohn, J. (2010). The role of proficiency and working memory in gender and number agreement processing in L1 and L2 Spanish. Lingua, 120, 20222039.CrossRefGoogle Scholar
Sagarra, N., & Herschensohn, J. (2011). Proficiency and animacy effects on L2 gender agreement processes during comprehension. Language Learning, 61, 80116.CrossRefGoogle Scholar
Schenker, N., & Gentleman, J. (2001). On judging the significance of differences by examining the overlap between confidence intervals. The American Statistician, 55, 182186.CrossRefGoogle Scholar
Sharwood-Smith, M., & Truscott, J. (2005). Stages or continua in second language acquisition: A MOGUL solution. Applied Linguistics, 26, 219240.CrossRefGoogle Scholar
Song, Y. (2015). L2 processing of plural inflection in English. Language Learning, 65, 233267.CrossRefGoogle Scholar
Thomas, M. (2006). Research synthesis and historiography: The case of assessment of second language proficiency. In Norris, J. & Ortega, L. (Eds.), Synthesising research on language learning and teaching (pp. 279298). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
Tokowicz, N., & Warren, T. (2010). Beginning adult L2 learners’ sensitivity to morphosyntactic violations: A self-paced reading study. European Journal of Cognitive Psychology, 22, 10921106.CrossRefGoogle Scholar
Tolentino, L., & Tokowicz, N. (2011). Across languages, space and time: A review of the role of cross-language similarity in L2 (morpho)syntactic processing as revealed by fMRI and ERP Methods. Studies in Second Language Acquisition, 33, 91125.CrossRefGoogle Scholar
VanPatten, B. (2007). Input processing in adult second language acquisition. In VanPatten, B. & Williams, J. (Eds.), Theories in second language acquisition: An introduction (pp. 115135). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
VanPatten, B., & Jegerski, J. (Eds.), (2010). Research in second language processing and parsing. Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
VanPatten, B., Keating, G., & Leeser, M. (2012). Missing verbal inflections as a representational problem. Linguistic Approaches to Bilingualism, 2, 109140.CrossRefGoogle Scholar
Supplementary material: File

Avery and Marsden supplementary material

Avery and Marsden supplementary material 1

Download Avery and Marsden supplementary material(File)
File 136.9 KB