Skip to main content Accessibility help
×
Home

A META-ANALYSIS OF SENSITIVITY TO GRAMMATICAL INFORMATION DURING SELF-PACED READING: TOWARDS A FRAMEWORK OF REFERENCE FOR READING TIME EFFECT SIZES

  • Nick Avery (a1) and Emma Marsden (a1)

Abstract

Despite extensive theoretical and empirical research, we do not have estimations of the magnitude of sensitivity to grammatical information during L2 online processing. This is largely due to reliance on null hypothesis significance testing (Plonsky, 2015). The current meta-analysis draws on data from one elicitation technique, self-paced reading, across 57 studies (N = 3,052), to estimate sensitivity to L2 morphosyntax and how far L1 background moderates this. Overall, we found a reliable sensitivity to L2 morphosyntax at advanced proficiencies (d = .20, 95% CIs .15, .25), with some evidence that this was reliably lower than for native speakers (NSs). These patterns were not generally moderated by linguistic feature or sentence region. However, effects for anomaly detection were larger among NSs than L2 learners and the effects among L2 learners appeared to show a trend toward L1 influence. Finding smaller effects than in other subdomains, we provide an initial framework of reference for L2 reading time effect sizes.

Copyright

Corresponding author

*Correspondence concerning this article should be addressed to Nicholas Avery, Department of Education, University of York, Heslington, York, United Kingdom, YO10 5DD. E-mail: nicholas.avery@york.ac.uk

Footnotes

Hide All

Aspects of this article were presented at the 2017 Second Language Research Forum, Ohio State University, and at the 2nd Biennial Conference of the French Second Language Acquisition Network: Research Methodology in the Field of Second Language Acquisition and Learning, Université Paul-Valéry, Montpellier 3, 2018. We are grateful to the editors and three anonymous SSLA reviewers for their valuable comments. We thank Luke Plonsky for useful statistical advice and the authors who shared their data with us: Filiz Çele, Masahiro Hara, Alan Juffs (NSF grant SBR-9709152), Sehoon Jung, and Natasha Tokowicz.

The experiment in this article earned an Open Data badge for transparent practices. The materials are available at https://www.iris-database.org/iris/app/home/detail?id=york%3a936220&ref=search/.

Footnotes

References

Hide All
Adesope, O., Lavin, T., Thompson, T., & Ungerleider, C. (2010). A systematic review and meta-analysis of the cognitive correlates of bilingualism. Review of Educational Research, 80, 207245.
Aldawayan, S., Fiorentino, R., & Gabriele, A. (2010). Evidence of syntactic constraints in the processing of wh-movement: A study of Nadji Arabic learners of English. In VanPatten, B. & Jegerski, J. (Eds.), Research in second language processing and parsing (pp. 6586). Amsterdam, The Netherlands: John Benjamins.
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
Bel, A., Sagarra, N., Comínguez, J., & García-Alcaraz, E. (2016). Transfer and proficiency effects in L2 processing of subject anaphora. Lingua, 184, 134159.
Bley-Vroman, R. (1990). The logical problem of foreign language learning. Linguistic Analysis, 20, 349.
Bley-Vroman, R. (2009). The evolving context of the fundamental difference hypothesis. Studies in Second Language Acquisition, 31, 175198.
Borenstein, M., Hedges, L., & Rothstein, H. (2007). Meta-analysis: Fixed effect vs. random effects. Retrieved from www.meta-analysis.com.
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1, 120.
Bultena, S., Dijkstra, T., & van Hell, J. (2014). Cognate effects in sentence context depend on word class, L2 proficiency, and task. The Quarterly Journal of Experimental Psychology, 67, 12141241.
Chan, H. (2012). Tense-aspect processing in second language learners (Unpublished doctoral dissertation). University of Pittsburgh.
Clahsen, H., & Felser, C. (2006a). Grammatical processing in language learners. Applied Psycholinguistics, 27, 342.
Clahsen, H., & Felser, C. (2006b). How native-like is non-native language processing? Trends in Cognitive Sciences, 10, 564570.
Clahsen, H., & Felser, C. (2018). Some notes on the shallow structure hypothesis. Studies in Second Language Acquisition, 40, 693706.
Clahsen, H., Felser, C., Neubauer, C., Sato, M., & Silva, R. (2010). Morphological structure in native and nonnative language processing. Language Learning, 60, 2143
Coe, R. (2002). It’s the effect size, stupid: What the effect size is and why it is important. Paper presented at the British Educational Research Association annual conference, Exeter, 12–14 September 2002.
Cohen, J. (1969). Statistical power for the behavioural sciences (2nd ed.). Hove, UK: Laurence Erlbaum.
Cumming, G. (2009). Inference by eye: Reading the overlap of independent confidence intervals. Statistics in Medicine, 28, 205220.
Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge
Dekydtspotter, L., Edmonds, A., Fultz, A., & Renaud, C. (2010). Modularity of L2 sentence processing: Prosody, context and morphology in relative clause ambiguity in English-French interlanguage. In Iverson, M., Ivanov, I., Judy, T., Rothman, J., Slabakova, R., & Tryzna, M. (Eds.), Proceedings of the 2009 mind/context divide workshop (pp. 1327). Somerville, MA: Cascadilla Proceedings Project.
Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology, 5, 781.
Dussias, P. (2003). Syntactic ambiguity resolution in L2 learners: Some effects of bilinguality on L1 and L2 processing strategies. Studies in Second Language Acquisition, 25, 529557.
Ellis, N. (2006). Meta-analysis, human cognition and language learning. In Norris, J. & Ortega, L. (Eds.), Synthesizing research on language learning and teaching (pp. 301322). Amsterdam, The Netherlands: John Benjamins.
Felser, C., Roberts, L., Marinis, T., & Gross, R. (2003). The processing of ambiguous sentences by first and second language learners of English. Applied Psycholinguistics, 24, 453489.
Fender, M. (2003). English word recognition and word integration skills of native Arabic and Japanese-speaking learners of English as a second language. Applied Psycholinguistics, 24, 289315.
Foote, R. (2011). Integrated knowledge of agreement in early and late English-Spanish bilinguals. Applied Psycholinguistics, 32, 187220.
Gerth, S., Otto, C., Felser, C., & Nam, Y. (2017). Strength of garden path effects in native and non-native speakers’ processing of subject-object ambiguities. International Journal of Bilingualism, 21, 125144.
Greenland, S., Senn, S., Rothman, K., Carlin, J., Poole, C., Goodman, S., & Altmann, D. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31, 337350.
Havik, E., Roberts, L., van Hout, R., Schreuder, R., & Haverkort, M. (2009). Processing subject-object ambiguities in the L2: A self-paced reading study with German L2 learners of Dutch. Language Learning, 59, 73112.
Hedge, C., Powell, G., Bompas, A., Vivian-Griffiths, S., & Sumner, P. (2018). Low and variable correlation between reaction time costs and accuracy costs explained by two accumulation models: Meta-analysis and simulations. Psychological Bulletin, 144, 12001227.
Hopp, H. (2006). Syntactic features and re-analysis in near-native processing. Second Language Research, 22, 369397.
Hopp, H. (2009). The syntax-discourse interface in near-native L2 acquisition: Off-line and on-line performance. Bilingualism: Language and Cognition, 12, 463483.
Hopp, H. (2010). Ultimate attainment in L2 inflection: Performance similarities between non-native and native speakers. Lingua, 120, 901931.
Howell, D. (2013). Statistical methods for psychology (8th ed.). Belmont, MA: Wadsworth Cengage Learning.
Hulstijn, J. (2019). An individual differences framework for comparing non-native with native speakers: Perspectives from basic language cognition theory. Language Learning, 69, 157183.
Jackson, C. (2008). Proficiency level and the interaction of lexical and morphosyntactic information during L2 sentence processing. Language Learning, 58, 875909.
Jackson, C. (2010). The processing of subject-object ambiguities by English and Dutch L2 learners of German. In VanPatten, B. & Jegerski, J. (Eds.), Research in second language processing and parsing (pp. 207230). Amsterdam, The Netherlands: John Benjamins.
Jackson, C., & Dussias, P. (2009). Cross-linguistic differences and their impact on L2 sentence processing. Bilingualism: Language and Cognition, 12, 6582.
Jackson, C., & van Hell, J. (2011). The effects of L2 proficiency level on the processing of wh-questions among Dutch second language speakers of English. International Review of Applied Linguistics, 49, 195219.
Jarvis, S., & Pavlenko, A. (2008). Cross-linguistic influence in language and cognition. London, UK: Routledge.
Jegerski, J. (2012). The processing of subject-object ambiguities in native and near-native Mexican Spanish. Bilingualism: Language and Cognition, 15, 721735.
Jegerski, J. (2014). Self-paced reading. In Jegerski, J. & VanPatten, B. (Eds.), Research methods in second language psycholinguistics (pp. 2049). New York, NY: Routledge.
Jegerski, J. (2016). Number attraction effects in near-native Spanish comprehension. Studies in Second Language Acquisition, 38, 533.
Jegerski, J. (2018). Sentence processing in Spanish as a heritage language: A self-paced reading study of relative clause attachment. Language Learning, 68, 598634.
Jiang, N. (2004). Morphological insensitivity in second language processing. Applied Psycholinguistics, 25, 603634.
Jiang, N. (2007). Selective integration of linguistic knowledge in adult second language learning. Language Learning, 57, 133.
Jiang, N., Novokshanova, E., Masuda, K., & Wang, X. (2011). Morphological congruency and the acquisition of L2 morphemes. Language Learning, 61, 940967.
Juffs, A. (1998a). Main verb versus reduced relative clause ambiguity resolution in L2 sentence processing. Language Learning, 48, 107147.
Juffs, A. (1998b). Some effects of first language argument structure and morphosyntax on second language sentence processing. Second Language Research, 14, 406424.
Juffs, A. (2005). The influence of first language on the processing of wh-movement in English as a second language. Second Language Research, 21, 121151.
Juffs, A. (2006). Processing reduced relative vs. main verb ambiguity in English as a Second Language: A replication study with working memory. In Slabakova, R., Montrul, S., & Prevost, P. (Eds.), Inquiries in linguistic development in honor of Lydia White (pp. 213232). Amsterdam, The Netherlands: John Benjamins.
Juffs, A., & Harrington, M. (1995). Parsing effects in second language sentence processing: Subject and object asymmetries in wh-extraction. Studies in Second Language Acquisition, 17, 483516.
Juffs, A., & Rodriguez, G. (2015). Second language sentence processing. New York, NY: Routledge.
Jung, S. (2010). Second language processing of wh-movement in English: The effects of first language and learning environment (Unpublished MA dissertation). Michigan State University.
Just, M., Carpenter, P., & Woolley, J. (1982). Paradigms and processes in reading comprehension. Journal of Experimental Psychology: General, 111, 228238.
Kaan, E., Ballantyne, J., & Wijnen, F. (2015). Effects of reading speed on second-language sentence processing. Applied Psycholinguistics, 36, 799830.
Keating, G., & Jegerski, J. (2015). Experimental designs in sentence processing research: A methodological review and user’s guide. Studies in Second Language Acquisition, 37, 132.
Keating, G., Jegerski, J., & VanPatten, B. (2016). Online processing of subject pronouns in monolingual and heritage bilingual speakers of Mexican Spanish. Bilingualism: Language and Cognition, 19, 3649.
Lakshmanan, U., Kim, K., McCreary, R., Park, K., Suen, U., & Lee, S. (2009). L2 learners’ sensitivity to strong and weak subjacency-violations in online processing. In Bowles, M., Ionin, T., Montrul, S., & Tremblay, A. (Eds.), Proceedings of the 10th Generative Approaches to Second Language Acquisition Conference (GASLA 2009) (pp. 136143). Somerville, MA: Cascadilla Proceedings Project.
Larson-Hall, J. (2016). A guide to doing statistics in second language research using SPSS and R. New York, NY: Routledge.
Lauro, J., & Schwartz, A. (2017). Bilingual non-selective lexical access in sentence contexts: A meta-analytic review. Journal of Memory and Language, 92, 217233.
Lee, E., Lu, D., & Garnsey, S. (2013). L1 word order and sensitivity to verb bias in L2 processing. Bilingualism: Language and Cognition, 16, 761775.
Lee, J., Jang, J., & Plonsky, L. (2015). The effectiveness of second language pronunciation instruction: A meta-analysis. Applied Linguistics, 36, 345366.
Lee, S., & Huang, H. (2008). Visual input enhancement and grammar learning: A meta-analytic review. Studies in Second Language Acquisition, 30, 307331.
Lipsey, M., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
MacWhinney, B. (2005). Extending the competition model. International Journal of Bilingualism, 9, 6984.
Marinis, T. (2007). On-line processing of passives in L1 and L2 children. In Belikova, A., Meroni, L., & Umeda, M. (Eds.), Proceedings of the 2nd Conference on Generative Approaches to Language Acquisition North America (GALANA) (pp. 265276). Somerville, MA: Cascadilla Proceedings Project.
Marinis, T. (2010). Using on-line processing methods in language acquisition research. In Blunsworth, S. & Blom, E. (Eds.), Experimental methods in language acquisition research (pp. 139162). Amsterdam, The Netherlands: John Benjamins.
Marinis, T., Roberts, L., Felser, C., & Clahsen, H. (2005). Gaps in second language processing. Studies in Second Language Acquisition, 27, 5378.
Marsden, E., Thompson, S., & Plonsky, L. (2018). A methodological synthesis of self-paced reading in second language research. Applied Psycholinguistics, 39, 861904.
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68, 321–291.
Morgan-Short, K., Marsden, E., Heil, J., Issa, B. II, Leow, R., Mikhaylova, A., Mikołajczak, S., Moreno, N., Slabakova, R., & Szudarski, P. (2018). Multi-site replication in second language acquisition research: Attention to form during listening and reading comprehension. Language Learning, 68, 392437.
Morris, S., & DeShon, R. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105125.
Mueller, J., & Jiang, N. (2013). The acquisition of the Korean honorific affix (u)si by advanced L2 learners. The Modern Language Journal, 97, 318339.
Norouzian, R., de Miranda, M., & Plonsky, L. (2018). The Bayesian revolution in second language research. Language Learning, 68, 10321075.
Norris, J., & Ortega, L. (Eds.), (2006). Synthesizing research on language learning and teaching. Amsterdam, The Netherlands: John Benjamins.
Odlin, T. (2005). Cross-linguistic influence. In Doughty, C., & Long, M. (Eds.), The handbook of second language acquisition (pp. 436486). Oxford, UK: Blackwell.
Odlin, T. (2012). Cross-linguistic influence in second language acquisition. In Chappelle, C. (Ed.), The encyclopaedia of applied linguistics (pp. 15621568). Malden, MA: Blackwell.
O’Grady, W. (2005). Syntactic carpentry: An emergentist approach to syntax. Mahwah, NJ: Lawrence Erlbaum
Oswald, F., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 30, 85110.
Pan, H., Schimke, S., & Felser, C. (2015). Referential context effects in non-native relative clause ambiguity resolution. International Journal of Bilingualism, 19, 298313.
Papadopoulou, D. (2005). Reading time studies of second language ambiguity resolution. Second Language Research, 21, 98120.
Papadopoulou, D., & Clahsen, H. (2003). Parsing strategies in L1 and L2 sentence processing: A study of relative clause attachment in Greek. Studies in Second Language Acquisition, 25, 501528.
Perpiñán, S. (2014). L2 grammar and L2 processing in the acquisition of Spanish prepositional relative clauses. Bilingualism: Language and Cognition, 18, 577596.
Plonsky, L. (2011). The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 61, 9931038.
Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655687.
Plonsky, L. (Ed.), (2015). Advancing quantitative methods in second language research. New York, NY: Routledge.
Plonsky, L., & Brown, D. (2015). Domain definition and search techniques in meta-analyses of L2 research (Or why 18 meta-analyses of feedback have different results). Second Language Research, 31, 267278.
Plonsky, L., & Oswald, F. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878911.
Rah, A., & Adone, D. (2010). Processing of the reduced relative clause versus main verb ambiguity in L2 learners at different proficiency levels. Studies in Second Language Acquisition, 32, 79109.
Rayner, K., Schotter, E., Masson, M., Potter, M., & Treiman, R. (2016). So much to read, so little time: How do we read, and can speed reading help? Psychological Science in the Public Interest, 17, 434.
Renaud, C. (2014). A processing investigation of the accessibility of the uninterpretable gender feature in L2 French and L2 Spanish adjective agreement. Linguistic Approaches to Bilingualism, 4, 222255.
Roberts, L. (2007). Investigating real-time sentence processing in the second language. Stem-, Spraak- en Taalpathologie, 15, 115127.
Roberts, L. (2012). Psycholinguistic techniques and resources in second language acquisition research. Second Language Research, 28, 113127.
Roberts, L. (2013). Sentence processing in bilinguals. In van Gompel, R. (Ed.), Sentence processing (pp. 221246). Hove, UK: Psychology Press.
Roberts, L. (2016). Self-paced reading and L2 grammatical processing. In Mackey, A. & Marsden, E. (Eds.), Advancing methodology and practice: The IRIS repository for instruments for research into second languages (pp. 5872). New York, NY: Routledge.
Roberts, L., & Felser, C. (2011). Plausibility and recovery from garden paths in second language sentence processing. Applied Psycholinguistics, 32, 299331.
Roberts, L., & Liszka, S. (2013). Processing tense/aspect-agreement violations on-line in the second language: A self-paced reading study with French and German L2 learners of English. Second Language Research, 29, 413439.
Rosenthal, R. (1979). The “file-drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–461.
Sagarra, N., & Herschensohn, J. (2010). The role of proficiency and working memory in gender and number agreement processing in L1 and L2 Spanish. Lingua, 120, 20222039.
Sagarra, N., & Herschensohn, J. (2011). Proficiency and animacy effects on L2 gender agreement processes during comprehension. Language Learning, 61, 80116.
Schenker, N., & Gentleman, J. (2001). On judging the significance of differences by examining the overlap between confidence intervals. The American Statistician, 55, 182186.
Sharwood-Smith, M., & Truscott, J. (2005). Stages or continua in second language acquisition: A MOGUL solution. Applied Linguistics, 26, 219240.
Song, Y. (2015). L2 processing of plural inflection in English. Language Learning, 65, 233267.
Thomas, M. (2006). Research synthesis and historiography: The case of assessment of second language proficiency. In Norris, J. & Ortega, L. (Eds.), Synthesising research on language learning and teaching (pp. 279298). Amsterdam, The Netherlands: John Benjamins.
Tokowicz, N., & Warren, T. (2010). Beginning adult L2 learners’ sensitivity to morphosyntactic violations: A self-paced reading study. European Journal of Cognitive Psychology, 22, 10921106.
Tolentino, L., & Tokowicz, N. (2011). Across languages, space and time: A review of the role of cross-language similarity in L2 (morpho)syntactic processing as revealed by fMRI and ERP Methods. Studies in Second Language Acquisition, 33, 91125.
VanPatten, B. (2007). Input processing in adult second language acquisition. In VanPatten, B. & Williams, J. (Eds.), Theories in second language acquisition: An introduction (pp. 115135). Mahwah, NJ: Lawrence Erlbaum.
VanPatten, B., & Jegerski, J. (Eds.), (2010). Research in second language processing and parsing. Amsterdam, The Netherlands: John Benjamins.
VanPatten, B., Keating, G., & Leeser, M. (2012). Missing verbal inflections as a representational problem. Linguistic Approaches to Bilingualism, 2, 109140.
Type Description Title
UNKNOWN
Supplementary materials

Avery and Marsden supplementary material
Avery and Marsden supplementary material 1

 Unknown (137 KB)
137 KB

A META-ANALYSIS OF SENSITIVITY TO GRAMMATICAL INFORMATION DURING SELF-PACED READING: TOWARDS A FRAMEWORK OF REFERENCE FOR READING TIME EFFECT SIZES

  • Nick Avery (a1) and Emma Marsden (a1)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed