Skip to main content Accessibility help
×
Home

DECONSTRUCTING COMPREHENSIBILITY: Identifying the Linguistic Influences on Listeners’ L2 Comprehensibility Ratings

  • Talia Isaacs (a1) and Pavel Trofimovich (a2)

Abstract

Comprehensibility, a major concept in second language (L2) pronunciation research that denotes listeners’ perceptions of how easily they understand L2 speech, is central to interlocutors’ communicative success in real-world contexts. Although comprehensibility has been modeled in several L2 oral proficiency scales—for example, the Test of English as a Foreign Language (TOEFL) or the International English Language Testing System (IELTS)—shortcomings of existing scales (e.g., vague descriptors) reflect limited empirical evidence as to which linguistic aspects influence listeners’ judgments of L2 comprehensibility at different ability levels. To address this gap, a mixed-methods approach was used in the present study to gain a deeper understanding of the linguistic aspects underlying listeners’ L2 comprehensibility ratings. First, speech samples of 40 native French learners of English were analyzed using 19 quantitative speech measures, including segmental, suprasegmental, fluency, lexical, grammatical, and discourse-level variables. These measures were then correlated with 60 native English listeners’ scalar judgments of the speakers’ comprehensibility. Next, three English as a second language (ESL) teachers provided introspective reports on the linguistic aspects of speech that they attended to when judging L2 comprehensibility. Following data triangulation, five speech measures were identified that clearly distinguished between L2 learners at different comprehensibility levels. Lexical richness and fluency measures differentiated between low-level learners; grammatical and discourse-level measures differentiated between high-level learners; and word stress errors discriminated between learners of all levels.

Copyright

Corresponding author

*Correspondence regarding this work should be addressed to: Talia Isaacs, Graduate School of Education, University of Bristol, 35 Berkeley Square, Clifton, Bristol, United Kingdom BS8 1JA; e-mail: talia.isaacs@bristol.ac.uk.

References

Hide All
Anderson-Hsieh, J., Johnson, R., & Koehler, K. (1992). The relationship between native speaker judgments of nonnative pronunciation and deviance in segmentals, prosody, and syllable structure. Language Learning, 42, 529555.
Barkaoui, K. (2008). Effects of scoring method and rater experience on ESL essay rating processes and outcomes. Unpublished doctoral dissertation, University of Toronto, Canada.
Boersma, P., & Weenink, D. (2010). Praat: Doing phonetics by computer (Version 5.1.29) [Computer program]. Retrieved March 10, 2010, fromwww.praat.org.
Brace, N., Kemp, R., & Snelgar, R. (2006). SPSS for psychologists: A guide to data analysis using SPSS for Windows (3rd ed.). Mahwah, NJ: Erlbaum.
Breiner-Sanders, K., Lowe, P., Miles, J., & Swender, E. (2000). ACTFL proficiency guidelines—Speaking revised 1999. Foreign Language Annals, 33, 1318.
British Council, IDP: IELTS Australia, & UCLES. (n.d.). IELTS speaking band descriptors (public version). Retrieved April 8, 2011, fromwww.ielts.org/PDF/UOBDs_SpeakingFinal.pdf.
Brown, A., Iwashita, N., & McNamara, T. (2005). An examination of rater orientations and test-taker performance on English for academic purposes speaking tasks. TOEFL Monograph, 29. Princeton, NJ: Educational Testing Service.
Chalhoub-Deville, M. (1995). A contextualized approach to describing oral language proficiency. Language Learning, 45, 251281.
Cobb, T. (2000). The complete lexical tutor. Retrieved April 17, 2012, fromhttp://www.lextutor.ca.
Cohen, L., Manion, L., & Morrison, K. R. B. (2000). Research methods in education (5th ed.). London: Routledge.
Colby-Kelly, C., & Turner, C. E. (2007). AFL research in the L2 classroom and evidence of usefulness: Taking formative assessment to the next level. Canadian Modern Language Review, 64, 937.
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. New York: Cambridge University Press.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.
Derwing, T. M., & Munro, M. J. (1997). Accent, intelligibility, and comprehensibility: Evidence from four L1s. Studies in Second Language Acquisition, 19, 116.
Derwing, T. M., & Munro, M. J. (2009). Putting accent in its place: Rethinking obstacles to communication. Language Teaching, 42, 115.
Derwing, T. M., Munro, M. J., & Thomson, R. I. (2008). A longitudinal study of ESL learners’ fluency and comprehensibility development. Applied Linguistics, 29, 359380.
Derwing, T. M., & Rossiter, M. J. (2003). The effects of pronunciation instruction on the accuracy, fluency, and complexity of L2 accented speech. Applied Language Learning, 13, 117.
Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I. (2004). Second language fluency: Judgments on different tasks. Language Learning, 54, 665679.
Deterding, D. (2001). The measurement of rhythm: A comparison of Singapore and British English. Journal of Phonetics, 29, 217230.
de Villiers, P. A. (2004). Assessing pragmatic skills in elicited production. Seminars in Speech and Language, 25, 5772.
Educational Testing Service. (2005). TOEFL iBT tips: How to prepare for the next generation TOEFL test and communicate with confidence. Princeton, NJ: Author.
Fayer, J. M., & Krasinski, E. (1987). Native and nonnative judgments of intelligibility and irritation. Language Learning, 37, 313326.
Field, J. (2005). Intelligibility and the listener: The role of lexical stress. TESOL Quarterly, 39, 399423.
Foote, J. A., Holtby, A., & Derwing, T. M. (2011). 2010 survey of pronunciation teaching in adult ESL programs in Canada. TESL Canada Journal, 29, 122.
Foster, P., & Skehan, P. (1996). The influence of planning and task type on second language performance. Studies in Second Language Acquisition, 18, 299323.
Fulcher, G. (1996). Does thick description lead to smart tests? A data-based approach to rating scale construction. Language Testing, 13, 208238.
Gaies, S. J. (1980). T-unit analysis in second language research: Applications, problems and limitations. TESOL Quarterly, 14, 5360.
Gilbert, J. B. (2010). Pronunciation as orphan: What can be done? Speak Out! 43, 37.
Goldwater, S., Jurafsky, D., & Manning, C. D. (2010). Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Speech Communication, 52, 181200.
Hahn, L. D. (2004). Primary stress and intelligibility: Research to motivate the teaching of suprasegmentals. TESOL Quarterly, 38, 201233.
Hamp-Lyons, L. (1991). Scoring procedures for ESL contexts. In Hamp-Lyons, L. (Ed.), Assessing second language writing in academic contexts (pp. 241276). Westport, CT: Ablex.
Harding, L. (in press). Pronunciation assessment. In Chapelle, C. A. (Ed.), The encyclopedia of applied linguistics. Hoboken, NJ: Wiley-Blackwell.
Isaacs, T. (2009). Integrating form and meaning in L2 pronunciation instruction. TESL Canada Journal, 26, 112.
Isaacs, T. (in press). Assessing pronunciation. In Kunnan, A. J. (Ed.), The companion to language assessment. Hoboken, NJ: Wiley-Blackwell.
Isaacs, T., & Thomson, R. I. (2009, March). Judgments of L2 comprehensibility, accentedness, and fluency: The listeners’ perspective. Paper presented at the Language Testing Research Colloquium, Denver, CO.
Isaacs, T., & Thomson, R. I. (in press). Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions. Language Assessment Quarterly.
Isaacs, T., & Trofimovich, P. (2011). Phonological memory, attention control, and musical ability: Effects of individual differences on rater judgments of second language speech. Applied Psycholinguistics, 32, 113140.
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29, 2449.
Jenkins, J. (2000). The phonology of English as an international language. Oxford: Oxford University Press.
Jenkins, J. (2002). A sociolinguistically based, empirically researched pronunciation syllabus for English as an international language. Applied Linguistics, 23, 83103.
Kang, O., Rubin, D., & Pickering, L. (2010). Suprasegmental measures of accentedness and judgments of language learner proficiency in oral English. Modern Language Journal, 94, 554566.
Kennedy, S. (2009). L2 proficiency: Measuring the intelligibility of words and extended speech. In Benati, A. G. (Ed.), Issues in second language proficiency (pp. 132146). New York: Continuum.
Kennedy, S., & Trofimovich, P. (2010). Language awareness and second language pronunciation: A classroom study. Language Awareness, 19, 171185.
Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307322.
Lennon, P. (1990). Investigating fluency in EFL: A quantitative approach. Language Learning, 40, 387417.
Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
Levis, J. M. (2005). Changing contexts and shifting paradigms in pronunciation teaching. TESOL Quarterly, 39, 369377.
Levis, J. M. (2006). Pronunciation and the assessment of spoken language. In Hughes, R. (Ed.), Spoken English, TESOL and applied linguistics: Challenges for theory and practice (pp. 245270). New York: Palgrave Macmillan.
Martin, J. R., & Rose, D. (2003). Working with discourse: Meaning beyond the clause. New York: Continuum.
Morley, J. (1994). A multidimensional curriculum design for speech-pronunciation instruction. In Morley, J. (Ed.), Pronunciation pedagogy and theory (pp. 6491). Alexandria, VA: TESOL.
Munro, M. J. (2008). Foreign accent and speech intelligibility. In Hansen Edwards, J. & Zampini, M. (Eds.), Phonology and second language acquisition (pp. 193218). Amsterdam: Benjamins.
Munro, M. J., & Derwing, T. M. (1999). Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Language Learning, 49, 285310.
Munro, M. J., & Derwing, T. M. (2006). The functional load principle in ESL pronunciation instruction: An exploratory study. System, 34, 520531.
Munro, M. J., & Derwing, T. M. (2011). The foundations of accent and intelligibility in pronunciation research. Language Teaching, 44, 316327.
North, B. (2000). The development of a common framework scale of language proficiency. Bern: Peter Lang.
Peperkamp, S., & Dupoux, E. (2002). A typological study of stress “deafness.” In Gussenhoven, C. & Warner, N. (Eds.), Laboratory phonology 7 (pp. 203236). Berlin: Mouton de Gruyter.
Pickering, L. (2001). The role of tone choice in improving ITA communication in the classroom. TESOL Quarterly, 35, 233255.
Riggenbach, H. (1991). Toward an understanding of fluency: A microanalysis of non-native speaker conversations. Discourse Processes, 14, 423441.
Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin, 57, 416428.
Saito, K., & Lyster, R. (2011). Effects of form-focused instruction and corrective feedback on L2 pronunciation development of /ɹ/ by Japanese learners of English. Language Learning, 62. Retrieved April 17, 2012, fromhttp://onlinelibrary.wiley.com/doi/10.1111/j.1467-9922.2011.00639.x/abstract;jsessionid=490BD498483DC416780FB0D418D12EE2.d04t04?userIsAuthenticated=false&deniedAccessCustomisedMessage=.
Segalowitz, N. (2010). The cognitive bases of second language fluency. London: Routledge.
Skehan, P., & Foster, P. (1999). The influence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93120.
Stein, N. L., & Glenn, C. G. (1979). An analysis of story comprehension in elementary school children. In Freedle, R. (Ed.), New directions in discursive processing (pp. 53120). Westport, CT: Ablex.
Swan, M. (1997). The influence of the mother tongue on second language vocabulary acquisition and use. In Schmitt, N. & McCarthy, M. (Eds.), Vocabulary: Description, acquisition and pedagogy (pp. 156180). New York: Cambridge University Press.
Swan, M., & Smith, B. (Eds.). (2001). Learner English: A teacher’s guide to interference. New York: Cambridge University Press.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.
Thomson, R. I., & Isaacs, T. (2010, June). Variation in L2 oral performance: An examination of task type, topic, and speaker involvement. Paper presented at the annual conference of the Canadian Association of Applied Linguistics, Montreal, QC.
Trofimovich, P., Gatbonton, E., & Segalowitz, N. (2007). A dynamic look at L2 phonological learning: Seeking processing explanations for implicational phenomena. Studies in Second Language Acquisition, 29, 407448.
University of Cambridge Local Examinations Syndicate (UCLES). (2008). Certificate of Proficiency in English: Handbook for teachers. Cambridge: Author.
Upshur, J. A., & Turner, C. E. (1999). Systemic effects in the rating of second-language speaking ability: Test method and learner discourse. Language Testing, 16, 82111.
Varonis, E. M., & Gass, S. (1982). The comprehensibility of non-native speech. Studies in Second Language Acquisition, 4, 114136.
Wennerstrom, A. (2001). The music of everyday speech: Prosody and discourse analysis. Oxford: Oxford University Press.
Zielinski, B. W. (2008). The listener: No longer the silent partner in reduced intelligibility. System, 36, 6984.

Related content

Powered by UNSILO

DECONSTRUCTING COMPREHENSIBILITY: Identifying the Linguistic Influences on Listeners’ L2 Comprehensibility Ratings

  • Talia Isaacs (a1) and Pavel Trofimovich (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.