Hostname: page-component-8448b6f56d-sxzjt Total loading time: 0 Render date: 2024-04-18T06:26:51.467Z Has data issue: false hasContentIssue false
  • English
  • Français

Development of a scale to measure learners’ perceived preferences and benefits of data-driven learning

Published online by Cambridge University Press:  04 November 2015

Atsushi Mizumoto ,
Kiyomi Chujo  and
Kenji Yokota
Atsushi Mizumoto
Affiliation:
Kansai University, Japan (mizumoto@kansai-u.ac.jp)
Kiyomi Chujo
Affiliation:
Nihon University, Japan (chuujou.kiyomi@nihon-u.ac.jp)
Kenji Yokota
Affiliation:
Nihon UniversityJapan (yokota.kenji@nihon-u.ac.jp)
Get access Rights & Permissions [Opens in a new window]

Abstract

In spite of researchers’ and practitioners’ increasing attention to data-driven learning (DDL) and increasing numbers of DDL studies, a multi-item scale to measure learners’ attitude toward DDL has not been developed thus far. In the present study, we developed and validated a psychometric scale to measure learners’ perceived preferences and benefits of DDL for research and pedagogical purposes. First, we created an item pool by referring to open-ended responses from learners; second, the items were pilot tested with target-level learners of English as a foreign language; and third, with item analyses and exploratory factor analysis, the revised version of the questionnaire was prepared. Finally, the questionnaire was administered, and its psychometric properties were examined with confirmatory factor analysis and fit indices. The final phase also included a measure of task values to explore the convergent evidence of the construct validity of the proposed scale. The results suggest that the scale is a valid measure of learners’ attitudes toward DDL, with the hypothesized model providing a good fit with the data. We propose that the scale can be used in future studies that utilize the same type of questionnaire research to facilitate further investigation of DDL. Suggestions for further research are also provided.

Keywords

data-driven learning (DDL)learner concordancingcomputer use in classroomattitudescale developmentCALL
Type
Regular papers
Information
ReCALL , Volume 28 , Issue 2 , May 2016 , pp. 227 - 246
Check for updates
Copyright
Copyright © European Association for Computer Assisted Language Learning 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aijmer, K. (ed.) (2009) Corpora and language teaching. Amsterdam/Philadelphia: John Benjamins.Google Scholar
Anthony, L., Chujo, K. and Oghigian, K. (2011) A novel, web-based, parallel concordancer for use in the ESL/EFL classroom. In Newman, J., Baayen, H. and Rice, S. (eds.), Corpus-based studies in language use, language learning, and language documentation. Amsterdam/New York: Rodopi Press, 123–138.Google Scholar
Aston, G. (ed.) (2001) Learning with corpora. Houston, TX: Athelstan.Google Scholar
Azzaro, G. (2012) Phrasal verbs through DDL. Journal of theories and research in education, 7(2): 1–24.Google Scholar
Boulton, A. (2009) Testing the limits of data-driven learning: Language proficiency and training. ReCALL, 21(1): 37–54. doi:10.1017/S0958344009000068CrossRefGoogle Scholar
Boulton, A. (2010) Data-driven learning: Taking the computer out of the equation. Language Learning, 60(3): 534–572. doi:10.1111/j.1467-9922.2010.00566.xCrossRefGoogle Scholar
Boulton, A. and Pérez-Paredes, P. (2014) ReCALL special issue: Researching uses of corpora for language teaching and learning [Editorial]. ReCALL, 26(2): 121–127. doi:10.1017/S0958344014000068Google Scholar
Cheng, W. (2010) What can a corpus tell us about language teaching? In O’Keeffe, A. and McCarthy, M. (eds.), The Routledge handbook of corpus linguistics. London, England: Routledge, 319–332.Google Scholar
Chujo, K., Anthony, L. and Oghigian, K. (2009 DDL for the EFL classroom: Effective uses of a Japanese-English parallel corpus and the development of a learner-friendly, online parallel concordancer. In Mahlberg, M., González-Díaz, V. and Smith, C. (eds.) Proceedings of 5th Corpus Linguistics Conference 2009, University of Liverpool, UK. http://ucrel.lancs.ac.uk/publications/cl2009Google Scholar
Chujo, K., Anthony, L., Oghigian, K. and Uchibori, A. (2012) Paper-based, computer-based, and combined data-driven learning using a web-based concordancer. Language Education in Asia, 3(2): 132–145. http://www.camtesol.org/Download/LEiA_Vol3_Iss2_2012/LEiA_V3_I2_02_Chujo_et_al_Paper_Computer_and_Combined_DDL.pdfGoogle Scholar
Chujo, K., Anthony, L., Oghigian, K. and Yokota, K. (2013) Teaching remedial grammar through data-driven learning using AntPConc. Taiwan International ESP Journal, 5: 65–90. http://tespaj.tespa.org.tw/index.php/TESPJ/article/view/48Google Scholar
Chujo, K. and Oghigian, K. (2008) A DDL approach to learning noun and verb phrases in the beginner level EFL classroom. In Frankenberg-Garcia, A. (ed.), Proceedings of the 8th Teaching and Language Corpora Conference. Lisbon, Portugal: Associação de Estudos e de Investigação Cientifíca do ISLA-Lisbo Fichier, 65–71.Google Scholar
Chujo, K. and Oghigian, K. (2012) DDL for EFL beginners: A report on student gains and views on paper-based concordancing and the role of L1. In Thomas, J. and Boulton, A. (eds.), Input, process and product: Developments in teaching and language corpora. Brno, Czech Republic: Masaryk University Press, 169–182.Google Scholar
Cobb, T. and Boulton, A. (2015) Classroom applications of corpus analysis. In Biber, D. and Reppen, R. (eds.), The Cambridge handbook of corpus linguistics. Cambridge University Press, 478–497.Google Scholar
Council of Europe. (2001) Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Google Scholar
Cresswell, A. (2007) Getting to “know” connectors? Evaluating data-driven learning in a writing skills course. In Hidalgo, E., Quereda, L. and Santana, J. (eds.), Corpora in the foreign language classroom. Amsterdam: Rodopi, 267–287.Google Scholar
Diamantopoulos, A., Sarstedt, M., Fuchs, C., Wilczynski, P. and Kaiser, S. (2012) Guidelines for choosing between multi-item and single-item scales for construct measurement: A predictive validity perspective. Journal of the Academy of Marketing Science, 40(3): 434–449. doi:10.1007/s11747-011-0300-3CrossRefGoogle Scholar
Dörnyei, Z. (2005) The psychology of the language learner: Individual differences in second language acquisition. Language. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Dörnyei, Z. (2007) Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford: Oxford University Press.Google Scholar
Dörnyei, Z. and Taguchi, T. (2010) Questionnaires in second language research: Construction, administration, and processing, 2nd edn. New York, NY: Routledge.Google Scholar
Educational Testing Service. (2007 Mapping the TOEIC and TOEIC Bridge tests on the Common European Framework of Reference for Languages. http://www.ets.org/s/toeic/pdf/toeic_cef_mapping_flyer.pdfGoogle Scholar
Flowerdew, L. (2009) Applying corpus linguistics to pedagogy: A critical evaluation. International Journal of Corpus Linguistics, 14(3): 393–417. doi:10.1075/ijcl.14.3.05floGoogle Scholar
Flowerdew, L. (2012) Corpora and language education. Basingstoke, England: Palgrave Macmillan.Google Scholar
Frankenberg-Garcia, A. (2014) The use of corpus examples for language comprehension and production. ReCALL, 26(2): 128–146. http://doi.org/10.1017/S0958344014000093Google Scholar
Friginal, E. (2013) Developing research report writing skills using corpora. English for Specific Purposes, 32(4): 208–220. doi:10.1016/j.esp.2013.06.001Google Scholar
Geluso, J. (2013) Phraseology and frequency of occurrence on the web: native speakers’ perceptions of Google-informed second language writing. Computer Assisted Language Learning, 26(2): 144–157. doi:10.1080/09588221.2011.639786Google Scholar
Geluso, J. and Yamaguchi, A. (2014) Discovering formulaic language through data-driven learning: Student attitudes and efficacy. ReCALL, 26(2): 225–242. doi:10.1017/S0958344014000044CrossRefGoogle Scholar
Gilquin, G. and Granger, S. (2010) How can DDL be used in language teaching? In O’Keeffe, A. and McCarthy, M. (eds.), The Routledge handbook of corpus linguistics. London, England: Routledge, 359–369.Google Scholar
Granath, S. (2009) Who benefits from learning how to use corpora? In Aijmer, K. (ed.), Corpora and language teaching. Amsterdam/Philadelphia: John Benjamins, 47–65.Google Scholar
Grgurović, M., Chapelle, C. A. and Shelley, M. C. (2013) A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL, 25(2): 165–198. doi:10.1017/S0958344013000013CrossRefGoogle Scholar
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. and Tatham, R. L. (2006) Multivariate data analysis, (6th edn). Upper Saddle River, NJ: Pearson Prentice Hall.Google Scholar
Hatch, E. and Lazaraton, A. (1991) The research manual: Design and statistics for applied linguistics. Boston, MA: Heinle & Heinle.Google Scholar
Huang, Z. (2014) The effects of paper-based DDL on the acquisition of lexico-grammatical patterns in L2 writing. ReCALL, 26(2): 163–183. doi:10.1017/S0958344014000020Google Scholar
Hunston, S. (2002) Corpora in applied linguistics. Cambridge: Cambridge University Press.Google Scholar
Johansson, S. (2009) Some thoughts on corpora and second-language acquisition. In Aijmer, K. (ed.), Corpora and language teaching. Amsterdam/Philadelphia: John Benjamins, 33–44.CrossRefGoogle Scholar
Johns, T. (1990) From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning. CALL Austria, 10: 14–34.Google Scholar
Kennedy, C. and Miceli, T. (2001) An evaluation of intermediate students’ approaches to corpus investigation. Language Learning & Technology, 5(3): 77–90. http://www.llt.msu.edu/vol5num3/pdf/kennedy.pdfGoogle Scholar
Loewen, S. and Reinders, H. (2011) Key concepts in second language acquisition. Basingstoke, England: Palgrave Macmillan.Google Scholar
Lin, M. H. and Lee, J.-Y. (2015) Data-driven learning: Changing the teaching of grammar in EFL classes. ELT Journal, 69(3): 264–274. doi:10.1093/elt/ccv010Google Scholar
Long, M. H. and Doughty, C. J. (eds.) (2011) The handbook of language teaching. Oxford, England: John Wiley & Sons.Google Scholar
Mackey, A. and Gass, S. M. (eds.) (2012) Research methods in second language acquisition: A practical guide. Chichester, England: John Wiley & Sons, Ltd.Google Scholar
Messick, S. (1995) Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9): 741–749. doi:10.1037/0003-066X.50.9.741Google Scholar
Mizumoto, A. and Chujo, K. (2015) A meta-analysis of data-driven learning approach in the Japanese EFL classroom. English Corpus Studies, 22: 1–18. http://mizumot.com/files/ecs2015.pdfGoogle Scholar
Norris, J. M. (2012) Meta-analysis. In Chapelle, C. A. (ed.), The encyclopedia of applied linguistics. Malden, MA: John Wiley & Sons, 3653–3662. doi:10.1002/9781405198431.wbeal0761Google Scholar
Norris, J. M. and Ortega, L. (2000) Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50(3): 417–528. doi:10.1111/0023-8333.00136CrossRefGoogle Scholar
O’Keeffe, A., McCarthy, M. and Carter, R. (2007) From corpus to classroom: Language use and language teaching. Cambridge: Cambridge University Press.Google Scholar
O’Sullivan, Í. (2007) Enhancing a process-oriented approach to literacy and language learning: The role of corpus consultation literacy. ReCALL, 19(3): 269–286. doi:10.1017/S095834400700033XGoogle Scholar
Pintrich, P. R., Smith, D. A. F., Garcia, T. and McKeachie, W. J. (1993) Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3): 801–813. doi:10.1177/0013164493053003024CrossRefGoogle Scholar
Plonsky, L. and Oswald, F. L. (2014) How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4): 878–912. doi:10.1111/lang.12079CrossRefGoogle Scholar
Core Team., R (2014) R: A language and environment for statistical computing (Version 3.1.1) [Computer software]. Vienna, Austria. http://www.r-project.org/.Google Scholar
Reppen, R. (2010) Using corpora in the language classroom. Cambridge: Cambridge University Press.Google Scholar
Römer, U. (2009) Corpus research and practice: What help do teachers need and what can we offer? In Aijmer, K. (ed.), Corpora and language teaching. Amsterdam/Philadelphia: John Benjamins, 83–98.Google Scholar
Römer, U. (2010) Using general and specialized corpora in English language teaching: Past, present and future. In Campoy-Cubillo, M. C., Bellés Fortuño, B. and Gea-Valor, M. L. (eds.), Corpus-based approaches to English language teaching. London, England: Continuum, 18–38.Google Scholar
Römer, U. (2011) Corpus research applications in second language teaching. Annual Review of Applied Linguistics, 31: 205–225. doi:10.1017/S0267190511000055Google Scholar
Shohamy, E. and Hornberger, N. H. (eds.) (2008) Encyclopedia of language and education, Volume 7: Language testing and assessment, 2nd ed. New York, NY: Springer.Google Scholar
Sinclair, J. M. (ed.) 2004) How to use corpora in language teaching. Amsterdam/Philadelphia: John Benjamins.Google Scholar
Smart, J. (2014) The role of guided induction in paper-based data-driven learning. ReCALL, 26(2): 184–201. doi:10.1017/S0958344014000081Google Scholar
Spada, N. and Tomita, Y. (2010) Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 60(2): 263–308. doi:10.1111/j.1467-9922.2010.00562.xGoogle Scholar
Taylor, L. and Barker, F. (2008) Using corpora for language assessment. In Shohamy, E. and Hornberger, N. H. (eds.), Encyclopedia of language and education, Volume 7: Language testing and assessment, (2nd edn). New York, NY: Springer, 241–256.Google Scholar
Tomlinson, B. (ed.) (2013) Developing materials for language teaching, (2nd edn). London, England: Bloomsbury.Google Scholar
Tono, Y. and Negishi, M. (2012) The CEFR-J: Adapting the CEFR for English language teaching in Japan. Framework & Language Portfolio (FLP) SIG Newsletter, 8: 5–12. http://www.tufs.ac.jp/ts/personal/tonolab/cefr-j/Tono%26Negishi2012forJALT_FLPSIG.pdfGoogle Scholar
Utiyama, M. and Isahara, H. (2003) Reliable measures for aligning Japanese-English news articles and sentences. In Hinrichs, E. and Roth, D. (eds.), Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. Sapporo, Japan: The Association for Computational Linguistics, 72–79. http://acl.ldc.upenn.edu/acl2003/main/index.htmlGoogle Scholar
van Lier, L. (2004) The ecology and semiotics of language learning. Boston. MA: Kluwer Academic.Google Scholar
Wintergerst, A. C., DeCapua, A. and Itzen, R. C. (2001) The construct validity of one learning styles instrument. System, 29(3): 385–403. doi:10.1016/S0346-251X(01)00027-6Google Scholar
Yoon, C. (2011) Concordancing in L2 writing class: An overview of research and issues. Journal of English for Academic Purposes, 10(3): 130–139. doi:10.1016/j.jeap.2011.03.003Google Scholar
Yoon, H. and Hirvela, A. (2004) ESL student attitudes toward corpus use in L2 writing. Journal of Second Language Writing, 13(4): 257–283. doi:10.1016/j.jslw.2004.06.002CrossRefGoogle Scholar
Yoon, H. and Jo, J. W. (2014) Direct and indirect access to corpora: An exploratory case study comparing students’ error correction and learning strategy use in L2 writing. Language Learning & Technology, 18(1): 96–117. http://llt.msu.edu/issues/february2014/yoonjo.pdfGoogle Scholar