Skip to main content Accessibility help
×
Home

MEASURING COGNITIVE TASK DEMANDS USING DUAL-TASK METHODOLOGY, SUBJECTIVE SELF-RATINGS, AND EXPERT JUDGMENTS: A Validation Study

  • Andrea Révész (a1), Marije Michel (a2) and Roger Gilabert (a3)

Abstract

This study explored the usefulness of dual-task methodology, self-ratings, and expert judgments in assessing task-generated cognitive demands as a way to provide validity evidence for manipulations of task complexity. The participants were 96 students and 61 English as a second language (ESL) teachers. The students, 48 English native speakers and 48 ESL speakers, carried out simple and complex versions of three oral tasks—a picture narrative, a map task, and a decision-making task. Half of the students completed the tasks under a dual-task condition. The remaining half performed the tasks under a single-task condition without a secondary task. Participants in the single condition were asked to rate their perceived mental effort and task difficulty. The ESL teachers provided expert judgments of anticipated mental effort and task difficulty along with explanations for their ratings via an online questionnaire. As predicted, the more complex task versions were found and judged to pose greater cognitive effort on most measures.

Copyright

References

Hide All
Bachman, L. F. (2002). Some reflections on task-based language performance assessment. Language Testing, 19, 453476.
Baralt, M. (2013). The impact of cognitive complexity on feedback efficacy during online versus face-to-face interactive tasks. Studies in Second Language Acquisition, 35, 689725.
Brown, J. D., Hudson, T., Norris, J. M., & Bonk, W. (2002). An investigation of second language task-based performance assessments (Technical Report No. 24). University of Hawaii, Second Language Teaching & Curriculum Center.
Brünken, R., Plass, J. L., & Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science, 32, 115132.
Brünken, R., Seufert, T., & Paas, F. (2010). Measuring cognitive load. In Plass, J. L., Moreno, R., & Brünken, R. (Eds.), Cognitive load theory (pp. 181202). Cambridge: Cambridge University Press.
Chandler, P., & Sweller, J. (1996). Cognitive load while learning to use a computer program. Applied Cognitive Psychology, 10, 120.
Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in Human Behavior, 25, 315324.
Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd ed.). New York: Academic Press.
Dave, A. (2004). Oxford Placement Test 1. Oxford: Oxford University Press.
Declerck, M., & Kormos, J. (2012). The effect of dual task demands and proficiency on second language speech production. Bilingualism: Language and Cognition, 15, 782796.
DeKeyser, R. (1997). Beyond explicit rule learning: Automatizing second language morphosyntax. Studies in Second Language Acquisition, 19, 195221.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175191.
Fredericks, T. K., Choi, S. D., Hart, J., Butt, S. E., & Mital, A. (2005). An investigation of myocardial aerobic capacity as a measure of both physical and cognitive workloads. International Journal of Industrial Ergonomics, 35, 10971107.
Gilabert, R. (2006). The simultaneous manipulation of task complexity along planning time and (+/− here-and-now): Effects on L2 oral production. In Mayo, M. P. García (Ed.), Investigating tasks in formal language learning (pp. 4468). Clevedon, England: Multilingual Matters.
Gilabert, R. (2007). Effects of manipulating task complexity on self-repairs during L2 oral production. International Review of Applied Linguistics in Language Teaching, 45, 215240.
Gilabert, R., & Barón, J. (2013). The impact of increasing task complexity on L2 pragmatic moves. In Mackey, A. & McDonough, K. (Eds.), Second language interaction in diverse educational settings (pp. 4569). Amsterdam: John Benjamins.
Gilabert, R., Barón, J., & Levkina, M. (2011). Manipulating task complexity across task types and modes. In Robinson, P. (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 105140). Amsterdam: John Benjamins.
Gilabert, R., Barón, J., & Llanes, À. (2009). Manipulating cognitive complexity across task types and its impact on learners’ interaction during oral performance. International Review of Applied Linguistics in Language Teaching, 47, 367395.
Housen, A., & Kuiken, F. (Eds.). (2009). Complexity, accuracy and fluency in second language acquisition research [Special issue]. Applied Linguistics, 30, 461601.
Kim, Y. (2009). The effects of task complexity on learner-learner interaction. System, 37, 254268.
Kim, Y., & Tracy-Ventura, N. (2011). Task complexity, language anxiety, and the development of the simple past. In Robinson, P. (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 287306). Amsterdam: Benjamins.
Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
Levkina, M., & Gilabert, R. (2012). The effects of cognitive task complexity on L2 oral production. In Housen, A., Vedder, I., & Kuiken, F. (Eds.), Dimensions of L2 performance and proficiency investigating complexity, accuracy, and fluency in SLA (pp. 171198). Amsterdam: John Benjamins.
Levkina, M., & Gilabert, R. (2014). Task sequencing in the L2 development of spatial expressions. In Baralt, M., Gilabert, R., & Robinson, P. (Eds.), Task sequencing and instructed second language learning. New York: Bloomsbury.
Malicka, A., & Levkina, M. (2012). Measuring task complexity: Does L2 proficiency matter? In Shehadeh, A. & Coombe, C. (Eds.), Task-based language teaching in foreign language contexts: Research and implementation (pp. 4366). Amsterdam: John Benjamins.
Michel, M. (2011). Effects of task complexity and interaction on L2 performance. In Robinson, P. (Ed.), Second language task complexity: Researching the Cognition Hypothesis of language learning and performance (pp. 141174). Amsterdam: John Benjamins.
Murata, A. (2005). An attempt to evaluate mental workload using wavelet transform of EEG. Human Factors, 47, 498508.
Norris, J. M. (2010, September). Understanding instructed SLA: Constructs, contexts, and consequences. Plenary address delivered at the annual conference of the European Second Language Association (EUROSLA), Reggio Emilia, Italy.
Norris, J., & Ortega, L. (2003). Defining and measuring L2 acquisition. In Doughty, C. J. & Long, M. H. (Eds.), The handbook of second language acquisition (pp. 717761). Malden, MA: Blackwell.
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30, 555578.
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429434.
Paas, F., Tuovinen, J., Tabbers, H., & van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 6371.
Paas, F., & van Merriënboer, J. J. G. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 351372.
Révész, A. (2009). Task complexity, focus on form, and second language development. Studies in Second Language Acquisition, 31, 437470.
Révész, A. (2011). Task complexity, focus on L2 constructions, and individual differences: A classroom-based study. Modern Language Journal, 95, 168181.
Révész, A. (2014). Towards a fuller assessment of cognitive models of task-based learning: Investigating task-generated cognitive demands and processes. Applied Linguistics, 35, 8792.
Révész, A., Sachs, R., & Hama, M. (2014). The effects of task complexity and input frequency on the acquisition of the past counterfactual construction through recasts. Language Learning, 64, 615650.
Robinson, P. (2001a). Task complexity, cognitive resources, and syllabus design: A triadic framework for investigating task influences on SLA. In Robinson, P. (Ed.), Cognition and second language instruction (pp. 287318). New York: Cambridge University Press.
Robinson, P. (2001b). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22, 2757.
Robinson, P. (2003). Attention and memory during SLA. In Doughty, C. J. & Long, M. H. (Eds.), Handbook of second language acquisition (pp. 631678). Malden, MA: Blackwell.
Robinson, P. (2007). Task complexity, theory of mind, and intentional reasoning: Effects on L2 speech production, interaction, uptake and perceptions of task difficulty. International Review of Applied Linguistics, 45, 193213.
Robinson, P. (2011). Task-based language learning: A review of issues. Language Learning, 61, 136.
Sasayama, S. (2013, October). Is a “complex” task really complex? Measuring task complexity independently from linguistic production. Paper presented at the 5th Biennial International Conference on Task-Based Language Teaching, Banff, Alberta, Canada.
Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-Prime user’s guide. Pittsburgh, PA: Psychology Software Tools.
Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.
Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency and lexis. Applied Linguistics, 30, 510532.
Wickens, C. (2007). Attention to the second language. International Review of Applied Linguistics, 45, 177191.

MEASURING COGNITIVE TASK DEMANDS USING DUAL-TASK METHODOLOGY, SUBJECTIVE SELF-RATINGS, AND EXPERT JUDGMENTS: A Validation Study

  • Andrea Révész (a1), Marije Michel (a2) and Roger Gilabert (a3)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed