Skip to main content Accessibility help
×
×
Home

The Ecological and Construct Validity of a Newly Developed Measure of Executive Function: The Virtual Library Task

  • Belinda Renison (a1), Jennie Ponsford (a1) (a2), Renee Testa (a1) (a3), Barry Richardson (a4) and Kylie Brownfield (a1)...

Abstract

Virtual reality (VR) assessment paradigms have the potential to address the limited ecological validity of pen and paper measures of executive function (EF) and the pragmatic and reliability issues associated with functional measures. To investigate the ecological validity and construct validity of a newly developed VR measure of EF, the Virtual Library Task (VLT); a real life analogous task—the Real Library Task (RLT); and five neuropsychological measures of EF were administered to 30 patients with traumatic brain injury (TBI) and 30 healthy Controls. Significant others for each participant also completed the Dysexecutive Questionnaire (DEX), which is a behavioral rating scale of everyday EF. Performances on the VLT and the RLT were significantly positively correlated indicating that VR performance is similar to real world performance. The TBI group performed significantly worse than the Control group on the VLT and the Modified Six Elements Test (MSET) but the other four neuropsychological measures of EF failed to differentiate the groups. Both the MSET and the VLT significantly predicted everyday EF suggesting that they are both ecologically valid tools for the assessment of EF. The VLT has the advantage over the MSET of providing objective measurement of individual components of EF. (JINS, 2012, 18, 440–450)

Copyright

Corresponding author

Correspondence and reprint requests to: Jennie Ponsford, School of Psychology and Psychiatry, Monash University, Clayton, 3800, Victoria, Australia. E-mail: jennie.ponsford@monash.edu

References

Hide All
Alderman, N., Burgess, P.W., Knight, C., Henman, C. (2003). Ecological validity of a simplified version of the multiple errands shopping test. Journal of the International Neuropsychological Society, 9(1), 3144.
Axelrod, B.N., Goldman, R.S., Heaton, R.K., Curtiss, G., Thompson, L.L.., Ghelune, G.J., et al. (1996). Discriminability of the Wisconsin Card Sorting Test using the standardization sample. Journal of Clinical & Experimental Neuropsychology Society, 18, 338342.
Bennett, P.C., Ong, B., Ponsford, J. (2005a). Assessment of executive dysfunction following traumatic brain injury: Comparison of the BADS with other clinical neuropsychological measures. Journal of the International Neuropsychological Society, 11(05), 606613. doi:10.1017/S1355617705050721
Bennett, P.C., Ong, B., Ponsford, J. (2005b). Measuring executive dysfunction in an acute rehabilitation setting: Using the dysexecutive questionnaire (DEX). Journal of the International Neuropsychological Society, 11(4), 376385.
Benton, A. (1968). Differential behavioural effects on frontal lobe disease. Neuropsychologia, 6, 5360.
Brooks, B.M., Rose, F.D., Potter, J., Jayawardena, S., Morling, A. (2004). Assessing stroke patients’ prospective memory using virtual reality. Brain Injury, 18(4), 391401.
Burgess, P.W., Alderman, N., Evans, J., Emslie, H., Wilson, B.A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 10(4), 547558.
Burgess, P.W., Alderman, N., Forbes, C., Costello, A., Coates, L.M., Dawson, D.R., Channon, S. (2006). The case for the development and use of ‘ecologically valid’ measures of executive function in experimental and clincial neuropsychology. Journal of the International Neuropsychological Society, 12(2), 194209 .
Burgess, P.W., Alderman, N., Volle, E., Benoit, R.G., Gilbert, S.J. (2009). Mesulam's frontal lobe mystery re-examined. Resotrative Neurology and Neuroscience, 27(5), 493506.
Burgess, P.W., Shallice, T. (1996). The Hayling and Brixton Tests: Test manual. Edmunds, England: Themes Valley Test Company Limited.
Busch, R.M., McBride, A., Curtiss, G., Vanderploeg, R.D. (2005). The components of executive functioning in traumatic brain injury. Journal of Clinical & Experiemental Neuropsychology, 27, 10221032.
Chan, R.C. (2001). Dysexecutive symptoms among a non-clinical sample: A study with the use of the Dysexecutive Questionnaire. British Journal of Psychology, 92(3), 551565.
Chan, R., Manly, T. (2002). The application of “dysexecutive syndrome” measures across cultures: Performance and checklist assessment in neurologically healthy and traumatically brain-injured Hong Kong Chinese volunteers. Journal of the International Neuropsychological Society, 8(6), 771780.
Chan, R., Shum, D., Toulopoulou, T., Chen, E. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23, 201216.
Chaytor, N., Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13(4), 181197.
Chaytor, N., Schmitter-Edgecombe, M., Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21(3), 217227.
Coakes, S.J., Steed, L.G. (2003). SPSS Analysis Without Anguish. Queensland: John Wiley & Sons Australia.
Crawford, J.R., Henry, J.D. (2005). Assessment of executive dysfunction. New York, NY: Oxford University Press.
Cushman, L.A., Duffy, C.J. (2008). Detecting navigational deficits in cognitive aging and Alzheimer disease using virtual reality. Neurology, 71, 888895.
Damasio, A.R., Anderson, S.W. (2003). The frontal lobes. In K.M. Heilman & E. Valenstein (Eds.), Clinical Neuropsychology (4th ed.). New York: Oxford University Press.
Dawson, D.R., Anderson, N.D., Burgess, P., Cooper, E., Krpan, K.M., Stuss, D.T. (2009). Further development of the Multiple Errands Test: Standardized scoring, reliability, and ecological validity for the Baycrest version. Archives of Physical Medicine and Rehabilitation, 90(11, Suppl. 1), S41S51.
Draper, K., Ponsford, J. (2008). Cognitive functioning ten years following traumatic brain injury and rehabilitation. Neuropsychology, 22(5), 618625.
Duncan, J. (2005). Frontal lobe function and general intelligence: Why it matters. Cortex, 41, 215217.
Duncan, J., Johnson, R., Swales, M., Freer, C. (1997). Frontal lobe deficits after head injury: Unity and diversity of function. Cognitive Neuropsychology, 14(5), 713741.
Elkind, J.S., Rubin, E., Rosenthal, S., Skoff, B., Prather, P. (2001). A simulated reality scenario compared with the computerized Wisconsin Card Sorting Test: An analysis of Preliminary results. CyberPsychology & Behavior, 4(4), 489496.
Evans, J., Chua, S., McKenna, P., Wilson, B. (1997). Assessment of the dysexecutive syndrome in schizophrenia. Psychological Medicine, 27(3), 635646.
Greve, K.W., Brooks, J., Crouch, J.A., Williams, M.C., Rice, W.J. (1997). Factorial structure of the Wisconsin Card Sorting Test. British Journal of Clinical Psychology, 36, 283285.
Heaton, R.K. (2005). Wisconsin Card Sorting Test-64: Computer version 2 – research edition (WCST-64CV2). Lutz, FL: Psychological Assessment Resources.
Jansari, A., Agnew, R., Akesson, K., Murphy, L (2004). Using virtual reality to create an ecologically valid measure of real world problems in patients with dysexecutive syndrome. Paper presented at the Symposium on Neurological Rehabilitation: A Satellite Symposium to the Joint 27th Conference of the Australian Society for the Study of Brain Impairment and the 27th Mid-year Meeting of the International Neuropsychological Society Uluru, Australia.
Jovanovski, D. (2004). Cognitive set shifting using functional magnetic reasonance imaging and virtual reality: A comparison between traditional and a novel ecologically valid exectuive function task. Master of Arts, University of Toronto, Toronto.
Joyner, K.B., Silver, C.H., Stavinoha, P.L. (2009). Relationship between parenting stress and ratings of executive functioning in children with ADHD. Journal of Psychoeducational Assessment, 27(6), 452464. doi:10.1177/0734282909333945
Katz, N., Tadmor, I., Felzen, B., Hartman-Maeir, A. (2007). Validity of the Executive Function Performance Test in individuals with schizophrenia. OTJR: Occupation, Participation and Health, 27(2), 4451.
Klinger, E., Chemin, I., Lebreton, S., Marie, R.-M. (2006). Virtual action planning in Parkinson's Disease: A control study. CyberPsychology & Behaviour, 9(3), 342347.
Knight, C., Alderman, N. (2002). Development of a simplified version of the Multiple Errands Test for use in hospital settings. Neuropsychological Rehabilitation, 12(3), 231255.
Knight, R.G., Titov, N. (2009). Use of virtual reality tasks to assess prospective memory: Applicability and evidence. Brain Impairment, 10(1), 313.
Lamberts, K.F., Evans, J.J., Spikman, J.M. (2010). A real-life, ecologically valid test of executive functioning: The executive secretarial task. Journal of Clinical and Experimental Neuropsychology, 32(1), 5665.
Lezak, M.D., Howieson, D.B., Loring, D.W. (2004). Neuropsychological assessment (4th ed.). New York: Oxford Univerity Press.
Long, C.J., Collins, L.F. (1997). Ecological validity and forensic neuropsychological assessment. In R.J. McCaffrey, A.D. Williams, J.M. Fisher, & L.C. Laign (Eds.), The practice of forensic neuropsychology: Meeting challenges in the courtroom. New York: Plenum Press.
Manchester, D., Priestley, N., Jackson, H. (2004). The assessment of executive functions: Coming out of the office. Brain Injury, 18(11), 10671081.
Manly, T., Hawkins, K., Evans, J., Woldt, K., Robertson, I.H. (2002). Rehabilitation of executive function: Facilitation of effective goal management on complex tasks using periodic auditory alerts. Neuropsychologia, 40(3), 271281.
Matheis, R.J. (2004). Expanding the boundaries of neuropsychology: The application of VR for memory assessment. Doctoral Thesis, Fairleigh Dickinson University, New Jersey.
McGeorge, P., Phillips, L.H., Crawford, J.R., Garden, S.E., Della Sala, S., Milne, A.B. (2001). Using virtual environments in the assessment of executive dysfunction. Presence, 10(4), 375383.
Morganti, F. (2004). Virtual interaction in cognitive neuropsychology. In G. Riva, C. Botella, P. Legeron, & G. Optale (Eds.), Cybertherapy: Internet and virtual reality as assessment and rehabilitation tolls for clinical psychology and neuroscience. Amsterdam: IOS Press.
Moriyama, Y., Mimura, M., Kato, M., Yoshino, A., Hara, T., Kashima, H., Watanabe, A. (2002). Executive dysfunction and clinical outcome in chronic alcoholics. Alcoholism: Clinical and Experimental Research, 26(8), 12391244 .
Morris, R.G., Kotitsa, M., Bramham, J., Brooks, B.M., Rose, D. (2002). Virtual reality investigation of strategy formation, rule breaking and prospective memory in patients with focal prefrontal neurosurgical lesions. Paper presented at the International Conference of Disability, Virtual Reality & Associated Technology, Hungry.
Norris, G., Tate, R.L. (2000). The Behavioural Assessment of the Dysexecutive Syndrome (BADS): Ecological, concurrent and construct validity. Neuropsychological Rehabilitation, 10(1), 3345.
Ord, J.S., Greve, K.W., Bianchini, K.J., Aguerrevere, L.E. (2009). Executive dysfunction in traumatic brain injury: The effects of injury severity and effort on the Wisconsin Card Sorting Test. Journal of Clinical and Experimental Neuropsychology, 32(2), 132140.
Ponsford, J., Kinsella, G. (1991). The use of a rating scale of attentional behaviour. Neuropsychological Rehabilitation, 1(4), 241257.
Rand, D., Basha-Abu Rukan, S., Weiss, P.L., Katz, N. (2009). Validation of the Virtual MET as an assessment tool for executive functions. Neuropsychological Rehabilitation, 19(4), 583602.
Renison, B., Ponsford, J., Testa, R., Richardson, B., Brownfield, K. (2008). Virtual library task. Melbourne: Monash University.
Shallice, T., Burgess, P.W. (1991). Deficits in strategy application following frontal lobe damage in man. Brain, 114(2), 727741.
Shores, E.A., Marosszeky, J.E., Sandanam, J., Batchelor, J. (1986). Preliminary validation of a clinical scale for measuring the duration of posttraumatic amnesia. The Medical Journal of Australia, 144, 569572.
Spooner, D.M., Pachana, N.A. (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology, 21(4), 327337.
Stuss, D.T. (2007). New approaches to prefrontal lobe testing. In B. L. Miller & J. Cummings (Eds.), The human frontal lobes: Functions and disorders (2nd ed., pp. 292305). New York: Guilford Press.
Testa, R., Bennett, P.C., Ponsford, J. (2011). Factor Analysis of Nineteen Executive Function Tests in a Healthy Adult Population. Manuscript accepted for publication to Archives of Clinical Neuropsychology. In Press.
Titov, N., Knight, R.G. (2005). A computer-based procedure for assessing functional cognitive skills in patients with neurological injuries: The virtual street. Brain Injury, 19(5), 315322.
Tyson, P.J., Laws, K.R., Flowers, K.A., Mortimer, A.M., Schulz, J. (2008). Attention and executive function in people with schizophrenia: Relationship with social skills and quality of life. International Journal of Psychiatry in Clinical Practice, 12(2), 112119.
Wilson, B.A., Alderman, N., Burgess, P.W., Emslie, H., Evans, J.J. (1996). Behavioural assessment of the dysexecutive syndrome: Test manual. England: Thames Valley Test Company.
Wilson, B.A., Evans, J.J., Emslie, H., Alderman, N., Burgess, P.W. (1998). The development of an ecologically valid test for assessing patients with dysexecutive syndrome. Neuropsychological Rehabilitation, 8(3), 213228.
Zhang, L., Abreu, B.C., Seale, G.S., Masel, B., Christiansen, C.H., Ottenbacher, K.J. (2003). A virtual reality environment for evaluation of a daily living skill in brain injury rehabilitation: Reliability and validity. Archives of Physical Medicine and Rehabilitation, 84(8), 11181124.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of the International Neuropsychological Society
  • ISSN: 1355-6177
  • EISSN: 1469-7661
  • URL: /core/journals/journal-of-the-international-neuropsychological-society
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed