Skip to main content Accessibility help

Neuropsychological detection of cognitive impairment: Inter-rater agreement and factors affecting clinical decision-making



The agreement between neuropsychologists identifying cognitive impairment (CI) in older adults was examined, as were factors influencing the classification process. Twenty four neuropsychologists in 18 study centers classified cases with or without CI after reviewing neuropsychological findings and other relevant information. All cases were participants in the third wave of the Canadian Study of Health and Aging, a study of CI in later life. For 117 randomly selected cases, a second neuropsychologist reviewed the same material and reclassified the cases. Cases given the same (concordant) or different (discordant) classifications were compared with respect to patient and rater characteristics. The inter-rater agreement was moderate (77.7% agreement, kappa = .49). On all measures of cognitive functioning, the concordant group without impairment obtained a higher mean score than the discordant group, and the discordant group obtained a higher mean score than the concordant group with impairment. For 5 out of 8 cognitive measures, the concordant group with impairment differed from the concordant group without impairment and the discordant group, but the latter two groups did not differ significantly. The findings are comparable to others in the field and highlight the need for neuropsychologists to further clarify procedures for identifying subtle, or mild, forms of cognitive impairment. (JINS, 2006, 12, 72–79.)


Corresponding author

Reprint requests to: Holly A. Tuokko, Centre on Aging, Sedgewick Building Room A104, University of Victoria, PO Box 1700 STN CSC, Victoria, B.C., Canada V8W 2Y2. E-mail:


Hide All


Andresen, E.M., Malmgren, J.A., Carter, W.B., & Patrick, D.L. (1994). Screening for depression in well older adults: Evaluation of a short form of the CES-D. American Journal of Preventive Medicine, 10, 7784.
Baldereschi, M., Amato, M.P., Nencini, P., Pracucci, G., Lippi, A., Amaducci, L., Gauthier, S., Beatty, L., Quiroga, P., & Klassen, G. (1994). Cross-national interrater agreement on the clinical diagnostic criteria for dementia. WHO-PRA Age-associated Dementia Working group, WHO-program for Research on Aging, Health of Elderly Program. Neurology, 44, 239242.
Ballantyne, B.T., French, A.K., Heimsoth, S.L., Kachingwe, A.F., Lee, J.B., & Soderberg, G.L. (1995). Influence of examiner experience and gender on interrater reliability of KT-10000 arthrometer measurements. Physical Therapy, 75, 898906.
Brooks, D. & Thomas, J. (1995). Interrater reliability of auscultation of breath sounds among physical therapists. Physical Therapy, 75, 10821088.
Canadian Study of Health and Aging Working Group. (1994). Canadian Study of Health and Aging: Study methods and prevalence of dementia. Canadian Medical Association Journal, 150, 899913.
Fisher, N.J., Tierney, M.C., & Snow, W.G. (1999). Odd/even short forms of the Boston Naming Test: Preliminary geriatric norms. Clinical Neuropsychologist, 13, 359364.
Graham, J.E., Rockwood, K., Beattie, B.L., McDowell, I., Eastwood, R., & Gauthier, S. (1996). Standardization of the diagnosis of dementia in the Canadian Study of Health and Aging. Neuroepidemiology, 15, 246256.
Graham, J.E., Rockwood, K., Beattie, B.L., Eastwood, R., Gauthier, S., Tuokko, H., & McDowell, I. (1997). Prevalence and severity of cognitive impairment with and without dementia in an elderly population. Lancet, 349, 17931796.
Hogervorst, E., Barnetson, L., Jobst, K.A., Nagy, Z., Combrinck, M., & Smith, A.D. (2000). Diagnosing dementia: Interrater reliability assessment and accuracy of the NINCDS-ADRDA criteria versus CERAD histopathological criteria for Alzheimer's disease. Dementia and Geriatirc Cognitive Disorders, 11, 107113.
Jorm, A.F., Rodgers, B., Henderson, A.S., Korten, A.E., Jacomb, P.A., Christensen, H., & Mackinnon, A. (1998). Occupation type as a predictor of cognitive decline and dementia in old age. Age and Ageing, 27, 477483.
Kukull, W.A., Larson, E.B., Reifler, B.V., Lampe, T.H., Yerby, M., & Hughes, J. (1990). Interrater reliability of Alzhiemer's disease diagnosis. Neurology, 40, 257260.
Levy, R. (1994). Age-associated cognitive decline. Working Party of the International Psychogeriatric Association in collaboration with the World Health Organization. International Psychogeriatrics, 6, 6368.
Lopez, O.L., Swihart, A.A., Becker, J.T., Reinmuth, O.M., Reynolds, C.F., Rezek, D.L., & Daly, F.L. (1990). Reliability of the NINCDS-ADRDA clinical criteria for the diagnosis of Alzheimer's disease. Neurology, 40, 15171522.
Maruff, P., Collie, A., Darby, D., Weaver-Cargin, J., & Masters, C. (2004). Subtle memory decline over 12 months in mild cognitive impairment. Dementia and Geriatric Cognitive Disorders, 18, 342348.
O'Connor, D.W., Blessed, G., Cooper, B., Jonker, C., Morris, J.C., Presnell, I.B., Ames, D., Kay, D.W.K., Bickel, H., Schäufele, M., Wind, A., Coats, M., & Berg, L. (1996). Cross-national interrater reliability of dementia diagnosis in the elderly and factors associated with disagreement. Neurology, 47, 11941199.
O'Connor, D.W., Pollitt, P.A., & Treasure, F.P. (1991). The influence of education and social class on the diagnosis of dementia in a community population. Psychological Medicine, 21, 219224.
Petersen, R.C. (2003). Mild cognitive impairment: Aging to Alzheimer's disease. New York: Oxford University Press.
Petersen, R.C. (2004). Mild cognitive impairment as a diagnostic entity. Journal of Internal Medicine, 256, 183194.
Petersen, R.C., Smith, G.E., Waring, S.C., Ivnik, R.J., Tangalos, E.G., & Kokmen, E. (1999). Mild cognitive impairment. Archives of Neurology, 56, 303308.
Petersen, R.C., Stevens, J.C., Ganguli, M., Tangalos, E.G., Cummings, J.L., & DeKosky, S.T. (2001). Practice parameter: Early detection of dementia: Mild cognitive impairment (an evidence-based review). Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology, 56, 11331142.
Reitan, R.M. & Wolfson, D. (1996). Relationships of age and education to Wechsler Adult Intelligence Scale IQ values in brain-damaged and non-brain damaged groups. Clinical Neuropsychologist, 10, 293304.
Ritchie, K., Artero, S., & Touchon, J. (2001). Cognitive classification criteria for mild cognitive impairment: A population-based validation study. Neurology, 56, 3742.
Ritchie, K. & Touchon, J. (2000). Mild cognitive impairment: Conceptual basis and current nosological status. Lancet, 355, 225228.
Rosen, W.G. (1980). Verbal fluency in aging and dementia. Journal of Clinical Neuropsychology, 2, 135146.
Satz, P. & Mogel, S. (1962). An abbreviation of the WAIS for clinical use. Journal of Clinical Psychology, 18, 7779.
Shapiro, E. & Tate, R.B. (1997). The use and cost of community care services by elders with unimpaired cognitive function, with cognitive impairment/no dementia and with dementia. Canadian Journal on Aging, 16, 665681.
Solari, A., Cossa, F.M., Denes, F., Gainotti, G., Grossi, D., Nichelli, P., & Filippini, G. (1994). Agreement in clinical diagnosis of dementia: Evaluation of a case series with mild cognitive impairment. Neuroepidemiology, 13, 8996.
Spreen, O. & Benton, A.L. (1977). Neurosensory Center Comprehensive Examination for Aphasia: Manual of instructions. Victoria, B.C.: University of Victoria.
Steinhausen, H.C. & Erdin, A. (1991). The inter-rater reliability of child and adolescent psychiatric disorders in the ICD-10. Journal of Child Psychology and Psychiatry, 32, 921928.
Stern, Y., Gurland, B., Tatemichi, T.K., Tang, M.X., Wilder, D., & Mayeux, R. (1994). Influence of education and occupation on the incidence of Alzheimer's disease. Journal of the American Medical Association, 271, 10041010.
Stones, M.J., Kozma, A., Hirdes, J., Gold, D., Arbuckle, T., & Kolopack, P. (1996). Short Happiness and Affect Research Protocol (SHARP). Social Indicators Research, 37, 7591.
Teng, E.L. & Chui, H.C. (1987). The Modified Mini-Mental State (3MS) Examination. Journal of Clinical Psychiatry, 48, 314318.
Touchon, J. & Ritchie, K. (1999). Prodromal cognitive disorder in Alzheimer's disease. International Journal of Geriatric Psychiatry, 14, 556563.
Tuokko, H. & Frerichs, R. (2000). Cognitive impairment with no dementia (CIND): Longitudinal studies, the findings, and the issues. The Clinical Neuropsychologist, 14, 504525.
Tuokko, H., Frerichs, R., Graham, J., Rockwood, K., Kristjansson, B., Fisk, J., Bergman, H., Kozma, A., & McDowell, I. (2003). Five-year follow-up of cognitive impairment with no dementia. Archives of Neurology, 60, 577582.
Tuokko, H., Kristjansson, E., & Miller, J.A. (1995). Neuropsychological detection of dementia: An overview of the neuropsychological component of the Canadian Study of Health and Aging. Journal of Clinical and Experimental Neuropsychology, 17, 353373.
Tuokko, H., Vernon-Wilkinson, R., Weir, J., & Beattie, B.L. (1991). Cued recall and early identification of dementia. Journal of Clinical and Experimental Neuropsychology, 13, 871879.
Tuokko, H. & Woodward, T.S. (1996). Development and validation of a demographic correction system for neuropsychological measures used in the Canadian Study of Health and Aging. Journal of Clinical and Experimental Neuropsychology, 18, 479616.
Wechsler, D. (1975). Wechsler Memory Scale. New York: Psychological Corporation.
Wechsler, D. (1981). Wechsler Adult Intelligence Scale–Revised. New York: Psychological Corporation.
White, R.F., James, K.E., Vasterling, J.J., Marans, K., Delaney, R., Krengel, M., & Rose, F. (2002). Interrater reliability of neuropsychological diagnoses: A Department of Veterans Affairs cooperative study. Journal of the International Neuropsychological Society, 8, 555565.
Wilkinson, G.S. (1993). Wide Range Achievement Test (WRAT-3) administration manual. Wilmington, DE: Wide Range, Inc.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed