Skip to main content Accessibility help

Multivariate Base Rates of Low Scores on Tests of Learning and Memory Among Latino Adult Populations

  • Diego Rivera (a1), Laiene Olabarrieta-Landa (a1), Brian L. Brooks (a2) (a3) (a4), Melissa M. Ertl (a5), Itziar Benito-Sánchez (a1) (a6), Maria Cristina Quijano (a7), Walter Rodriguez-Irizarry (a8), Adriana Aguayo Arelis (a9), Yaneth Rodríguez-Agudelo (a10) and Juan Carlos Arango-Lasprilla (a1) (a11)...



To determine the prevalence of low scores for two neuropsychological tests with five total scores that evaluate learning and memory functions.


N = 5402 healthy adults from 11 countries in Latin America and the commonwealth of Puerto Rico were administered the Rey–Osterrieth Complex Figure (ROCF) and the Hopkins Verbal Learning Test (HVLT-R). Two-thirds of the participants were women, and the average age was 53.5 ± 20.0 years. Z-scores were calculated for ROCF Copy and Memory scores and HVLT-R Total Recall, Delayed Recall, and Recognition scores, adjusting for age, age2, sex, education, and interaction variables if significant for the given country. Each Z-score was converted to a percentile for each of the five subtest scores. Each participant was categorized based on his/her number of low scoring tests in specific percentile cutoff groups (25th, 16th, 10th, 5th, and 2nd).


Between 57.3% (El Salvador) and 64.6% (Bolivia) of the sample scored below the 25th percentile on at least one of the five scores. Between 27.1% (El Salvador) and 33.9% (Puerto Rico) scored below the 10th percentile on at least one of the five subtests. Between 5.9% (Chile, El Salvador, Peru) and 10.3% (Argentina) scored below the 2nd percentile on at least one of the five scores.


Results are consistent with other studies that found that low scores are common when multiple neuropsychological outcomes are evaluated in healthy individuals. Clinicians should consider the higher probability of low scores when evaluating learning and memory using various sets of scores to reduce false-positive diagnoses of cognitive deficits.


Corresponding author

*Correspondence and reprint requests to: Juan Carlos Arango Lasprilla, Biocruces Bizkaia Health Research Institute, Cruces University Hospital, Plaza de Cruces s/n. 48903, Barakaldo, Spain; Ikerbasque, Basque Foundation for Science, Bilbao, Spain. E mail:


Hide All
Aiken, L.S., West, S.G., & Reno, R.R. (1991). Multiple Regression: Testing and Interpreting Interactions. Thousand Oaks, CA: Sage Publications, Inc.
Arango-Lasprilla, J.C., & Rivera, D. (2015). Neuropsicología en Colombia: Datos normativos, estado actual y retos a futuro [Neuropsychology in Colombia: Normative Data, Current State and Future Challenges]. Manizales, Colombia: Editorial Universidad Autónoma de Manizales.
Arango-Lasprilla, J.C., Rivera, D., Garza, M.T., Saracho, C.P., Rodríguez, W., Rodríguez-Agudelo, Y., … & Martínez, C. (2015). Hopkins verbal learning test-revised: Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation, 37(4), 699718.
Arango-Lasprilla, J.C., Stevens, L., Morlett Paredes, A., Ardila, A., & Rivera, D. (2017). Profession of neuropsychology in Latin America. Applied Neuropsychology: Adult, 24(4), 318330.
Ardila, A., & Rosselli, M. (1994). Development of language, memory, and visuospatial abilities in 5- to 12-year-old children using a neuropsychological battery. Developmental Neuropsychology, 10(2), 97120. doi: 10.1080/87565649409540571
Arrufat, A.S., Olmos, J.G., & Blanxart, M.F. (1999). Introducción a la estadística en Psicología, Vol. 27. Edicions Universitat Barcelona.
Benedict, R.H.B. (1997). Brief Visuospatial Memory Test—Revised. Odessa, FL: Psychological Assessment Resources, Inc.
Benedict, R.H.B., Schretlen, D., Groninger, L., & Brandt, J. (1998). Hopkins verbal learning test—revised: Normative data and analysis of inter-form and test–retest reliability. Clinical Neuropsychologist, 12(1), 4355. doi: 10.1076/clin.
Berry, D.T., Allen, R.S., & Schmitt, F.A. (1991). Rey-Osterrieth complex figure: Psychometric characteristics in a geriatric sample. The Clinical Neuropsychologist, 5(2), 143153. doi: 10.1080/13854049108403298
Binder, L.M., Iverson, G.L., & Brooks, B.L. (2009). To err is human: “Abnormal” neuropsychological scores and variability are common in healthy adults. Archives of Clinical Neuropsychology, 24(1), 3146. doi: 10.1093/arclin/acn001
Brandt, J. (1991). The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. The Clinical Neuropsychologist, 5(2), 125142. doi: 10.1080/13854049108403297
Brandt, J., & Benedict, R.H. (2001). Hopkins Verbal Learning Test-Revised: Professional Manual. Odessa, FL: Psychological Assessment Resources.
Brooks, B.L., Holdnack, J.A., & Iverson, G.L. (2011). Advanced clinical interpretation of the WAIS-IV and WMS-IV: Prevalence of low scores varies by level of intelligence and years of education. Assessment, 18, 156167. doi: 10.1177/1073191110385316
Brooks, B.L., Holdnack, J.A., & Iverson, G.L. (2017). Reliable change on memory tests is common in healthy children and adolescents. Archives of Clinical Neuropsychology, 32(8), 10011009.
Brooks, B.L. & Iverson, G.L. (2010). Comparing actual to estimated base rates of “abnormal” scores on neuropsychological test batteries: Implications for interpretation. Archives of Clinical Neuropsychology, 25(1), 1421. doi: 10.1093/arclin/acp100
Brooks, B.L., Iverson, G.L., Feldman, H.H., & Holdnack, J.A. (2009). Minimizing misdiagnosis: Psychometric criteria for determining possible or probable memory impairment. Dementia and Geriatric Cognitive Disorders, 27, 439450. doi: 10.1159/000215390
Brooks, B.L., Iverson, G.L., & Holdnack, J.A. (2013). Understanding multivariate base rates, In Holdnack, J.A., Drozdick, L., Weiss, L.G., & Iverson, G.L.(Eds.), WAIS-IV, WMS-IV, & ACS: Clinical Use and Interpretation (pp. 75102). New York: Elsevier. doi: 10.1016/b978-0-12-386934-0.00002-x
Brooks, B.L., Iverson, G.L., Holdnack, J.A., & Feldman, H.H. (2008). The potential for misclassification of mild cognitive impairment: A study of memory scores on the Wechsler Memory Scale-III in healthy older adults. Journal of the International Neuropsychological Society, 14(3), 463478. doi: 10.1017/s1355617708080521
Brooks, B.L., Iverson, G.L., & White, T. (2007). Substantial risk of “accidental MCI” in healthy older adults: Base rates of low memory scores in neuropsychological assessment. Journal of the International Neuropsychological Society, 13, 490500. doi: 10.1017/s1355617707070531
Brooks, B.L., Iverson, G.L., & White, T. (2009). Advanced interpretation of the Neuropsychological Assessment Battery (NAB) with older adults: Base rate analyses, discrepancy scores, and interpreting change. Archives of Clinical Neuropsychology, 24, 647657. doi: 10.1093/arclin/acp061
Brooks, B.L., Sherman, E.M., & Iverson, G.L. (2010). Healthy children get low scores too: Prevalence of low scores on the NEPSY-II in preschoolers, children, and adolescents. Archives of Clinical Neuropsychology, 25(3), 182190. doi: 10.1093/arclin/acq005
Brooks, B.L., Strauss, E., Sherman, E., Iverson, G.L., & Slick, D.J. (2009). Developments in neuropsychological assessment: Refining psychometric and clinical interpretive methods. Canadian Psychology/Psychologie Canadienne, 50(3), 196209. doi: 10.1037/a0016066
Buschke, H. (1973). Selective reminding for analysis of memory and learning. Journal of Verbal Learning and Verbal Behavior, 12(5), 543550.
Buschke, H., & Fuld, P.A. (1974). Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology, 24(11), 10191019.
Cherner, M., Suarez, P., Lazzaretto, D., Fortuny, L.A., Mindt, M.R., Dawes, S., … HNRC group. (2007). Demographically corrected norms for the Brief Visuospatial Memory Test-revised and Hopkins Verbal Learning Test-revised in monolingual Spanish speakers from the US–Mexico border region. Archives of Clinical Neuropsychology, 22(3), 343353. doi: 10.1016/j.acn.2007.01.009
Cysique, L.A., Jin, H., Franklin, D.R., Morgan, E.E., Shi, C., Yu, X., … Ake, C. (2007). Neurobehavioral effects of HIV-1 infection in China and the United States: A pilot study. Journal of the International Neuropsychological Society, 13(5), 781790. doi: 10.1017/s1355617707071007
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A. (1987). CVLT: California Verbal Learning Test-Adult Version: Manual. Psychological Corporation.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A. (2000). CVLT-II: California Verbal Learning Test: Adult Version. Psychological Corporation.
Fastenau, P.S. (1996). Development and preliminary standardization of the “Extended Complex Figure Test” (ECFT). Journal of Clinical and Experimental Neuropsychology, 18(1), 6376. doi: 10.1080/01688639608408263
Fastenau, P.S., Denburg, N.L., & Hufford, B.J. (1999). Adult norms for the Rey-Osterrieth Complex Figure Test and for supplemental recognition and matching trials from the Extended Complex Figure Test. The Clinical Neuropsychologist, 13(1), 3047. doi: 10.1076/clin.
Fernando, K., Chard, L., Butcher, M., & McKay, C. (2003). Standardization of the Rey Complex Figure Test in New Zealand children and adolescents. New Zealand Journal of Psychology, 32(1), 3338.
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). “Minimental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189198. doi: 10.1016/0022-3956(75)90026-6
Galindo, G. & Cortes, J.F. (2003). The ROCF and the complex figure for children in Spanish speaking populations. In J.Knight, A. &Kaplan, E. (Eds.), The Handbook of Rey-Osterrieth Complex Figure Usage: Clinical and Research Applications. Lutz, FL: Psychological Assessment Resources.
Guàrdia, J., Jarne, A., Pena-Casanova, J., & Gil, D. (2005). Análisis de resultados y proceso de normalización [Analysis of Results and Process of Normalization]. Barcelona Test-Revised. Barcelona: Masson.
Guàrdia-Olmos, J., Peró-Cebollero, M., Rivera, D., & Arango-Lasprilla, J.C. (2015a). Methodology for the development of normative data for ten Spanish-language neuropsychological tests in eleven Latin American countries. NeuroRehabilitation, 37(4), 493499. doi: 10.3233/nre-151277
Guàrdia-Olmos, J., Rivera, D., Peró-Cebollero, M., & Arango-Lasprilla, J.C. (2015b). Metodología para la creación de datos normativos para pruebas neuropsicológicas en población Colombiana. In Arango-Lasprilla, J.C. &Rivera, D. (Eds.), Neuropsicología en Colombia: Datos normativos, estado actual y retos a futuro (pp. 4780). Manizales, Colombia: Editorial Universidad Autónoma de Manizales.
Hoare, J., Westgarth-Taylor, J., Fouche, J.P., Combrinck, M., Spottiswoode, B., Stein, D.J., & Joska, J.A. (2012). Relationship between apolipoprotein E4 genotype and white matter integrity in HIV-positive youth adults in South Africa. European Archives of Psychiatry and Clinical Neuroscience, 263(3), 189195. doi: 10.1007/s00406-012-0341-8
Holdnack, J.A., Tulsky, D.S., Brooks, B.L., Slotkin, J., Gershon, R., Heinemann, A.W., & Iverson, G.L. (2017). Interpreting patterns of low scores on the NIH toolbox cognition battery. Archives of Clinical Neuropsychology, 32(5), 574584.
IBM Corp. (2015). SPSS Version 23.0. Armonk, NY: IBM Corp.
Kanmogne, G.D., Kuate, C.T., Cysique, L.A., Fonsah, J.Y., Eta, S., Doh, R., … McCutchan, J.A. (2010). HIV-associated neurocognitive disorders in sub-Saharan Africa: A pilot study in Cameroon. BMC Neurology, 10(1), 60. doi: 10.1186/1471-2377-10-60
Kroenke, K., Spitzer, R.L., & Williams, J.B. (2001). The PHQ-9. Journal of General Internal Medicine, 16(9), 606613. doi: 10.1046/j.1525-1497.2001.016009606.x
Kuslansky, G., Katz, M., Verghese, J., Hall, C.B., Lapuerta, P., LaRuffa, G., & Lipton, R.B. (2004). Detecting dementia with the Hopkins verbal learning test and the mini-mental state examination. Archives of Clinical Neuropsychology, 19(1), 89104. doi: 10.1016/s0887-6177(02)00217-2
Kutner, M.H., Nachtsheim, C.J., Neter, J., & Li, W. (2005). Applied Linear Statistical Models (5th ed.). New York: McGraw Hill.
Mahoney, F.I. & Barthel, D. (1965). Functional evaluation: The Barthel Index. Maryland State Medical Journal, 14, 5661.
Oltra-Cucarella, J., Sánchez-SanSegundo, M., Lipnicki, D.M., Sachdev, P.S., Crawford, J.D., Pérez-Vicente, J.A., … Alzheimer’s Disease Neuroimaging Initiative. (2018). Using base rate of low scores to identify progression from Amnestic Mild Cognitive Impairment to Alzheimer’s Disease. Journal of the American Geriatrics Society, 66(7), 13601366.
Osterrieth, P.A. (1944). Le test de copie d’une figure complexe; contribution à l’étude de la perception et de la mémoire [Test of copying a complex figure; contribution to the study of perception and memory]. Archives de Psychologie, 30, 206356.
Peña-Casanova, J., Blesa, R., Aguilar, M., Gramunt-Fombuena, N., Gómez-Ansón, B., Oliva, R., … Martínez-Parra, C. (2009). Spanish multicenter normative studies (NEURONORMA project): Methods and sample characteristics. Archives of Clinical Neuropsychology, 24(4), 307319.
Rey, A. (1941). L’examen psychologique dans les cas d’encéphalopathie traumatique (les problems). Archives de Psychologie, 28, 286340.
Rey, A. (1958). L’examen clinique en psychologie. Paris: Presse Universitaire de France.
Rey, A. (2009). REY: Test de copia y de reproducción de memoria de figuras geométricas complejas. Madrid: TEA Ediciones.
Rivera, D. & Arango-Lasprilla, J.C. (2017). Methodology for the development of normative data for Spanish-speaking pediatric populations. NeuroRehabilitation, 41(3), 581592. doi: 10.3233/nre-172275
Rivera, D., Olivera Plaza, S.L., Quijano, M.C., Calderón Chagualá, J.A., De los Reyes Aragón, C.J., Utria Rodríguez, O.E., … Arango-Lasprilla, J.C. (2015b). Datos normativos del test de aprendizaje verbal de Hopkins—Revisado para población Colombiana. In Arango-Lasprilla, J.C. & Rivera, D. (Eds.), Neuropsicología en Colombia: Datos normativos, estado actual y retos a futuro (pp. 239252). Manizales, Colombia: Editorial Universidad Autónoma de Manizales.
Rivera, D., Perrin, P.B., Morlett-Paredes, A., Galarza-del-Angel, J., Martinez, C., Garza, M.T., … Aliaga, A. (2015a). Rey–Osterrieth complex figure–copy and immediate recall: Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation, 37(4), 677698. doi: 10.3233/nre-151285
Schretlen, D.J., Testa, S.M., Winicki, J.M., Pearlson, G.D., & Gordon, B. (2008). Frequency and bases of abnormal performance by healthy adults on neuropsychological testing. Journal of the International Neuropsychological Society, 14(3), 436445. doi: 10.1017/s1355617708080387
Shapiro, A.M., Benedict, R.H., Schretlen, D., & Brandt, J. (1999). Construct and concurrent validity of the Hopkins Verbal Learning Test–Revised. The Clinical Neuropsychologist, 13(3), 348358. doi: 10.1076/clin.13.3.348.1749
Sherman, E.M.S. & Brooks, B.L. (2015). Child and adolescent memory profile (ChAMP). Lutz, Florida: Psychological Assessment Resources.
Sherman, E.M.S. & Brooks, B.L. (2017). Multidimensional Everyday Memory Ratings for Youth (MEMRY). Lutz, Florida: Psychological Assessment Resources.
Shimamura, A.P., Salmon, D.P., Squire, L.R., & Butters, N. (1987). Memory dysfunction and word priming in dementia and amnesia. Behavioral Neuroscience, 101(3), 347351.
Shin, M.S., Park, S.Y., Park, S.R., Seol, S.H., & Kwon, J.S. (2006). Clinical and empirical applications of the Rey-Osterrieth complex figure test. Nature Protocols, 1(2), 892899. doi: 10.1038/nprot.2006.115
Silverstein, S.M., Osborn, L.M., & Palumbo, D.R. (1998). Rey-Osterrieth complex figure test perfomance in acute, chronic, and remitted schizophrenia patients. Journal of Clinical Psychology, 54(7), 985994.
Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary. New York: Oxford University.
Tierney, M.C., Nores, A.N., Snow, W.G., Fisher, R.H., Zorzitto, M.L., & Reid, D.W. (1994). Use of the key auditory verbal learning test in differentiating normal aging from Alzheimer’s and Parkinson’s Dementia. Psychological Assessment, 6(2), 129134.
Trahan, D.E. & Larrabee, G.J. (1988). Continuous Visual Memory Test: Professional Manual. Psychological Assessment Resources.
Van Der Elst, W., Van Boxtel, M.P., Van Breukelen, G.J., & Jolles, J. (2006a). Normative data for the animal, profession and letter M naming verbal fluency tests for Dutch speaking participants and the effects of age, education, and sex. Journal of the International Neuropsychological Society, 12(1), 8089. doi: 10.1017/s1355617706060115
Van Der Elst, W., Van Boxtel, M.P., Van Breukelen, G.J., & Jolles, J. (2006b). The letter digit substitution test: Normative data for 1, 858 healthy participants aged 24-81 from the Maastricht Aging Study (MAAS): Influence of age, education, and sex. Journal of Clinical and Experimental Neuropsychology, 28(6), 9981009. doi: 10.1080/13803390591004428
Villaseñor-Cabrera, T., Guàrdia-Olmos, J., Jiménez-Maldonado, M., Rizo-Curiel, G., & Peró-Cebollero, M. (2010). Sensitivity and specificity of the Mini-Mental State Examination in the Mexican population. Quality & Quantity, 44(6), 11051112. doi: 10.1007/s11135-009-9263-6
Vogel, A., Stokholm, J., & Jorgensen, K. (2012). Performances on Rey Auditory Verbal Learning Test and Rey Complex Figure Test in a healthy, elderly Danish sample: Reference data and validity issues. Scandinavian Journal of Psychology, 53(1), 2631. doi: 10.1111/j.1467-9450.2011.00909.x
Waber, D.P. & Holmes, J.M. (1985). Assessing children’s copy productions of the Rey-Osterrieth Complex Figure. Journal of Clinical and Experimental Neuropsychology, 7(3), 264280. doi: 10.1080/01688638508401259
Wechsler, D. (2009). Advanced Clinical Solutions for the WAIS-IV and WMS-IV. San Antonio, TX: The Psychological Corporation.
Yepthomi, T., Paul, R., Vallabhaneni, S., Kumarasamy, N., Tate, D.F., Solomon, S., & Flanigan, T. (2006). Neurocognitive consequences of HIV in southern India: A preliminary study of clade C virus. Journal of the International Neuropsychological Society, 12(3), 424430. doi: 10.1017/s1355617706060516


Type Description Title
Supplementary materials

Rivera et al. supplementary material
Rivera et al. supplementary material 1

 Unknown (191 KB)
191 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed