Skip to main content Accessibility help

Intra-individual Cognitive Variability: An Examination of ANAM4 TBI-MIL Simple Reaction Time Data from Service Members with and without Mild Traumatic Brain Injury

  • Wesley R. Cole (a1) (a2) (a3), Emma Gregory (a1) (a3), Jacques P. Arrieux (a1) (a2) (a3) and F. Jay Haran (a4)
  • Please note a retraction has been issued for this article.


Objectives: The Automated Neuropsychological Assessment Metrics 4 TBI-MIL (ANAM4) is a computerized cognitive test often used in post-concussion assessments with U.S. service members (SMs). Existing evidence, however, remains mixed regarding ANAM4’s ability to identify cognitive issues following mild traumatic brain injury (mTBI). Studies typically examine ANAM4 by comparing mean scores to baseline or normative scores. A more fine-grained approach involves examining inconsistency within an individual’s performance. Methods: Data from a sample of 231 were healthy control SMs and 100 SMs within 7 days of mTBI who took the ANAM4 were included in analyses. We examine each individual’s performance on a simple reaction time (SRT) subtest that is administered at the beginning (SRT1) and end (SRT2) of the ANAM4 battery, and calculate the standard deviation of difference scores by trial across administrations. Results: Multivariate analysis of variance and univariate analyses revealed group differences across all comparisons (p<.001) with pairwise comparisons revealing higher intra-individual variability and slower raw reaction time for the mTBI group compared with controls. Effect sizes were small though exceeded the recommended minimum practical effect size (ES>0.41). Conclusions: While inconsistencies in performance are often viewed as noise or test error, the results suggest intra-individual cognitive variability may be more sensitive than central tendency measures (i.e., comparison of means) in detecting changes in cognitive function in mTBI. Additionally, the findings highlight the utility of ANAM4’s repeating a subtest at two points in a battery to explore within-subject differences in performance. (JINS, 2017, 23, 1–6)


Corresponding author

Correspondence and reprint requests to: Wesley R. Cole, Intrepid Spirit, Womack Army Medical Center, Fort Bragg, NC 28310. E-mail:


Hide All
Adam, O., Mac Donald, C.L., Rivet, D., Ritter, J., May, T., Barefield, M., & Brody, D.L. (2015). Clinical and imaging assessment of acute combat mild traumatic brain injury in Afghanistan. Neurology, 85(3), 219227.
Arrieux, J.P., Cole, W.R., & Ahrens, A.P. (2017). A review of the validity of computerized neurocognitive assessment tools in mild traumatic brain injury assessment. Concussion, 2.
Bleiberg, J., Garmoe, W.S., Halpern, E.L., Reeves, D.L., & Nadler, J.D. (1997). Consistency of within-day and across-day performance after mild brain injury. Cognitive and Behavioral Neurology, 10(4), 247253.
Cognitive Science Research Center, CSRC. (2014). ANAM military battery, administration manual. Norman, OK: CSRC.
Cole, W.R., Arrieux, J.P., Dennison, E.M., & Ivins, B.J. (2017). The impact of administration order in studies of computerized neurocognitive assessment tools (NCATs). Journal of Clinical and Experimental Neuropsychology, 39(1), 3545.
Defense Health Board. (2016). Review of the scientific evidence of using population normative values for post-concussive computerized neurocognitive assessments. Report prepared for the Undersecretary of Personnel and Readiness, Falls Church, VA: DHB.
Delis, D.C., Kaplan, E., & Kramer, J.H. (2001). Delis-Kapn Executive Function System (D-KEFS) technical manual. San Antonio, TX: The Psychological Corporation.
Department of Defense Instruction (DoDI) 6490.13. (2013). Comprehensive policy on traumatic brain injury-related neurocogitive assessments by the military services. Washington, DC: Department of Defense.
Defense and Veterans Brain Injury Center. (2016). DoD worldwide numbers for TBI worldwide totals. Retrieved from
Fjell, A.M., Rosquist, H., & Walhovd, K.B. (2009). Instability in the latency of P3a/P3b brain potentials and cognitive function in aging. Neurobiology of Aging, 30(12), 20652079.
Friedl, K.E., Grate, S.J., Proctor, S.P., Ness, J.W., Lukey, B.J., & Kane, R.L. (2007). Army research needs for automated neuropsychologial tests: Monitoring soldier health and performance status. Archives of Clinical Neuropsychology, 22, 714.
Ferguson, C.J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532538.
Haran, F.J., Alphonso, A.L., Creason, A., Campbell, J.S., Johnson, D., Young, E., & Tsao, J.W. (2016). Reliable change estimates for assessing recovery from concussion using the ANAM4 TBI-MIL. The Journal of Head Trauma Rehabilitation, 31(5), 329338.
Haran, F.J., Dretsch, M.N., Slaboda, J.C., Johnson, D.E., Adam, O.R., & Tsao, J.W. (2016). Comparison of baseline-referenced versus norm-referenced analytical approaches for in-theatre assessment of mild traumatic brain injury neurocognitive impairment. Brain Injury, 30(3), 280286. doi: 10.3109/02699052.2015.1118766
Hill, B.D., Rohling, M.L., Boettcher, A.C., & Meyers, J.E. (2013). Cognitive intra-individual variability has a positive association with traumatic brain injury severity and suboptimal effort. Archives of Clinical Neuropsychology, 28(7), 640648.
Kelly, M.P., Coldren, R.L., Parish, R.V., Dretsch, M.N., & Russell, M.L. (2012). Assessment of acute concussion in the combat environment. Archives of Clinical Neuropsychology, 27(4), 375388.
Lovden, M., Li, S.C., Shing, Y.L., & Lindenberger, U. (2007). Within-person trial-to-trial variability precedes and predicts cognitive decline in old and very old age: Longitudinal data from the Berlin Aging Study. Neuropsychologia, 45, 28272838.
Makdissi, M., Collie, A., Maruff, P., Darby, D.G., Bush, A., McCrory, P., & Bennell, K. (2001). Computerised cognitive assessment of concussed Australian Rules footballers. British Journal of Sports Medicine, 35(5), 354360.
Meyer, J.E., & Arnett, P.A. (2015). Changes in symptoms in concussed and non-concussed athletes following neurochological assessment. Developmental Neuropsychology, 40(1), 2428.
McCrea, M., Pliskin, N., Barth, J., Cox, D., Fink, J., French, L., & Powell, M. (2008). Official position of the Military TBI Task Force on the Role of Neuropsychology and Rehabilitation Psychology in the Evaluation, Management, and Research of Military Veterans with Traumatic Brain Injury: APPROVED by: American Academy of Clinical Neuropsychology (AACN) American Psychological Association Division 40 (Neuropsychology) American Psychological Association Division 22 (Rehabilitation Psychology) National Academy of Neuropsychology (NAN). The Clinical Neuropsychologist, 22(1), 1026.
McCrory, P., Meeuwisse, W.H., Aubry, M., Cantu, B., Dvořák, J., Echemendia, R.J., & Sills, A. (2013). Consensus statement on concussion in sport: The 4th International Conference on Concussion in Sport held in Zurich, November 2012. British Journal of Sports Medicine, 47(5), 250–258.
Nelson, L.D., LaRoche, A.A., Pfaller, A.Y., Lerner, E.B., Hammeke, T.A., Randolph, C., & McCrea, M.A. (2016). Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): Reliability and validity for the assessment of sport-related concussion. Journal of the International Neuropsychological Society, 22(1), 2437.
Norris, J.N., Carr, W., Herzig, T., Labrie, D.W., & Sams, R. (2013). ANAM4 TBI reaction time-based tests have prognostic utility for acute concussion. Military Medicine, 178(7), 767774.
Parks, A.C., Moore, R.D., Wu, C.T., Broglio, S.P., Covassin, T., Hillman, C.H., & Pontifex, M.B. (2015). The association between a history of concussion and variability in behavioral and neuroelectric indices of cognition. International Journal of Psychophysiology, 98(3), 426434.
Rabinowitz, A.R., & Arnett, P.A. (2013). Intraindividual cognitive variability before and after sports-related concussion. Neuropsychology, 27(4), 481490.
Resch, J.E., McCrea, M.A., & Cullum, C.M. (2013). Computerized neurocognitive testing in the management of sport-related concussion: An update. Neuropsychology Review, 23(4), 335349.
Roebuck-Spencer, T.M., Vincent, A.S., Gilliland, K., Johnson, D.R., & Cooper, D.B. (2013). Initial clinical validation of an embedded performance validity measure within the automated neuropsychological metrics (ANAM). Archives of Clinical Neuropsychology, 28(7), 700710.
Segalowitz, S.J., Dywan, J., & Unsal, A. (1997). Attentional factors in response time variability after traumatic brain injury: An ERP study. Journal of the International Neuropsychological Society, 3(2), 95107.
Silverberg, N.D., & Iverson, G.L. (2013). Is rest after concussion “the best medicine?”: Recommendations for activity reumption following concussion in athletes, civilians, and military service members. The Journal of Head Trauma Rehabilitation, 28(4), 250259.
Sosnoff, J.J., Broglio, S.P., Hillman, C.H., & Ferrara, M.S. (2007). Concussion does not impact intraindividual response time variability. Neuropsychology, 21(6), 796802.
Stuss, D.T., Stethem, L.L., Hugenholtz, H., Picton, T., Pivik, J., & Richard, M.T. (1989). Reaction time after head injury: Fatigue, divided and focused attention, and consistency of performance. Journal of Neurology, Neurosurgery, & Psychiatry, 52(6), 742748.
Wechsler, D. (2008). Wechsler Adult Intelligence Scale 4th ed., San Antonio, TX: Psychological Corporation.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed

A retraction has been issued for this article: