The prevalence of ADHD diagnoses more than doubled in VA settings between 2009 and 2016 (Hale et al., 2020). However, attentional difficulties are not exclusive to ADHD and can also be seen in non-neurodevelopmental disorders, including depression, anxiety, substance use, and PTSD (Marshall et al., 2018, Suhr et al., 2008). Further, patients can easily feign symptoms of ADHD with few available instruments for accurate detection (Robinson & Rogers, 2018). Given the significant symptom overlap and rising rates of reported ADHD among Veterans, accurate detection of feigned ADHD is essential.

This study examined the utility of the experimental Dissimulation ADHD scale (Ds-ADHD; Robinson & Rogers, 2018) on the MMPI-2, in detecting feigned ADHD presentation within a mixed sample of Veterans.

In this retrospective study, 173 Veterans (Mage = 36.18, SDage = 11.10, Medu = 14.01, SDedu = 2.11, 88% male, 81% White, and 17% Black) were referred for neuropsychological evaluation of ADHD that included the MMPI-2 and up to 10 PVTs. Participants were assigned to a credible group (n=146) if they passed all PVTs or a non-credible group (n=27) if they failed two or more PVTs. Group assignment was also clinically confirmed. The Ds-ADHD was used to differentiate groups who either had credible or non-credible performance on cognitive measures. Consistent with Robinson and Rogers’ study, “true” answers (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded as 2, creating a 10- to 20-point scale. Lower scores were associated with a higher likelihood of a feigned ADHD presentation.

Preliminary analyses revealed no significant group differences in age, education, race, or gender (ps > .05). An ANOVA indicated a significant difference between groups (F[1, 171] = 10.44, p = .001; Cohen’s d = .68) for Ds-ADHD raw scores; Veterans in the non-credible group reported more “erroneous stereotypes” of ADHD (M raw score = 13.33, SD = 2.20) than those in the credible group (M = 14.82, SD = 2.20). A ROC analysis indicated AUC of .691 (95% CI = .58 to .80). In addition, a cut score of <12 resulted in specificity of 91.8% and sensitivity of 18.5%, whereas a cut score of <13 resulted in specificity of 83.6% and sensitivity of 44.4%.

The Ds-ADHD scale demonstrated significant differences between credible and non-credible respondents in a real-world setting. Previously, this scale has primarily been studied within laboratory settings. Further, results indicate a cut score of <12 could be used in order to achieve adequate specificity (i.e., >90%), which were similar findings to a study examining SVT-based groups (Winiarski et al., 2023). These results differ slightly from prior research by Robinson and Rogers (2018), who indicated a cut score of <13 based on the initial simulation-based study. In similar clinical settings, where there are high rates of psychiatric comorbidity, a cut score of <12 may prove clinically useful. However, this cut-score was associated with low sensitivity within this mixed Veteran sample. Further research should focus on replicating findings within other clinical settings, including ones with larger non-credible samples.

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) is one of the most widely used measures in neuropsychological assessment. Studies of practice effects on the RBANS have largely been limited to studies assessing one or two repeated assessments. The aim of the current study is to examine practice effects across four years after baseline in a longitudinal study of cognitively healthy older adults. Practice effects were estimated using a pseudo-replacement participants approach which has been previously applied in other aging studies.

453 Participants from the Louisiana Aging Brain Study (LABrainS) completed the RBANS Form A on up to four annual assessments after a baseline evaluation. Practice effects were calculated using a modified participants-replacement method where scores of individuals who were administered RBANS Form A multiple times are compared to the baseline scores of matched participants with additional adjustment for attrition effects.

Practice effects were observed primarily in the immediate memory, delayed memory, and total score indices. For example, an increase of nearly half a standard deviation was observed for delayed memory.

These findings extend past work on the RBANS and other neuropsychological batteries more broadly in showing the susceptibility of memory measures to practice effects. Given that memory and total score indices of the RBANS have the most robust relationships with diagnostic status and biomarkers for pathological cognitive decline, these findings raise concerns about the ability to recruit those at risk for decline from longitudinal studies using the same form of the RBANS for multiple years.

Inconsistent relationships between subjective and objective performance have been found across various clinical groups. Discrepancies in these relationships across studies have been attributed to various factors such as patient characteristics (e.g., level of insight associated with cognitive impairment) and test characteristics (e.g., using too few measures to assess different cognitive domains). Although performance and symptom invalidity are common in clinical and research settings and have the potential to impact responding on testing, previous studies have not explored the role of performance and symptom invalidity on relationships between objective and subjective performance. Therefore, the current study examined the impact of invalidity on performance and symptom validity tests (PVTs and SVTs, respectively) on the relationship between subjective and objective cognitive functioning.

Data were obtained from 299 Veterans (77.6% male, mean age of 48.8 years (SD = 13.5)) assessed in a VA medical center epilepsy monitoring unit from 2008-2018. Participants completed a measure of subjective functioning (i.e., the Patient Competency Rating Scale), PVTs (i.e., Word Memory Test, Test of Memory Malingering, Reliable Digit Span), SVTs (i.e., Minnesota Multiphasic Personality Inventory-2-Restructured Form Response Bias Scale, Structured Inventory of Malingered Symptomatology), and neuropsychological measures assessing objective cognitive performance (e.g., Trail Making Test parts A and B). Pearson correlations were conducted between subjective functioning and objective cognitive performance in the following groups: 1.) PVT and SVT valid, 2.) PVT and SVT invalid, 3.) PVT-only invalid, 4.) SVT-only invalid. Using Fisher’s r-to-z transformation, tests for the differences between correlation coefficients were then conducted between the PVT and SVT valid vs. PVT and SVT invalid groups, and the PVT-only invalid vs. SVT-only invalid groups.

Participants with fully valid PVT and SVT performances demonstrated generally stronger relationships between subjective and objective scores (r’s = .058 - .310) compared to participants with both invalid PVT and SVT scores (r’s = -.033 - .132). However, the only significant difference in the strengths of correlations between the groups was found on Trail Making Test Part B (p = .034). In separate exploratory analyses due to low group size, those with invalid PVT scores only (fully valid SVT) demonstrated generally stronger relationships between subjective and objective scores (r’s = -.101 - .741) compared to participants with invalid SVT scores only (fully valid PVT; r’s = -.088 - .024). However, the only significant difference in the strengths of correlations between the groups was found on Trail Making Test Part A (p = .028).

The present study suggests that at least some of the discrepancies in previous studies between subjective and objective cognitive performance may be related to performance and symptom validity. Specifically, very weak relationships between objective and subjective performance were found in participants who only failed SVTs, whereas relationships were stronger in those who only failed PVTs. Therefore, findings suggest that including measures of PVTs and SVTs in future studies investigating relationships between subjective and objective cognitive performance is critical to ensuring accuracy of conclusions that are drawn.

Workload is a useful construct in human factors and neuroergonomics research that describes “the perceived relationship between the amount of mental [and physical] processing capability or resources and the amount required by the task”. We apply this concept to neuropsychology and assess several dimensions of workload as it relates to performance on the Trail Making Test.

Twenty college students completed the Trail Making Test (TMT). After completion of each Part A and B, workload was assessed with the NASA-Task Load Index (NASA-TLX), a popular self-report measure of workload including subscales: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration, with an overall average total score as well.

Completion time differed of course between Parts A and B (p < .001). Of more interest, overall workload differed between TMT A (M = 20.33, SD = 13.32) and TMT B (M = 35.79, SD = 17.37) (p < .001, h2 = .68). The greatest subscale differences were with Mental Demand (p < .001, h2 = .68) and Effort (p < .001, h2 = .59), but Physical Demand also showed a difference (p < .007, h2 = .33). Temporal Demand showed the smallest and nonsignificant difference (p = .081, h2 = .152).

Based on previous research in our lab, most results were expected and understandable. As we know with the TMT, Part B is more cognitively demanding (in various ways) than Part A. The greater Physical Demand with Part B is a somewhat more complex finding, needing a solid explanation. Finally, the NASA-TLX appears to be a valid instrument of workload with a standard neuropsychologist test. We argue it can provide useful interesting information in the assessment of cognitive status in clinical populations.

Adverse childhood experiences (ACEs) is a broad construct that refers to negative events one may experience during childhood, including, but not limited to, childhood maltreatment, household dysfunction, and trauma. ACEs have consistently shown to be associated with negative physical and mental health outcomes. Although researchers have investigated the effects of trauma and abuse on personality measures, few studies have examined differences between those with high ACEs, low ACEs, and no ACEs on measures of personality in the context of neuropsychological evaluations.

The current study included 128 consecutive adult patients referred for outpatient neuropsychological evaluation of attention-deficit/hyperactivity disorder. The sample was 39.8% non-Hispanic White, 21.9% non-Hispanic Black, 16.4% Hispanic, 10.9% Asian/Pacific Islander, and 10.9% other race/ethnicity, with a mean age of 27.9 years (SD=6.3) and mean education of 16.1 years (SD=2.2). Multivariate analyses of variance were performed to evaluate differences on the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) between individuals who experienced high levels of ACEs (>4/10 on the Adverse Childhood Experiences Questionnaire), low levels of ACEs (1-3/10), and no ACEs (0/10).

When analyzing Higher-Order (H-O) scales, there was a significant group difference in mean elevation on the Behavioral/Externalizing Dysfunction (BXD) scale, F(2,113)=3.124, p < .05, such that individuals in the high ACEs group evidenced higher scores than those in the low ACEs group (p < .05). Additionally, there were significant differences on several Restructured Clinical (RC) scales. Specifically, there were group differences on the Low Positive Emotions (RC2) scale, F(2,113)=3.427, p < .05, such that those in the low ACEs group evidenced higher scores than those in the no ACEs group (p < .05). The Antisocial Behavior (RC4) scale also had significant differences, F(2,113)=13.703, p < .001, such that those in the high ACEs group had higher scores than those in the low and no ACEs groups (p < .001). Finally, the Ideas of Persecution (RC6) scale yielded significant group differences, F(2,113)=4.793, p < .05, such that those in the high ACEs group evidenced higher scores than those in the low and no ACEs groups (p < .05).

In sum, this study demonstrated that ACEs, particularly high levels of ACEs, are related to higher difficulties with problems with under-controlled and rule-breaking behaviors, low positive emotional responses, and beliefs that others pose a threat. As such, assessment of ACEs may serve an important role in characterizing patients’ psychological status as part of a comprehensive neuropsychological evaluation.

Previous research has found that subjective cognitive decline corresponds with assessed memory impairment and could even be predictive of neurocognitive impairment. The purpose of this study was to investigate whether a single self-report item of subjective cognitive decline corresponds with the results of a performance-based measure of episodic memory.

Older adults (n = 100; age 60-90) were given the single item measure of subjective cognitive decline developed by Verfaille et al. (2018).

Those who endorsed subjective cognitive decline (n = 68) had lower scores on the CVLT-II long delay free recall than those who did not endorse such a decline (n = 32). Additionally, older adults with a neurocognitive diagnosis believed their memory was becoming worse at a higher proportion than those without.

While a single item of subjective cognitive decline should not be substituted for a comprehensive evaluation of memory, the results suggest that it may have utility as a screening item.

Visual deficits are common after stroke and are powerful predictors for the chronic functional outcome. However, while basic visual field and recognition deficits are relatively easy to assess with standardized methods, selective deficits in visual primitives, such as shape or motion, are harder to identify, as they often require a symmetrical bilateral posterior lesion in order to provoke full field deficits. We aimed to investigate the prevalence and co-occurrence of hemifield “mid-range” visual deficits. In addition, we looked at the repercussions of these mid-range deficits on higher-order visual cognitive functions, such as visuoconstruction and memory. At a more theoretical level, we investigated whether associations between deficits in 'mid-range’ visual functions and deficits in higher-order visual cognitive functions are in line with a hierarchical, two-pathway model of the visual brain.

In 220 stroke patients and a healthy control group (N=49), we assessed the perception of colour (isoluminant stimuli in the red-green range), shape (Efron shapes), location (dot in a circle), orientation (lines at different angles), contrast (bars with converging grey-level differences), texture (from Brodatz grayscale texture album) and correlated motion (different percentages of dots moving in the same direction). All tasks started with a fixation dot presented at the centre of the screen. After one second, a target stimulus was presented on the horizontal midline at either 5° to the left or at 5° to the right side of the fixation. Then, after 1.5 seconds, two response items appeared in addition to the target stimulus for three seconds. To control for eye movements, we used an eye-tracker to present the target in a gaze contingent fashion. Thus, the target always remained in the correct retinal position independent of eye movements. In a subset of 182 ischemic stroke patients, we also assessed visuoconstruction (Copy Rey-Complex Figure Test), visual emotion recognition (FEEST test) and visual memory (Doors-test).

The results showed that deficits in motion-perception were most prevalent (26%), followed by colour (22%), texture (22%), location (21%), orientation (18%), contrast (14%), shape (14%) and glossiness (13%). 63% of the stroke patients showed one or more mid-range visual deficits. Overlap of deficits was small; they mostly occurred in isolation or co-occurred with only one or two other deficits. Impairments in mid-range visual functions could not predict performance on higher-order visual cognitive tasks. Impaired visuoconstruction and visual memory were only modestly predicted by a worse location perception. Impaired emotion perception was modestly predicted by a worse orientation perception. In addition, double dissociations were found: there were patients with selective deficits in 'mid-range’ visual functions without higher-order visual deficits and vice versa.

First, deficits in “mid-range” visual functions are very prevalent. Since we found no strong patterns of co-occurrences, we suggest that an assessment of deficits at this level of visual processing requires screening the full range of visual functions. Second, the relationship between mid-range visual tasks and higher-order visual cognitive tasks is weak. Finally, our findings are not supportive of the hierarchical, two-pathway model but more in line with an alternative patchwork model.

As part of the Research Domain Criteria (RDoC) initiative, the NIMH seeks to improve experimental measures of cognitive and positive valence systems for use in intervention research. However, many RDoC tasks have not been psychometrically evaluated as a battery of measures. Our aim was to examine the factor structure of 7 such tasks chosen for their relevance to schizophrenia and other forms of serious mental illness. These include the n-back, Sternberg, and self-ordered pointing tasks (measures of the RDoC cognitive systems working memory construct); flanker and continuous performance tasks (measures of the RDoC cognitive systems cognitive control construct); and probabilistic learning and effort expenditure for reward tasks (measures of reward learning and reward valuation constructs).

The sample comprised 286 cognitively healthy participants who completed novel versions of all 7 tasks via an online recruitment platform, Prolific, in the summer of 2022. The mean age of participants was 38.6 years (SD = 14.5, range 18-74), 52% identified as female, and stratified recruitment ensured an ethnoracially diverse sample. Excluding time for instructions and practice, each task lasted approximately 6 minutes. Task order was randomized. We estimated optimal scores from each task including signal detection d-prime measures for the n-back, Sternberg, and continuous performance task, mean accuracy for the flanker task, win-stay to win-shift ratio for the probabilistic learning task, and trials completed for the effort expenditure for reward task. We used parallel analysis and a scree plot to determine the number of latent factors measured by the 7 task scores. Exploratory factor analysis with oblimin (oblique) rotation was used to examine the factor loading matrix.

The scree plot and parallel analyses of the 7 task scores suggested three primary factors. The flanker and continuous performance task both strongly loaded onto the first factor, suggesting that these measures are strong indicators of cognitive control. The n-back, Sternberg, and self-ordered pointing tasks strongly loaded onto the second factor, suggesting that these measures are strong indicators of working memory. The probabilistic learning task solely loaded onto the third factor, suggesting that it is an independent indicator of reinforcement learning. Finally, the effort expenditure for reward task modestly loaded onto the second but not the first and third factors, suggesting that effort is most strongly related to working memory.

Our aim was to examine the factor structure of 7 RDoC tasks. Results support the RDoC suggestion of independent cognitive control, working memory, and reinforcement learning. However, effort is a factorially complex construct that is not uniquely or even most strongly related to positive valance. Thus, there is reason to believe that the use of at least 6 of these tasks are appropriate measures of constructs such as working memory, reinforcement learning and cognitive control.

Assessment of medication management, an instrumental activity of daily living (IADL), is particularly important among Veterans, who are prescribed an average of 2540 prescriptions per year (Nguyen et al., 2017). The Pillbox Test (PT) is a brief, performance-based measure that was designed as an ecologically valid measure of executive functioning (EF; Zartman, Hilsabeck, Guarnaccia, & Houtz, 2013), the cognitive domain most predictive of successful medication schedule management (Suchy, Ziemnik, Niermeyer, & Brothers, 2020). However, a validation study by Logue, Marceaux, Balldin, and Hilsabeck (2015) found that EF predicted performance on the PT more so than processing speed (PS), but not the language, attention, visuospatial, and memory domains combined. Thus, this project sought to increase generalizability of the latter study by replicating and extending their investigation utilizing a larger set of neuropsychological tests.

Participants included 176 patients in a mixed clinical sample (5.1% female, 43.2% Black/African American, 55.7% white, Mage = 70.7 years, SDage = 9.3, Medu = 12.6 years, SDedu = 2.6) who completed a comprehensive neuropsychological evaluation in a VA medical center. All participants completed the PT where they had five minutes to organize five pill bottles using a seven-day pillbox according to standardized instructions on the labels. Participants also completed some combination of 26 neuropsychological tests (i.e., participants did not complete every test as evaluations were tailored to disparate referral questions). Correlations between completed tests and number of pillbox errors were evaluated. These tests were then combined into the following six domains: language, visuospatial, working memory (WM), psychomotor/PS, memory, and EF. Hierarchical multiple regression was completed using these domains to predict pillbox errors.

Spearman’s correlation coefficients indicated that 25 tests had a weak to moderate relationship with PT total errors (rs = 0.23 -0.51); forward digit span was not significantly related (rs = 0.13). A forced-entry multiple regression was run to predict PT total errors from the six domains. The model accounted for 29% of the variance in PT performance, F(6, 169) = 11.56, p < .001. Of the domains, psychomotor/PS made the greatest contribution, f(169) = 2.73, p = .007, followed by language, f(169) = 2.41, p = .017, and WM, f(169) = 2.15, p = .033. Visuospatial performance and EF did not make significant contributions (ps>.05). Next, two hierarchical multiple regressions were run. Results indicated that EF predicted performance on the PT beyond measures of PS, AR2 = .02, p = .044, but not beyond the combination of all cognitive domains, AR2 = .00, p = .863.

Results of this study partially replicated the findings of Logue et al. (2015). Namely, EF predicted PT performance beyond PS, but not other cognitive domains. However, when all predictors were entered into the same model, visuospatial performance did not significantly contribute to the prediction of pillbox errors. These results suggest that providers may benefit from investigating medication management abilities when deficits in PS, WM, and/or language are identified. Further research is needed to better understand which domains best predict PT failure.

Craft story recall test in the National Alzheimer’s Coordinating Center Uniformed Data Set 3 (NACC UDS3) neuropsychological battery has been employed to assess verbal memory and assist clinical diagnosis of mild cognitive impairment (MCI) and dementia. While a Chinese version of the test was adapted, no existing literature has examined the diagnostic validity of the test in Chinese Americans. This study aimed to evaluate the predictive validity of both immediate and delayed recall.

The Chinese version of Craft Story was administered in to 78 Chinese participants per their language preference of Mandarin or Cantonese. Outcome measures were verbatim and paraphrase recall of the story immediately and after a 20-minute delay. A multiple linear regression was performed to investigate the association of each outcome measure with age, education, gender, age when moved to the U.S., years in the U.S., and testing language. To assess its diagnostic value, cutoff standard deviation scores of -1.5 and -2.0 from the mean of the clinically cognitive normal participants were generated for MCI and dementia diagnoses, respectively. Due to the small sample size, a normative group fitting the mean age (73 years), years of education (12 years), and the majority gender (female) of the current sample were used to identify standard cut points. A receiver-operating characteristic analysis was used to compare predicted diagnosis with actual clinical diagnosis obtained through patients’ overall performance and a consensus meeting by licensed clinicians.

Younger age (p < 0.05) and being tested in Mandarin (p < .01) were positively associated with immediate and delayed recall. Strong positive correlations between each measure were observed (all p < .001), indicating a significant relationship between information encoded and retained. Among all the participants, 15 (19.2%) were diagnosed with MCI and 22 (28.2%) with dementia. For MCI diagnosis, the standard cutoff scores demonstrated adequate sensitivity (verbatim=82%, paraphrase=91%) but low specificity (verbatim=44%, paraphrase=67%) in all outcome measures. For dementia diagnosis, delayed recall showed strong sensitivity (100%) and adequate specificity (75%) in both verbatim and paraphrasing scores. Immediate recall paraphrase (sensitivity = 95%, specificity = 50%) showed a better sensitivity but lower specificity than verbatim scoring (sensitivity = 86%, specificity = 58%). The accuracy was higher in delayed recall for both MCI and dementia diagnosis. A preliminary analysis on the optimal cut points indicated higher cutoff scores to distinguish MCI and dementia from clinically cognitive normal population, and from each other (e.g., the optimal cut point for delayed verbatim in distinguishing MCI from normal is 8.0 (sensitivity=89%, specificity=73%, AUC=84.3%)).

Consistent with previous literature, Craft Story delayed recall served as a more accurate diagnostic tool for both MCI and dementia compared to immediate recall in older Chinese Americans. However, poor specificity might increase the chance of following false positive subjects in clinical trials. In addition, testing language appeared to impact performance on verbal memory recall of constructed information. Thus, future studies should focus on developing normative scores that address both the overall cultural differences of Chinese Americans and the heterogeneity within this population.

Early detection is critical to the effective management of Alzheimer’s Disease (AD) and other forms of dementia. Existing screening assessments are often costly, require substantial expertise to administer, and may be insensitive to mild changes in cognition. A promising alternative is automatically measuring features of connected speech (c.f., Ostrand & Gunstad, 2021, Journal Geriatric Psych & Neurol) to predict impairment. Here, we built on prior work examining how well speech features predicted cognitive impairment. Unique to the current work, we attempted to capture more holistic effects of cognitive impairment by examining the relevance of linguistic features that measure sentential or discourse context properties of speech, including the context in which filler words (e.g., um) occur, and the predictability of individual words within their sentence context, computed from a large computational language model (GPT-2).

Participants completed the Cookie Theft picture description task, with data available in the DementiaBank corpus (Becker et al., 1994, Arch Neurol). Descriptions that contained at least 50 words (N = 214) were submitted to an automatic feature calculation pipeline written in Python to calculate various part-of-speech counts, lexical diversity metrics, and mean lexical frequency, as well as multiple metrics related to lexical surprisal (i.e., how surprising a word is given its context).

Surprisal of individual words was computed using the pre-trained GPT-2 transformer language model (Radford et al., 2019, Comput. Sci.) by computing word probability given the previous 12 words. Multiple linear regression was performed using 17 linguistic features jointly as predictors, and Mini-Mental State Examination (MMSE) score as the outcome variable. Simple regressions were calculated between each feature and MMSE scores to examine the predictability of each linguistic feature on cognitive decline.

A multiple linear regression model containing all linguistic features plus demographic information (age, sex, education) significantly predicted MMSE scores (Adjusted R2 = 0.41, F20, 193 = 8.37, p < .001), and explained significantly more variance in MMSE scores than did demographic variables alone (F17193 = 6.85, p < .001). Individual predictors that were significantly correlated with MMSE score included: how unexpected an individual’s word choices were, given the preceding context (median surprisal: r = -0.33, p < 0.001; interquartile range: r = 0.18, p < 0.02), mean lexical frequency (r = -0.50, p < .001), and usage of definite articles (r = 0.31, p < 0.001), nouns (r = 0.26, p < .001), and empty words (r = -0.25, p < 0.001).

Participants with lower MMSE scores, indicating greater impairment, used more frequent, yet more surprising, words, and produced more empty words and fewer definite articles and nouns. These results suggest that measures of semantic specificity and coherence of speech could be meaningful predictors of cognitive decline, and can be computed automatically from speech transcriptions. The results also provide novel evidence that computational approaches to estimating lexical predictability may have value in predicting the degree of decline, motivating future work in other speech elicitation tasks and differing clinical groups.

Diagnostic criteria for mild cognitive impairment (MCI) include a report of cognitive decline from the patient or a close informant. It is therefore important to understand the relationship between self- and informant-rated cognition and actual patient performance. Furthermore, it is unknown whether the nature of the relationship between the patient and their informant impacts accuracy of subjective reports. This study aimed to determine the association between informant report, self-report and objective cognitive performance based on relationship factors. We predicted that informant report would be more closely associated with objective performance than self-report after controlling for demographics and mood (Geriatric Depression Scale [mean= 1.4, SD=2]), especially among those who live with the participant and those who are spouses/partners.

Participants (n = 338; age= 73.5 ±6.7) of varying diagnoses and their respective informants were drawn from the longitudinal cohort of the Michigan Alzheimer’s Disease Research Center (MADRC). The majority of informants were spouses/significant others (55.6%), followed by 23.7% being other family members and 20.7% were non- family members; 58.9% of informants live with the participant. Both respondents completed the Cognitive Change Index (CCI) to rate the patient’s cognitive status (higher scores indicating worsening cognition) across three domains: memory (12 questions), language (1 question), and attention/executive functioning (7 questions). These domains were matched to objective cognitive performance measured using the MADRC neuropsychological battery. Executive functioning and attention were assessed using Number Span Test Forward and Backward (NSF, NSB) and Trail Making Test Part B and Trail- Making Test Part A and B ratio (TMTB, TMTB: A); memory was measured using Craft Story 21 (Immediate and Delayed), Hopkins Verbal Learning Test-Revised (HVLT-R) Total Recall, Delayed Recall, and Benson Complex Figure (BCF) Delayed Recall; and Language was measured by the Controlled Oral Word Association Test (COWAT) and Animal fluency.

Linear regression adjusted for sex, race, and mood indicated that both patient and informant CCI ratings were significantly (p<.05) associated with objective cognitive performance. For every one unit increase on executive CCI items, there was a significant decline in executive functioning (NSF patient and informant ß= -0.09, NSB: [ßP= -.14; ßp-0.13]) and TMTB [ßP= 3.85; ß= 3.10 [% change]). Memory performance also declined per unit increase on CCI memory items: (Craft Story 21 Immediate [ßP=-0.32; ß= -0.37] and Delayed [ßP=-.40; ßp -.47], HVLT-R Total Recall [ßP= -.31; ßI=-.37] and Delayed Recall [ßP= -.16; ß=-.20], and BCF Delayed Recall [ßP= -.18; ß= -.23]. Similarly, one unit increase on the single CCI language item was associated with a decline in COWAT (ßP= -2.27; ß= -4.61) and Animal fluency (ßP= -1.88; ß= -3.03). Effect modification by participant-informant relationship type or participant-informant cohabitation was not significant.

Patient and informant ratings are associated with objective measures of cognition regardless of the relationship between informant and patient or if they live together. This study was limited by a well-educated sample (mean= 16.1 years of education, SD= 2.4 years) with relatively limited diversity among participant-informant relationships. Future studies should replicate analyses across a larger and more diverse sample.

Accurate identification of Attention-Deficit/Hyperactivity Disorder (ADHD) is complicated by possible secondary gain, overlap of symptoms with psychiatric disorders, and face validity of measures (Suhr et al., 2011; Shura et al., 2017). To assist with diagnostic clarification, an experimental Dissimulation ADHD scale (Ds-ADHD; Robinson & Rogers, 2018) on the MMPI-2 was found to distinguish credible from non-credible respondents defined by Performance Validity Test (PVT)-based group assignment in Veterans (Burley et al., 2023). However, symptom and performance validity have been understood as unique constructs (Van Dyke et al., 2013), with Symptom Validity Tests (SVTs) more accurately identifying over-reporting of symptoms in ADHD (White et al., 2022). The current study sought to evaluate the effectiveness of the Ds-ADHD scale using an SVT, namely the Infrequency Index of CAARS (CII; Suhr et al., 2011), for group assignment within a mixed sample of Veterans.

In this retrospective study, 187 Veterans (Mage = 36.76, SDage = 11.25, Medu = 14.02, SDedu = 2.10, 83% male, 19% black, 78% white) were referred for neuropsychological evaluation of ADHD and administered a battery that included internally consistent MMPI-2 and CAARS profiles. Veterans were assigned to a credible group (n=134) if CII was <21 or a non-credible group (n=53) if CII was >21. The Ds-ADHD scale was calculated for the MMPI-2. Consistent with Robinson and Rogers (2018), “true” answers (i.e., erroneous stereotypes) were coded as 1 and “false” answers were coded as 2, creating a 10- to 20-point scale. Lower scores were associated with a higher likelihood of a feigned ADHD presentation.

Analyses revealed no significant differences in age, education, race, or gender (ps > .05) between credible and non-credible groups. An ANOVA indicated a significant difference between groups (F[1,185] = 24.78, p <.001; Cohen’s d = 0.80) for Ds-ADHD raw scores. Veterans in the non-credible group reported more “erroneous stereotypes” of ADHD (M raw score = 13.23, SD = 2.10) than those in the credible group (M = 14.94, SD = 2.13). A ROC analysis indicated AUC of .72 (95% CI = .64 to .80). In addition, a Ds-ADHD cut score of <12 resulted in specificity of 94.5% and sensitivity of 22.6%, whereas a cut score of <13 resulted in specificity of 85.8% and sensitivity of 50.9%. When analyzing other CII cut scores recommended in the literature, results were essentially similar. Specifically, analyses were repeated when group assignment was defined by cut score of CII<18 and by removing an intermediate group (CII = 18 to 21; n=24).

The Ds-ADHD scale demonstrated significant differences between credible and non-credible respondents in a Veteran population. Results suggest a cut score of <12 had adequate specificity (.95) with low sensitivity (.23). This is consistent with findings using PVTs for group assignment that indicated a cut score of <12 had adequate specificity (.92) with low sensitivity (.19; Burley et al., 2023). Taken together, findings suggest that the Ds-ADHD scale demonstrates utility in the dissociation of credible from non-credible responding. Further research should evaluate the utility of the scale in other clinical populations.

Few to no studies have directly compared the relative classification accuracies of the memory-based (Brief Visuospatial Memory Test-Revised Recognition Discrimination [BVMT-R RD] and Rey Auditory Verbal Learning Test Forced Choice [RAVLT FC]) and non-memory based (Reliable Digit Span [RDS] and Stroop Color and Word Test Word Reading trial [SCWT WR]) embedded performance validity tests (PVTs). This study’s main objective was to evaluate their relative classification accuracies head-to-head, as well as examine how their psychometric properties may vary among subgroups with and without genuine memory impairment.

This cross-sectional study included 293 adult patients who were administered the BVMT-R, WAIS-IV Digit Span, RAVLT and SCWT during outpatient neuropsychological evaluation at a Midwestern academic medical center. The overall sample was 58.0% female, 36.2% non-Hispanic White, 41.3% non-Hispanic Black, 15.7% Hispanic, 4.8% Asian/Pacific Islander, and 2.0% other, with a mean age of 45.7 (SD=15.8) and a mean education of 13.9 years (SD=2.8). Three patients had missing data, resulting in a final sample size of 290. Two hundred thirty-three patients (80%) were classified as having valid neurocognitive performance and 57 (20%) as having invalid neurocognitive performance based on performance across four independent, criterion PVTs (i.e., Test of Malingering Memory Trial 1, Word Choice Test, Dot Counting Test, Medical Symptom Validity Test). Of those with valid neurocognitive performance, 76 (48%) patients were considered as having genuine memory impairment through a memory composite band score (T<37 for (RAVLT Delayed Recall T-score + BVMT-R Delay Recall T-score/2).

The average memory composite band score for valid neurocognitive scores was T = 49.63 as compared to T = 27.57 for genuine memory impairment individuals. Receiver operating characteristic [ROC] curve analyses yielded significant areas under the curve (AUCs=.79-.87) for all four validity indices (p’s < .001). When maintaining acceptable specificity (91%-95%), all validity indices demonstrated acceptable yet varied sensitivities (35%-65%). Among the subgroup with genuine memory impairment, ROC curve analyses yielded significantly lower AUCs (.64-.69) for three validity indices (p’s < .001), except RDS (AUC=.644). At acceptable specificity (88%-93%), they yielded significantly lower sensitivities across indices (19%-39%). In the current sample, RAVLT FC and BVMT-R RD had the largest changes in sensitivities, with 19% and 26% sensitivity/90%-92% specificity at optimal cut-scores of <10 and <2, respectively, for individuals with memory impairment, compared to 65% and 61% sensitivity/94% specificity at optimal cut-scores of <13 and <4, respectively, for those without memory impairment.

Of the four validity scales, memory-based embedded PVTs yielded higher sensitivities while maintaining acceptable specificity compared to non-memory based embedded PVTs. However, they were also susceptible to the greatest declines in sensitivity among the subgroup with genuine memory impairment. As a result, careful consideration should be given to using memory-based embedded PVTs among individuals with clinically significant memory impairment based on other sources of information (e.g., clinical history, behavioral observation).

The Explosive Ordinance Disposal (EOD) community within the US Military is a specialized force in charge of the most fundamental aspects of the military operations in combat which include disarming and safely disposing explosive threats. EOD technicians have provided critical protection for our military and civilians exposed to improvised explosive devices (IEDs), which became the signature threat of both Afghan and Iraq wars. The nature of the job puts EOD technicians at high risk for blast exposures (from training and combat) resulting in traumatic brain injury (TBI) and sub-concussive head impact. Further, this population is exposed to high levels of combat with psychologically traumatic events. Given the groups neurological and psychological risk factors as well as their critical role in combat, we hypothesized that EOD technicians will present with increased psychological and neurobehavioral symptoms as well decreased cognitive functioning compared to other military personnel.

Participants were recruited from a military hospital with at least one diagnosed mild traumatic brain injury (MTBI). Exclusion criteria included TBI greater then mild severity and invalid performance on the Rey-15. Final sample included 10 EOD and 90 other military.

CognCognitive measures included Hopkins Verbal Learning Test-Reviseitive measures included Hopkins Verbal Learning Test-Revised (HVLT-R); DKEFS Color Word Condition 4 Switching (CW4), Trail Making Condition 3 Letter Sequencing (TM3) and Condition 4 Switching (TM4), and Paced Auditory Serial Addition Test (PASAT). Self-report measures included the Neurobehavioral Symptom Inventory (NSI), Key Behaviors Change Inventory (KBCI), Post-Traumatic Stress Disorder Checklist (PCL-M), Patient Health Questionnaire (PHQ), Combat Exposure Scale (CES) and Blast Exposure Threshold Survey (BETS). The Ohio State University Traumatic Brain Injury Identification Method (OSU) assessed TBI history.

EOD were older (EOD M=38.4, SD=4.06; Others M=33.32, SD=8.08; p=0.05), had a higher pre-morbid IQ (EOD M=110.90, SD=7.64; Other M=101.59, SD=10.55; p=0.008), more combat deployments (EOD M=5.5, SD=2.37; Others M=3.55, SD=2.98; p=0.049) and exposure to wartime atrocities (CES, p=0.003). They had greater number of MTBI (OSU EOD M=6.67, SD=3.33; Other M=3.67, SD=2.34; p=0.007), blast related MTBI (OSU-TBI EOD M=2.33, SD=1.63; Other M=0.67, SD=0.91; p<0.001), and exposure to large explosives (BETS p<0.0001). EOD reported better attention skills (KBCI Inattention, p=0.016, d=0.82; Impulsivity p=0.047, d=0.67). There was a trend for EOD to have lower neurobehavioral symptoms (NSI Total, d=0.32), post-traumatic stress (PCL d=0.39), and depression (PHQ d=0.50); however, despite the moderate effect sizes (p’s >0.05). EOD presented with significantly better scores on DKEFS TMT3 (p=0.037, d=0.70), HVLT-R-Total (p=0.001, d=1.10), HVLT-R-Delayed (p=0.03, d=0.74), and attention/executive functioning skills (PASAT p=0.001, d=1.12). DKEFTS CW4 Switching (d=0.51) and TMT4 Switching were approaching significance (d=0.61) with EOD performing better.

As expected, the EOD sample in this study had higher number of combat deployments, greater exposure to combat atrocities (e.g., death), higher levels of exposure to large explosives, as well as a higher number of MTBI. Inconsistent with our hypotheses, despite these psychological and neurological risk factors, EOD performed better on cognitive measures of memory, attention and executive functioning. They also reported less problems with inattention and impulsivity. Results may reflect the impact of psychological and cognitive resiliency.

Research evaluating mindfulness and cognition has produced mixed results. However, variability in mindfulness has not been previously evaluated as a predictor of cognitive ability. This study evaluated the relation between intra-individual variability (IIV) in mindfulness and cognitive performance.

274 university participants (M=19 years old, SD=1.5; 72.6% female, 67.2% White, 25.6% African American, 3.3% Asian American, 1.1% Hispanic American) completed the Five Facet Mindfulness Questionnaire (FFMQ) and the CNS Vital Signs computerized test battery. IIV was computed from the FFMQ facet T-scores. Additionally, high and low cognitive performance groups were formed from the top and bottom 16% of the sample using the neurocognition index (NCI) score from CNS Vital Signs (N=52 high NCI performance and N=46 low NCI performance).

Pearson r correlations were used to evaluate the relation between mindfulness IIV and CNS Vital Signs domains. Mindfulness IIV was negatively associated with performance on the domains of psychomotor speed [r=-.18; p=.003], composite memory [r=-.14; p=.023] and verbal memory [r=-.15; p=.015]. For the high NCI group, IIV mindfulness was positively associated with cognitive flexibility [r=.31; p=.024], executive functioning [r=.33; p=.016] and was negatively related to visual memory [r=-.28; p=.043]. For the low NCI group, IIV mindfulness was negatively related to psychomotor speed [r=-.49; p<.001], composite memory [r=-.32; p=.033] and verbal memory [r=-.31; p=.038]. There was no relation found for individual FFMQ facet scores and CNS Vital Sign domains.

Increased consistency in self-reported mindfulness (lower IIV) was associated with greater processing speed and memory performance in the overall sample. However, the relation been mindfulness IIV and cognitive performance changed greatly in high NCI performers compared to low NCI performers. The low NCI group may be a proxy for poor effort which would explain why more variable self-reported mindfulness was associated with worse performance for processing speed and memory and this could be driving the results for the overall sample. However, our findings for the high NCI performance group are unique and suggest an association between increased variability in mindfulness facets and improved cognitive flexibility and executive functioning. Further study of mindfulness variability and aspects of executive functioning is warranted.

Prior studies have presented demographic adjustment as beneficial because it helps equalize, across demographic groups, the percentage of participants (recruited from the general population without prior diagnosis) who fell beneath the test impairment cutoff (e.g., Smith, et al., 2008). This methodology ignores the possibility that group differences in those falling beneath an impairment cutoff could reflect cognitive impairment prevalence differences between demographic groups in the undiagnosed general population. Demographic group differences in cognitive test scores reflect a mixture of two categories of influences: measurement bias (item/test/examiner bias, language/cultural bias, stereotype threat, etc.) and factors which differentially increase the number of low scores in one group by increasing relative risk (RR) for cognitive impairment (biological aging processes, cognitive reserve, social determinants of health [SDoH], etc.). The current simulation study examined how the effect of demographic adjustment on the diagnostic accuracy of a hypothetical test (operationalized as the area under the curve [AUC] in an ROC analysis) varied as the mixture of influences which caused demographic differences in scores were varied.

215,040 samples were randomly generated. Each sample consisted of two demographic groups, with Group 0 always representing the lower scoring group. Across samples, Group 1's baseline risk of impairment and Group 0's relative risk were varied, and these determined the prevalence of cognitive impairment in the groups. Three facets of measurement bias were varied in the simulation: how much lower Group 0's average score was than Group 1's, the degree of non-homogeneity of variance between groups, and how much less reliable the measure was for Group 0. Additional parameters were included and varied to ensure the robustness of findings across a variety of situations. Samples reflected all possible combinations of all varied parameters. For each sample, a baseline AUC was calculated when impairment was regressed on the unadjusted test score. Then, test scores were adjusted for demographic group and difference in adjusted and unadjusted AUC was calculated. This adjusted/unadjusted AUC difference was then regressed on the simulation parameters to quantify their relative influence.

The more Group 0's average score was reduced by measurement bias, the more improvement in AUC was seen after adjustment (ß = 1.76). Trivial but significant main effects of variance non-homogeneity (ß = .09), increased relative risk (ß = -.08), and reduced reliability (ß = .02) were also found, but more importantly, each of these predictors significantly interacted with Group 0 mean score reduction, such that higher relative risks (ß = -1.22), lower reliability (ß = .36), and higher variance (ß = -.15) in Group 0 compared to Group 1 each reduced the association between Group 0 mean score reduction and improvement in AUC.

Demographic adjustment only improves AUC when the mean reduction in scores due to measurement bias is sufficiently high while risk for impairment, test reliability and test score variances are sufficiently equivalent among the demographic groups. When this is not the case, demographic adjustment can be counter-productive, reducing the AUC of the test. We conclude by proposing a novel method for adjusting test scores.

Judgment, defined as the capacity to make decisions carefully after consideration of available information, which may entail a variety of sources, has come to be regularly assessed within neuropsychology, and impairment of judgment has been demonstrated across multiple disorders (Rabin, Borgos, & Saykin, 2008). This study aimed to re-examine the relationship between judgment and performance on measure of cognitive functioning including (memory, attention, language, visuospatial abilities, speed, and aspects of executive functioning) in a mixed-clinical sample of older adult veterans.

Data for this study was collected from the Cognitive Functioning in Older Adult Veteran’s database repository (CFOAV) at a large Veteran Affairs Healthcare System (VAHCS). Participants were veterans seeking treatment in the Neuropsychology Assessment Clinic. Inclusion criteria were that participants must have answered the nine questions from the TOP-J and received a score based on the specific criteria. Participants were excluded if they appeared to lack adequate test engagement or had a serious mental illness. The final sample for the current study consisted of 83 veterans (73% male, n = 76), ranging from 50 to 89 years (m = 72.01, SD = 9.70), with and average of 13 years of education (SD = 3.21). Of the sample, 75% reported that they were White, 7% African American/Black, and 1% Latino/Hispanic, and ICD-10 diagnoses ranged from age-related cognitive decline, mild cognitive impairment, vascular dementia, and dementia in other disease classified.

Using SPSS (Version 27), Pearson correlations were conducted to examine the relationship between the TOP-J raw score, demographic variables, and measures of cognitive functioning, including the WTAR, the RBANS index scores, WAIS-DS, TMT A, TMT B, COWAT, and ANT. Missing data were excluded pairwise in the analyses. Correlation analyses revealed a significant small-to-medium correlation between the TOP-J and the. There were small to medium correlations between the TOP-J, WTAR (r = .31, p = .01), TMT A (r = .27, p = .02), WAIS DS (r = .30, p = .01), and RBANS Attention index (r = .35, p = .04). There was a significant large relationship between the TOP-J and the RBANS Immediate Memory index (r = .52, p = .002). There were no significant associations between the TOP-J, demographic variables (e.g., biological sex, age, and education), TMT-B, COWAT, or ANT.

The study supported previous decision making research (Moye, Karel, Gurrera & Asar, 2006) that has found the ability to attend to and immediately retain information to be an important foundational component. While the present study did not fully replicate previous findings that the Top-J was correlated to measures of executive functioning, strong correlations did emerge with verbal memory and a measure of crystalized verbal abilities similar to Rabin et al. (2007). Such research informs the assessment practical judgment. It also indicates that one’s ability to acquire and encoded unstructured and contextual verbal information, as well as pre-morbid verbal abilities, may provide potential targets to improve or compensate from decrements in overall practical judgement. This is certainly an areas for future research.

Limitations of traditional neuropsychological assessment include testing in a highly controlled environment designed to minimize distraction. While informative, it may not fully capture real-world cognitive functioning. This may be particularly important for individuals with mild traumatic brain injury (mTBI), a subset of whom report subtle challenges with complex cognitive functioning that are not consistently captured by neuropsychological assessment. The objective of this study was to extend previous work examining cognitive correlates of performance on functional assessment tool, the Goal Processing Sale (GPS), in a larger sample of Veterans with mTBI.

46 Veterans with chronic mTBI completed GPS and neuropsychological measures (mean age = 43.5; education = 15 years; 89% male). 93% of participants had clinically significant PTSD (PCL-M > 31). The GPS is an ecologically valid assessment in which participants plan and execute a complex task following specified rules under a time constraint. Performance is rated on a 0 (not able) to 10 (absolutely not a problem) scale in 8 domains: 1) Planning, 2) Initiation, 3) Self-Monitoring, 4) Maintenance of Attention, 5) Sequencing and Switching of Attention, 6) Flexible Problem Solving, 7) Task Execution, and 8) Learning and Memory. The GPS Overall Performance is average of 8 domain scores. Neuropsychological assessment data were scored using standardized norms and transformed into z-scores. Scores were averaged into 2 domains: 1) Overall Attention/Executive Function (4 subdomains: Working Memory [Auditory Consonant Trigrams, WAIS-III Letter Number Sequencing], Sustained Attention [Digit Vigilance Test], Inhibition [D-KEFS Stroop Inhibition], Mental Flexibility [Trail Making Test B, D-KEFS Stroop Inhibition Switching, Design Fluency Switching, Verbal Fluency Switching]) and 2) Overall Memory (2 subdomains: Total Recall [HVLT-R, BVMT-R], and Delayed Recall [HVLT-R, BVMT-R]).

Pearson correlation coefficients were used to determine relation between overall GPS and overall executive function performance, as well as 8 GPS subdomain and 8 neuropsychological domain/subdomain scores. To adjust for multiple comparisons, p < .01 was used.

Overall GPS performance was statistically significantly related to Overall Attention/Executive Functioning and Overall Memory. Investigating further, multiple significant subdomain relations emerged. GPS Planning was related to Inhibition. GPS Self-Monitoring and GPS Task Execution were related to Mental Flexibility. GPS Maintenance of Attention and GPS Flexible Problem Solving were related to Mental Flexibility and Inhibition. GPS Sequencing and Switching of Attention was related to Mental Flexibility and Total Recall.

GPS Learning and Memory was related to Working Memory, Mental Flexibility, and Inhibition. GPS Initiation was not related to neuropsychological measures.

Current findings build upon prior work establishing validity of GPS functional assessment measure (Novakovic-Agopian et al., 2012). Seven of 8 GPS subdomains were related to at least one aspect of executive functioning assessed with neuropsychological measures, with the majority related to mental flexibility. Taken together, findings suggest that the GPS converges with traditional measures, offering a method to capture multiple aspects of executive functioning applied together. Further, it may also be useful tool capturing aspects of executive functioning in complex, ecologically-valid settings often not captured with traditional neuropsychological assessment.

Assessment of performance validity during neuropsychology evaluation is essential to reliably interpret cognitive test scores. Studies (Webber et al., 2018; Wisdom, et al., 2012) have validated the use of abbreviated measures, such as Trial 1 (T1) of the Test of Memory Malingering (TOMM), to detect invalid performance. Only one study (Bauer et al., 2007) known to these authors has examined the utility of Green’s Word Memory Test (WMT) immediate recall (IR) as a screening tool for invalid performance. This study explores WMT IR as an independent indicator of performance validity in a mild TBI (mTBI) veteran population.

Participants included 211 (Mage = 32.1, SD = 7.4; Medu = 13.1, SD = 1.64; 94.8% male; 67.8% White) OEF/OIF/OND veterans with a history of mTBI who participated in a comprehensive neuropsychological evaluation at one of five participating VA Medical Centers. Performance validity was assessed using validated cut scores from the following measures: WMT IR and delayed recall (DR); TOMM T1; WAIS-IV reliable digit span; CVLT-II forced choice raw score; Wisconsin Card Sorting Test failure to maintain set; and the Rey Memory for Fifteen Items test, combo score. Sensitivity and specificity were calculated for each IR score compared with failure on DR. In addition, sensitivity and specificity were calculated for each WMT IR score compared to failure of at least one additional performance validity measure (excluding DR), two or more measures, and three or more measures, respectively.

Results indicated that 46.8% participants failed to meet cut offs for adequate performance validity based on the standard WMT IR cut score (i.e., 82.5%; M = 81.8%, SD = 17.7%); however, 50.2% participants failed to meet criteria based on the standard WMT DR cut score (M = 79.8% SD = 18.6%). A cut score of 82.5% or below on WMT IR correctly identified 82.4% (i.e., sensitivity) of subjects who performed below cut score on DR, with a specificity of 94.2%. Examination of IR cutoffs compared to failure of one or more other PVTs revealed that the standardized cut score of 82.5% or below had a sensitivity of 78.2% and a specificity of 72.4%; whereas, a cut score of 65% or below had a sensitivity of 41% and a specificity of 91.3%. Similarly, examination of IR cutoffs compared to failure of two or more additional PVTs revealed that the cut score of 60% or below had a sensitivity of 45.7% and specificity of 93.1% ; whereas, a cut score of 57.5% or below had a sensitivity of 57.9% and specificity of 90.9% when using failure of three or more PVTs as the criterion.

Results indicated that a cut score of 82.5% or below on WMT IR may be sufficient to detect invalid performance when considering WMT DR as criterion. Furthermore, WMT IR alone, with adjustments to cut scores, appears to be a reasonable way to assess symptom validity compared to other PVTs. Sensitivity and specificity of WMT IR scores may have been adversely impacted by lower sensitivity of other PVTs to independently identify invalid performance.