Recent consensus guidelines have advocated for the use of multivariate performance validity assessment on ability-based measures such those used in neuropsychological assessment. Further, previous research has demonstrated that aggregating performance validity indicators may produce superior classification accuracy. The present study builds upon this research by aggregating data from three of the most commonly used performance validity measures (Test of Memory Malingering [TOMM], Rey Fifteen Item Test with recognition trial [FIT plus recognition], and Reliable Digit Span [RDS]) to create a performance validity composite measure in a veteran mild traumatic brain injury (mTBI) population.

Data of patients evaluated at a VA hospital who had completed the RDS, FIT plus recognition, and TOMM as part of their clinical neuropsychological evaluation were analyzed (n = 20). Two composite performance validity indexes were created: a Single Cutoff Performance Validity Index (SC-PVI), which measures the quantity of failures across performance validity measures (PVMs) by summing the total number of PVM failures, and a Multiple Cutoff Performance Validity Index (MC-PVI) which measures the number of failures as well as degree of failure(s) across measures of performance validity (e.g., a participant would attain a score of 3 if their PVM performance failed to reach a conservative cut point; they would obtain a score of 1 if they met conservative cut point, yet failed to reach a liberal cut point).

Only one participant (5%) attained a score of 0 on the SC-PVI (i.e., passing all PVTs using standard cutoffs) and MC-PVI (i.e., passing the most liberal cut points on all three PVMs). Conversely, eight participants (40%) attained a score of 3 on the SC-PVI (i.e., failed all three PVMs) and four participants (20%) attained a score of 9 (i.e., failed the most conservative cut points on all three PVMs). Results showed a significant (p < .001) ordinal association between the two indices (G = .984); however, there was no significant agreement between SC-PVI and MC-PVI models (k = -.087; p = .127).

Data revealed discordant findings between the three PVMs utilized. The majority of participants (75%) scored between 2-8 on the MC-PVI, meaning that they did not exceed all liberal cut points or fail all conservative cut points. These “grey area” scores suggest an indeterminate range of performance validity, which cannot be captured by a solitary cut point or neatly classified as pass or fail. The utility of multiple cutoff performance validity models (i.e., aggregating PVMs to consider the severity of failure and number of failures) is that they capture the nuance of these data when determining and discussing the credibility of a profile. Multiple cut point data also highlight how the choice of cutoff influences the outcome of performance validity research and clinical decision making. As such, future research on the classification accuracy of this MC-PVI is needed.

The Test of Memory Malingering (TOMM) is a performance validity test (PVT) that aims to assess whether participants are giving adequate effort to perform well on tasks of memory performance (Tombaugh, 1996). Other PVTs, specifically the Forced Choice Recognition Trial in the California Verbal Learning Test, have shown that even single errors may indicate invalid performance (Erdodi et al., 2018). Finally, youth are often understudied in the PVT literature, and athletes are at increased risk of invalid performance on baseline testing due to many wanting to return to play following concussion (Erdal, 2012). Therefore, the objective of the current study is to examine whether single errors on TOMM Trial 1 are indicative of lower, and possibly invalid, cognitive performance in a youth sample, given that cognitive performance declines with even small decreases in effort (Green, 2007).

Healthy youth athletes (n=174) aged 8-16 years (M=12.07) completed the TOMM as well as other neuropsychological measures during baseline neuropsychological evaluation in a clinical research program for sports concussion. Independent samples t-tests compared youth athletes who scored 49 points on the TOMM (n=28) to youth athletes who scored a perfect 50 (n=50) across several groupings of neuropsychological measures. Participants who scored less than 49 or who didn’t complete the TOMM were excluded from the analyses.

Participants scoring 50/50 on TOMM Trial 1 scored significantly higher on Stroop Color Naming task (p=0.036), Verbal Learning Delayed task from the second edition of the Wide Range Assessment of Learning and Memory (WRAML-2, p=0.018), and Letter Number Sequencing task from the Weschler Intelligence Scale for Children (WISC-IV, p=0.025), relative to participants scoring 49/50. Though not statistically significant, results also showed a trend toward participants scoring 50/50 scoring higher on nearly every test in the battery.

Participants with a single error on TOMM Trial 1, as compared to participants with a perfect score, performed significantly worse on a processing speed task, a verbal learning task, and a working memory task as part of a comprehensive neuropsychological battery. The single-error group also trended toward scoring lower on nearly all of the remaining attention, processing speed, perceptual ability, memory, and executive functioning tasks in the battery. The results could lead to a more liberal interpretation of TOMM scores, given that the trend towards lower performance may be due to putting forth significantly less effort. These results point to the need for a similar comparison of the TOMM in a larger sample size, as greater power may reveal even more significant differences in performance. Findings also emphasize the importance of viewing performance validity on a continuum rather than as a dichotomous pass/fail. Understanding the TOMM and how single errors may be indicative of poorer performance in a youth sample could help to reframe the way PVT results are interpreted in clinical and forensic settings.

Neuropsychological evaluations are used to examine a person’s current cognitive functioning. Performance validity tests (PVT) are included in neuropsychological test batteries to ensure that examinees are performing to the best of their abilities and identify non-credible performance. There are two types of PVTs: freestanding and embedded. A freestanding PVT is a cognitive test created to evaluate performance validity and do not measure any type of cognition directly. Meanwhile, an embedded PVT is a task design to evaluate some sort of cognition (e.g., memory) by using traditional neuropsychological tests (e.g., Trail Making Test) and performance validity. Research suggests that undergraduate college students are not always performing to the best of their abilities when completing a comprehensive neuropsychological battery. In fact, in one study where an undergraduate college sample was given three PVTs, it was reported that 56% of the participants failed at least one PVT in their first session and 31% in their second session. Research has also shown that speaking multiple languages can influence cognition. The purpose of this study was to identify in three credible language groups of college students what PVTs does bilingualism influence higher failure rates. It was predicted that bilingual college students would significantly demonstrate higher PVTs failure rates compared to monolingual college students.

The sample consisted of 70 English first language monolinguals (EFLM), 33 English first language bilinguals (EFLB), and 68 English second language bilinguals (ESLB) that were psychologically and neurologically healthy. All participants completed a comprehensive neuropsychological battery in English. The Rey-Osterrith complex figure copy test, Comalli Stroop part A, B, and C, Trail Making Test part A and B, Symbol Digit Modalities Test written and oral parts, Controlled Oral Word Association Test (COWAT) letter fluency, and Finger Tapping Test were the tasks used as embedded PVTs to evaluate failure rates in our sample. Moreover, all participants were credible (i.e., they did not fail two or more PVTs). PVT cutoff scores were selected for each embedded PVT from previous literature. Chi-square analysis were used to evaluate failure rates between language groups on each PVT.

We found no significant failure rate differences between language groups on any of the PVTs. However, while no significant group differences were found, on the COWAT letter fluency results revealed higher failure rates between the three language groups (i.e., 13% EFLM, 24% EFLB, and 22% ESLB) compared to other PVTs.

Our data suggested no significant failure rate differences between language groups. It has been suggested in previous studies that linguistic factors impact PVT performance and test interpretation. On the COWAT letter fluency task, it is possible that language is driving higher failure rates between bilingual speakers, even though we found no significant failure rates or performance differences between the three language groups. Future studies should examine language groups and other cultural variables (e.g., time perspective) to determine what may be driving high failure rates on the COWAT letter fluency task in credible participants.

Some RCFT indices are effective Performance Validity Test (PVTs) during neuropsychological evaluations. A combination score that includes the copy score, true positive recognition, and atypical errors has proven to be especially useful (see Lu et al, 2003). However, this score was derived from administration that deviated from protocols outlined by Meyers & Meyers (1995) in that the Recognition trial was administered after the 3-minute delay instead of the 30-minute delay. The current study examined the utility of the RCFT combination score as a performance validity test (PVT) when completing the recognition trial after the 30-minute delay.

This study utilized archival data from 298 Veterans who presented for a clinical neuropsychological evaluation at a southern Veterans Affairs Medical Center. The evaluation included up to nine PVTs and all trials of the RCFT (per Meyers & Meyers, 1995). Patients were considered credible if all PVT performance fell within normal limits. This resulted in 232 patients in the credible group (Mage = 52.9 years, SDage = 15.2, Medu = SDedu = 2.5, 88% male, 71.2% White, 28.3% Black/African American). Patients were considered non-credible if they failed >2 PVTs. This resulted in 66 patients in the non-credible group (Mage = 51.6, SDage = 13.79, Medu = SDedu = 2.4, 92.4% male, 56.1% White, 43.9% Black/African American). Group assignment was also clinically confirmed. Receiver operating characteristic (ROC) curve analyses were conducted to discriminate between credible and non-credible groups utilizing the established RCFT combination score.

RCFT combination scores distinguished groups, with credible participants scoring higher than non-credible participants (F[1, 296]=63.76, p<.001, d=1.11; M = 56.9, SD = 9.3 vs. M = 46.5, SD = 9.5, respectively). A ROC analysis indicated AUC = .800 (95% CI = .73 to .86). When specificity was set at >90%, a cut-score of <46.5 yielded sensitivity at 46.0%. The analogous cut-score from the Lu et al. (2003) study (i.e., <47) was associated with a specificity of 88.7 and sensitivity of 46.0% in the current study.

As the Lu et al. (2003) established the combination score of the RCFT with procedures that deviated from the standardized protocol outlined by Meyers and Meyers (1995), clinicians who opted to adhere to Meyers and Meyers’ full protocol may have concerns about using the combination score as a PVT. The current study established a similar cut-off score to what Lu et al., (2003) reported (i.e., <46.5 vs. <47) while following a different administration procedure of the RCFT. Also, the index was moderately sensitive in the current study (i.e., 45.5%) but less so than what Lu et al. reported when using a cut-score that had >90% specificity (i.e., 75.9% sensitivity). This suggests that the index may be robust to deviations in administration procedures. Difference in sensitivity could be related to difference between samples. As the current sample was derived from a clinical, VA setting, current findings extend the generalizability of the index. Future research would benefit exploring if any subgroups would benefit from adjusted cut-scores to reduce the risk of false positive identification.

As neuropsychologists aim to collect valid data, maximize the utility of assessments, make effective use of time, and best serve patient populations, measurement of performance validity is considered a critical issue for the field. As effort may vary across an evaluation, including performance validity tests (PVTs) throughout the assessment is important. Incorporating embedded PVTs in addition to free standing PVTs can be particularly useful in this regard. COWAT and animal naming are commonly administered verbal fluency measures. While there have been past investigations into their potential for detecting invalid performance, they are limited, and more research is needed. Perhaps most promising, Sugarman and Axelrod (2015) described a logistic regression derived formula utilizing the combined raw scores of COWAT and animal naming. The current study aimed to investigate the use of embedded PVTs within COWAT and animal naming to provide further support for the use of embedded PVTs in these measures.

All subjects were from a mixed clinical sample comprising military veterans from two VA Medical Centers in the northeast U.S., who were referred for neuropsychological evaluation. Subjects deemed credible had zero PVT failures. Subjects were considered non-credible performers if they failed at least two out of a possible eight PVTs administered. Subjects who failed one PVT were excluded from the study (n = 53). The final sample consisted of 116 individuals with credible performance (Mean Age = 35.5, SD = 8.8; Mean Edu = 13.6, SD = 2; Mean Est. IQ = 106, SD = 7.9) and 94 individuals with psychometrically determined non-credible performance (Mean Age = 38.5, SD = 9.4; Mean Edu = 113, SD = 2.1; Mean Est. IQ = 101, SD = 8.7). Performance of COWAT and animals in detecting non-credible performances was evaluated through calculation of classification accuracy statistics and use of the logistic regression formulas reported in Sugarman and Axelrod (2015).

For COWAT, the optimal cutoff was a raw score of <27 (specificity = 89%; sensitivity = 31%), and a T-score of <35 (specificity = 92%; sensitivity = 31%). For animal naming, optimal cutoffs were <16 for raw score (specificity = 92%, sensitivity = 38%) and <37 for T-score (specificity = 91%; sensitivity = 33%). The logistic regression formula based on raw scores for both COWAT and animal naming was inadequately sensitive at the recommended cutoff in this sample, but a coefficient of > .28 was revealed to be optimal (91% specificity; 42% sensitivity). When the formula for T-scores was used, a coefficient of > .38 was optimal (91% specificity; 28% sensitivity).

Results of the current research suggest that PVTs embedded within the commonly administered COWAT and animal naming verbal fluency tests can effectively detect low effort, in concordance with generally accepted standards. A logistic regression formula using raw scores in particular appears to be most effective, consistent with findings reported by Sugarman and Axelrod (2015).

Performance validity tests (PVTs) are included in neuropsychological testing to ensure examinees are performing to the best of their abilities. There are two types of PVTs: embedded and free standing. Embedded PVTs are tests that are derived from standard neuropsychological tests of various cognitive domains. Freestanding PVTs are tests that are designed with the intention of being a PVT. Research studies show that undergraduate samples do not always performed to the best of their abilities. The purpose of this study was to cross-validate previous research on the topic of performance validity in a college sample. It was predicted that the non-credible group would demonstrate higher failure rates on embedded PVTs compared to the credible group.

The sample consisted of 198 neurologically and psychologically healthy undergraduate students with a mean age of 19.69 (SD = 2.11). Participants were broken into two groups: non-credible (i.e., participants that failed two or more PVTs) and credible (i.e., participants that did not failed two or more PVTs). The Rey-Osterrith copy test, Comalli Stroop part A (CSA), B (CSB), and C (CSC), Trail Making Test part A and B, Symbol Digit Modalities Test written (SDMT-W) and oral (SDMT-O) parts, Controlled Oral Word Association Test (COWAT) letter fluency, and Finger Tapping Test were used to evaluate failure rates in our sample. PVT cutoff scores were use from previously validated in the literature. Chi-square analysis was used to evaluate failure rates between the groups.

Chi-square analysis revealed significant failure rate differences between groups on several PVTs. Results revealed that 15% of the non-credible group failed the CSA compared to 1% of the credible group, X2=14.77, p=.000. Meanwhile, 26% of the non-credible group failed the CSB compared to 2% of the credible group, X2=24.72, p=.000. Furthermore, results showed that 11% of the non-credible group failed the CSC compared to 1% of the credible group, X2=13.05, p=.000.Next, 48% of the non-credible group failed the Trail Making Test part A compared to 8% of the credible group, X2=31.61, p=.000. We also found that 15% of the non-credible group failed the SDMT-W part compared to 1% of the credible group,X2=19.18, p=.000. Meanwhile, on the SDMT-O part 19% of the non-credible group failed compared to 1% of the credible group, X2=25.52, p =.000. On the COWAT letter fluency task 74% of the non-credible group failed compared to 19% of the credible group, X2=36.90, p=.000. Finally, results revealed on the Finger Tapping Test 19% of the non-credible group failed compared to 3% of the credible group, X2=10.01, p=.002.

As expected, the non-credible participants demonstrated significantly higher PVT failure rates compared to credible participants. A possible explanation driving higher failure rates in our sample can be due to cultural variables (e.g., bilingualism). It was suggested by researchers that linguistic factors may be impacting higher PVT failure rates and developing a false-positive error. Future research using undergraduate samples need to identify which PVT’s are being impacted by linguist factors.

The Brief Visuospatial Memory Test-Revised (BVMT-R) Recognition Discrimination (RD) index has emerged as an embedded performance validity test (PVT). However, there do not appear to be any studies that have examined its utility in Spanish-speaking samples. This pilot study examined the classification accuracy of the BVMT-R RD for detecting performance invalidity in a Spanish-speaking forensic sample.

This cross-sectional study utilized a sample of 89 Spanish speakers that were administered the BVMT-R during an outpatient neuropsychological evaluation. Out of the 89 Spanish speakers, 43 were subjects in litigation, 32 were neurological patients evaluated for clinical purposes, and 14 were healthy controls. The sample was 67% male/33% female, 53% South American, 33% Caribbean (Dominican, Puerto Rican, Cuban), 10% Central American, 3% North American (Mexican), and 1% Spanish, with a mean age of 44.2 years (SD = 14.2; range = 20-78) and mean education of 11 years (SD = 3.7; range = 0-20). Test administration for each patient was completed in Spanish by a fluent, Spanish-speaking examiner. In total, 64/89 (72%) were classified as valid and 25/89 (28%) as invalid based on performance across the Test of Memory Malingering (TOMM), at least one additional PVT (Rey-15 item memory test; Rey Dot Counting Test; Reliable Digit Span; WHO-AVLT recognition trial) and objective diagnostic criteria identifying invalid performance. Analyses included three univariate analyses of variance (ANOVA), with the groups (healthy vs neurological vs litigation) as independent variables and performance on BVMT-RD as the dependent variable.

Statistically significant differences among the groups were found F(2,86)=8.32, p < .001). Post-hoc analysis (Scheffe test) showed the mean of the litigation group to be significantly lower than the means of the other two groups (healthy and neurological), which showed no difference between them. An ANOVA with validity groups as the fixed factor and BVMT-R RD index as the dependent variable was significant F(1,85)= 21.02, p <.001). Results of a ROC curve analysis yielded statistically significant AUC (.794). The optimal cut-score was BVMT-R RD < 5 (48% sensitivity/88% specificity).

Results of the BVMT-R RD index in this Spanish-speaking population differed by subgroup, with worse performance seen in individuals involved in litigation, compared to those who were not (healthy and neurological). Notably, the BVMT-R RD index significantly differentiated validity groups, maintaining adequate sensitivity and good specificity. Overall, results demonstrate promise for BVMT-RD as a PVT for Spanish-speaking populations.

Typical evaluations of adult ADHD consist of behavior self-report rating scales, cognitive or intellectual functioning measures, and specific measures designed to measure attention. Boone (2009) suggested monitoring continuous effort is essential throughout psychological assessments. However, very few research studies have contributed to malingering literature on the ADHD population. Many studies have reported the adequate use of symptom validity tests, which assess effortful performance in ADHD evaluations (Jasinski et al., 2011; Sollman et al., 2010; Schneider et al., 2014). Because of the length of ADHD assessments, individuals are likely to become weary and tired, thus impacting their performance. This study investigates the eye movement strategies used by a clinical ADHD population, non-ADHD subjects, and malingering simulators when playing a common simple visual search task.

A total of 153 college students participated in this study. To be placed in the ADHD group, a participant must endorse four or more symptoms on the ASRS (N = 37). To be placed in the non-ADHD, participants should have endorsed no ADHD symptoms (N = 43). Participants that did not meet the above criteria for ADHD and not-ADHD were placed in an Indeterminate group and were not included in the analysis. A total of 20 participants were instructed to fake symptoms related to ADHD during the session. A total of twelve Spot the Difference images were used as the visual picture stimuli. Sticky by Tobii Pro (2020) was used for the collection of eye-movement data was utilized. Sticky by Tobii Pro is an online self-service platform that combines online survey questions with an eye-tracking webcam, allowing participants to see images from their home computers.

Results indicated on the participants classified as Malingering had a significantly Visit Count (M = 17.16; SD= 4.99) compared to the ADHD(M = 12.53; SD= 43.92) and not-ADHD groups (M =11.51; SD=3.23). Results also indicated a statistically significant Area Under the Curve (AUC) = .784; SE = .067; p -.003; 95% CI = .652-.916. Optimal cutoffs suggest a Sensitivity of 50% with a False Positive Rate of 10%.

Results indicated that eye-tracking technology could help differentiate simulator malingerers from non-malingerers with ADHD. Eye-tracking research’ relates to a patchwork of fields more diverse than the study of perceptual systems. Due to their close relation to attentional mechanisms, the study’s results can provide an insight into cognitive processes related to malingering performance.

Traditional methods of assessing performance validity have numerous weaknesses, among them, results can be consciously manipulated by examinees who wish to feign cognitive impairment. This study tested the ability of pupillary dilation patterns during a performance validity test (PVT) to enhance diagnostic accuracy in discriminating true from feigned impairment of traumatic brain injury (TBI). Pupillometry provides information about physiological and psychological processes related to cognitive load, familiarity, and deception and is outside of conscious control. Patrick, Rapport, Kanser, Hanks, and Bashem (2021) established proof of concept for the utility of pupillometry with PVTs applied to the Test of Memory Malingering (TOMM). This study replicated and extended this work by evaluating the incremental utility of pupillary-derived indices on the Warrington Recognition Memory Test for Words (RMT).

Participants included 214 adults in three groups: adults with bona fide TBI (TBI; n = 51) healthy comparisons instructed to perform their best (HC; n = 72), and healthy adults instructed and incentivized to simulate cognitive impairment due to TBI (SIM; n = 91). Moreover, this study examined pupillary pattern differences among successful (i.e., failed < 1 PVT and performed impaired on cognitive tests) and unsuccessful (i.e., failed > 2 PVTs or did not score impaired on a cognitive test) SIM, including SIM who did and did not fail the RMT. The RMT was administered in the context of a comprehensive neuropsychological battery. Indices included two pure pupil dilation (PD) indices: a simple measure of baseline arousal (PD-Baseline) and a nuanced measure of dynamic engagement (PD-Range). A pupillo-behavioral index was also evaluated: Dilation-response inconsistency (DRI) captured the frequency with which examinees displayed a pupillary familiarity response to the correct answer but selected the unfamiliar stimulus (incorrect answer).

The results generally replicated Patrick et al. (2021), as all three indices were useful in discriminating between groups and provided incremental utility to traditional accuracy scores. PD-Baseline appeared sensitive to oculomotor dysfunction due to TBI (i.e., increasing accurate identification of that group); adults with TBI displayed significantly lower chronic arousal as compared to the two groups of healthy adults (SIM, HC). In fact, the TBI group showed significantly lower PD-Baseline than both unsuccessful simulators who were detected as feigners and successful simulators who passed PVTs but effectively feigned TBI on other tests. Dynamic engagement (PD-Range) yielded a hierarchical structure such that SIM were more dynamically engaged than TBI followed by HC. As predicted, simulators engaged in DRI significantly more frequently than other groups. Moreover, DRI added unique information to RMT accuracy in classifying unsuccessful simulators from all other groups. Each of these three pupillary indices showed large effect sizes, and logistic regressions indicated that each contributed unique variance in predicting group membership on one or more of the paired contrasts (i.e., SIM-TBI, SIM-HC, HC-TBI).

Taken together, the findings support continued research on the application of pupillometry to performance validity assessment: Pupillometry provided unique information in enhancing classification accuracy beyond traditional PVT accuracy scores. Overall, the findings highlight the promise of biometric indices in multimethod assessments of performance validity.

Although studies have shown unique variance contributions from performance invalidity, it is difficult to interpret the meaning of cognitive data in the setting of failed performance validity tests (PVT). Furthermore, a clearer understanding of the clinical utility of cognitive data in the context of invalid PVTs is necessary to inform decisions about battery length once PVTs are failed. The primary aim of the current study is to broadly describe cognitive outcomes in the setting of PVT failure.

Two hundred and twenty-two veterans with a history of mild traumatic brain injury referred for clinical evaluation completed cognitive and performance validity measures. Standardized scores were characterized as Within Normal Limits and Below Normal Limits at the normative 16th percentile and number of Within Normal Limits scores were calculated for each participant. Cognitive outcomes are described across four commonly used PVTs. Rates of below normal limits cognitive performance, and PVT failure were assessed via student’s t tests among participants who were classified as productive or unproductive based on involvement in work and/or school.

Among participants who performed in the invalid range on TOMM trial 1, 36-81% of cognitive data reflected within normal limits performance. Similarly, 47-81% of those who demonstrated performance invalidity based on the Word Memory Test (WMT) earned broadly within normal limits scores across cognitive testing. For those with invalid performance based on the normative digit span scaled score, 35-88% of cognitive data was at or above the 16th percentile. Within normal limits across cognitive tests ranged from 16-71% when the California Verbal Learning Test-Second Edition forced choice was used as an indicator of performance validity. In the context of PVT failure, the average number of cognitive performances below the 16th percentile ranged from 5-7 of 14 tasks depending on which PVT measure was applied. Within the total sample, there were no differences in the total number of below normal limits performances on cognitive measures between productive and unproductive participants (T = 1.65, p = 1.00). Additionally, there were no differences in the total number of PVTs failed between the productive and unproductive groups (T = 0.33, p = 0.743).

Results of the current study suggest that the range of within normal limits cognitive performance in the context of failed performance validity measures varies greatly. Importantly, findings indicate that neurocognitive data may still provide important practical information regarding cognitive abilities (i.e., that test takers can oftentimes perform within broadly normal limits on many cognitive tasks), despite poor PVT outcomes. Further, given that neither rates of below normal limits cognitive performance nor rates of PVT failures differed among productivity groups, results have important implications for decisions to continue testing and recommendations in a clinical setting.

Research has found that child molesters (both pedophilic and non-pedophilic) tend to have poorer executive functioning (EF), particularly inhibition, as compared to other types of criminal offenders (Eastvold, Suchy, Strassberg, & 2011; Suchy, Whittaker, Strassberg, & Eastvold, 2009). Poorer performance on measures of inhibition may have different mechanisms for pedophilic child molesters (PCM; i.e., those offenders who are sexually attracted only to children) than non-pedophilic child molesters (N-PCM; i.e., those offenders whose sexual attraction is not limited to children). Specifically, poor inhibition in PCM may be explained by slower processing speed (Suchy, et al., 2009; Suchy, Eastvold, Strassberg, & Franchow, 2014), whereas it may be explained by impulsive errors in N-PCM (Eastvold, Suchy, & Strassberg, 2011). Intraindividual variability (IIV) refers to transient, short-term fluctuations in performance (Nesselroade, 1991). IIV is sometimes interpreted as a measure of cognitive control, an aspect of EF that could impact performance speed and accuracy due to poorer focus.

Greater IIV also appreas to be associated with greater vulnerability to EF depletion after behavioral inhibition (DesRuisseaux, Suchy, & Franchow, 2021), which could represent a mechanism whereby vulnerability to EF depletion could be a precursor of offense. However, given that poor performance on measures of inhibition seem to have different underlying mechanisms for PCM and N-PCM, it is unclear whether both groups would exhibit greater IIV compared to non-sexual offenders.

Participants were PCM (n = 76, M age = 33.61(7.74), Range = 1947; 92.1% White, 2.6% Hispanic/Latino, 2.6% Native American, 1.3% Black, 1.3% Other), N-PCM (n = 52, M age = 30-88(6.37), Range = 2045; 73.1% White, 13.5% Hispanic/Latino, 7.7% Other, 3.8% Native American, 1.9% Black), and non-sexual offenders (n = 25, M age = 29.96(8.16), Range = 22-45; 80% White, 8% Hispanic/Latino, 8% Other, 8% Asian) recruited as part of two larger studies examining cognition in sex offenders. IIV was assessed using the Push-Turn-Tap-Tap (PTT) task, an experimental computerized measure of EF from which IIV can be calculated (DesRuisseaux et al., 2021).

Independent samples t-tests found that both PCM and N-PCM had greater IIV than nonsexual offenders (t(99) = 2.13, p = .04; t(75) = 2.23, p = .03, respectively). Even on their fastest responses, PCM had greater time elapsed between correct sequences (i.e., slower response style; t(126) = 2.23, p = .03) than N-PCM. There were no significant differences in error rates between any groups (p > .05).

These results suggest that IIV varies between sexual and non-sexual offenders but does not vary between PCM and N-PCM. This is consistent with prior research suggesting that both PCM and N-PCM have poorer EF than non-sexual offenders. Additionally, consistent with prior research, PCM had a slower response style than N-PCM and non-sexual offenders. Unlike prior research that has found significant differences in accuracy rates between PCM and N-PCM, the present results did not find a significant difference. Since IIV has been associated with increased likelihood of EF depletion (likely increasing risk of lapses), future research could examine whether CM with greater IIV have an increased likelihood of reoffending.

Performance validity tests (PVTs) provide a methodological approach to detecting credible neurocognitive performances. This proves invaluable to the diagnostic process, as it allows neuropsychologists to objectively determine if an evaluation reflects a patient’s true neurocognitive abilities or if external factors are impacting the results. However, their addition to a testing battery can increase an already lengthy evaluation. As such, there is a need for sensitive but less time intensive PVTs. The purpose of this study is to validate the Coin-in-Hand (CIH) procedure as a quick and effective PVT within a veteran population.

68 English-speaking patients were identified from an outpatient neuropsychological assessment dataset. Performances were correlated to the well- validated Reliable Digit Span (RDS), and several other soft indicators of task engagement including expanded COWAT, BVMT-False Alarms (FA), WCST Failure to Maintain Set (FTM), TOMM, and the RBANS Effort Index (EI). All participants attempted CIH and RDS, testing was discontinued if 2 or more PVTs were invalid. An AUC analysis was conducted to determine how well the CIH discriminated between valid and invalid performance and determine the tests optimal cut-off score (sensitivity > 0.90 while maintaining the highest possible specificity). Logistic Regression was conducted to determine how well the CIH predicted performance validity.

Subject mean(SD) age and education were 55.25 (16.06) and 13.41 (2.55) years, respectively. 17% female, 60% Caucasian, and 32% Black. Descriptive statistics for each of the other performance validity tests were gathered. The CIH demonstrated low diagnostic accuracy (AUC = .66; p >.05; CI = .51 -.81); a cut score of <8 resulted in a sensitivity of .96 and a specificty of .64. Logistic Regression showed that CIH performance significantly predicted performance validity (X2 = -0.93; df = 1; N = 68; p < .05), accounting for 18-28% of the variance in performance classification (Cox & Snell R2 = .18; Nagelkerke R2 = .28). It correctly classified 96% of valid performers, but only correctly classified 35% of invalid performers, with an overall correct prediction rate of 83%. A predicted chase in log odds (B= -.93) and odd ratio [Exp (B) =.40] indicated that every unit increase in CIH score was associated with a decrease probability of performance invalidity. Logistic regression was also used to calculate the probability of performance invalidity at each possible CIH score (Table 1).

Results suggests that poor performance on CIH does not necessarily equate to invalid performances, but instead, should act as a screener to cue neuropsychologists working with Veterans that additional PVTs should be considered. Overall, it was determined that CIH was able to correctly predict 35% of invalid performers and 96% of valid performers, with an overall correct prediction rate of 83%, suggesting the procedure may be too simple to be an effective standalone PVT for clinical use. These results also highlight that every correct response on the CIH was associated with a decreased probability of performance invalidity. Additionally, an AUC analysis determined the tests optimal cut off score to be <8, suggesting that shortening the procedure may be as effective as giving the full 10 trials.

Cognitive tests requiring spoken responses, such as paragraph recall, are rich in cognitive-related information that is not captured using traditional scoring methods. This study aimed to determine if linguistic features embedded in spoken responses may differentiate between individuals who are and are not cognitively impaired.

Participants in the Long Life Family Study completed a neuropsychological assessment which included the WMS-R Logical Memory I paragraph recall. For a subset of participants (N=709), test responses were digitally recorded and manually transcribed. We used Linguistic Inquiry Word Count, a text analysis program, to quantify word counts, grammatical features (e.g, prepositions, verb tenses), and the use of content words related to specific semantic categories (e.g., work-related, numbers) for immediate (IR) and delayed recall (DR). We used regression models with Generalized Estimating Equations adjusted by age, sex, education, and within-family correlation to select features associated with cognitive status (normal cognition [NC] versus cognitive impairment [CI]; Bonferroni-corrected threshold p<0.001). Next, we developed a “polyfeature score” (PFS) for both immediate and delayed recall, each calculated as a weighted sum of the selected linguistic features. We then built a logistic regression model to evaluate the predictive value of each PFS for identifying cognitively impaired individuals. In secondary analyses, we used regression models as above to identify features associated with mild cognitive impairment subtype (amnestic [aMCI] versus nonamnestic [naMCI]; threshold p< .05).

The sample included 599 participants with NC and 110 with CI (mean age = 72.3 ± 11.0 years, 54% female). The regression identified 8 linguistic features for IR and 7 for DR that significantly predicted cognitive status. Decreased use of content words related to work (e.g., employed, school, police) and biological processes (e.g., cook, cafeteria, eat) and the use of negations (e.g., no, not, can’t) were predictive of cognitive impairment in both recall conditions. In contrast, the use of other content word categories were predictive of cognitive status in only one recall condition (IR: leisure, cognitive processes, space; DR: drives, number). The use of fewer prepositions in IR, more first-person pronouns in DR, and fewer words in the past tense in DR were each associated with cognitive impairment. Word count was not predictive of cognitive status. Both PFSs were highly associated with cognitive status (PFS_IR ß= 0.74, p< 0.001; PFS_DR ß= 0.86, p= 0.001) with high discriminative value (PFS_IR AUC= 0.93, sensitivity = 0.81, specificity= 0.91; PFS_DR AUC= 0.95, sensitivity= 0.77, specificity= 0.88). In the CI subset, linguistic features differed between those classified as aMCI (n= 24) and naMCI (n= 40). Two function word categories predicted aMCI in IR whereas decreased word count, two function word categories, and two content word categories predicted aMCI in DR (all p< .05)

Linguistic features from paragraph recall provide high predictive value for classifying cognitive status increasing its potential as a cognitive screener in clinical settings. Additionally, each recall condition identified unique linguistic features associated with cognitive impairment which may aid differentiation of cognitive impairment subtypes and elucidate processes underlying deficits in learning and recall.

Recent research has found that machine learning based analysis of patient speech can be used to classify Alzheimer’s Disease. We know of no studies, however, which systematically explore the value of pausing events in speech for detecting cognitive limitations. Using retrospectively acquired voice data from paragraph memory tests, we created two types of pause features: a) the number and duration of pauses, and b) frequency components in speech immediately following pausing. Multiple machine learning models were used to assess how these features could effectively discriminate individuals classified into two groups: Cognitively Compromised versus Cognitively Well.

Participants (age> 65 years, n= 67) completed the Newcomer paragraph memory test and a neuropsychological protocol as part of a federally funded prospective IRB approved investigation at the University of Florida. Participant vocal recordings were acquired for the immediate and delay conditions of the test. Speaker diarization was performed on the immediate free recall test condition to separate voices of patients from examiners. Features extracted from both test conditions included a) 3 pause characteristics (total number of pauses, total pause duration, and length of the longest pause), and b) 20 Mel Frequency Cepstral Coefficients (MFCC) pertaining to speech immediately (2.7 seconds) following pauses. These were combined with demographics (age, sex, race, education, and handedness) to create a total of 105 features that were used as inputs for multiple machine learning analytic models (random forest, logistic regression, naive Bayes, AdaBoost, Gradient Boost, and multi-layered perceptron). External neuropsychological metrics were used to initially classify Cognitively Compromised (i.e., < -1.0 standard deviation on > two of five test metrics: total immediate, delay, discrimination Hopkins Verbal Learning Test-Revised (HVLT-R),

Controlled Oral Word Association (COWA) test, category fluency ('animals')). Pearson Product Moment Correlations were used to assess the linear relationships between pauses and speech frequency categories and neuropsychological metrics.

Neuropsychology metric classification using -1SD cut-off identified 27% (18/67 participants) as Cognitively Compromised. The Cognitively Compromised group and the Cognitively Well group did not show any difference in distributions of individual pause/frequency features (Mann Whitney U-test, p> 0.11). A negative correlation was found between total duration of short pauses and HVLT total immediate free recall, while a positive correlation was found between MFCC-10 and HVLT total immediate free recall. The best classification model was AdaBoost Classifier which predicted the Cognitively Compromised label with 0.91 area under receiver operating curve, 0.81 accuracy, 0.43 sensitivity, 1.0 specificity, 1.0 precision, 0.6 f1 score.

Pause characteristics and frequency profiles of speech immediately following pauses from a paragraph memory test accurately identified older adults with compromised cognition, as measured by verbal learning and verbal fluency metrics. Furthermore, individuals with reduced HVLT immediate free recall generated more pauses, while individuals who recalled more words had higher power in mid-frequency bands (10th MFCC). Future research needs to replicate how paragraph recall pause characteristics and frequency the profile of speech immediately following pauses potentially provides a low resource alternative to automatic speech recognition models for detecting cognitive impairments.

Digitized cognitive assessment captures rich behavioral information that remains unmeasured using conventional methods. Data capture tools recently accessible only in specialized laboratories are now feasible at scale using off-the-shelf tablet devices. This study aims to share data from a digitized cognitive assessment embedded in an open-science research program collecting extensive neuroimaging, health, behavioral, neuropsychological, and psychiatric characterizations to advance translational cognitive neuroscience. In this research we present normative performance metrics from a digital version of the Trail Making Test.

The NKI-Rockland Sample (NKI-RS) has provided a model for openly-shared lifespan normative neuroimaging resources contributed by a community-ascertained sample (n=1,500, aged 6-85) and generating over 400 publications across diverse research areas. The next generation NKI-RS study (recruitment target= 600, aged 9-75) aims to enrich these resources for brain-behavioral research, normative reference, and biomarker discovery. One focus of innovation is the inclusion of digitized cognitive assessments (DCAs) utilizing an open-resource task development and data collection platform (Mindlogger, Child Mind Institute). We present preliminary data from a digitized version of the Trail Making Tests and report early descriptive metrics. The TMTs was administered via an iPad Pro using an Apple pen as part of a laboratory-based EEG procedure. The TMTs follows standard administration instructions, including a practice sample before each test condition. Error feedback is included in the task implementation such that an incorrect connection is marked with an “x” and the participant is directed to the last correct circle to continue. Feedback is automated within the task. Pixel-level spatial resolution and millisecond timing is captured across all drawing tasks. Task design, implementation, and preliminary performance metrics including speed, accuracy, and variability are reported.

Preliminary data include 12 participants from the NKI-RS2 study ranging in age from 11-75 years (M= 52.83, SD= 19.97); 67% female. Overall participants took longer to complete condition B (Mb = 51.71 secs) compared to condition A (Ma = 23.07 secs), p= 0.0005. Connections were made more slowly (Ma = 37.47 secs vs. Mb = 24.50 secs, p< 0.001) and connection speed was more variable (CVa = 0.90 vs. CVb = 1.22, p< 0.01) on condition B versus A. Connection speed decreased and speed variability increased with age (t[11 ]= -3.25, p= 0.05, t[11]= -3.63, p< 0.01, respectively). Time spent within circles (dwell time) was significantly greater in B versus A (t[11]= 6.81, p< 0.001). Number of errors were limited (MA = .89 and MB = 1.0, range 0-2 in both tests) with no difference between tests or effects of age (both ps >0.05).

These preliminary data from the NKI-RS2 normative neuroimaging study demonstrate that a digitized version of a classic neuropsychological test is feasible across a diverse range of community participants, and replicates known age effects. The advantages of growing access to these DCA tools and the shared data resources they will produce has the potential to revolutionize neuropsychological research and clinical practice.

Digital cognitive assessments (DCAs) provide insight into cognition and behavior that remains inaccessible through standard assessment approaches. However, the availability of DCAs and the requisite toolkits to extract and analyze meaningful features from these datasets are largely constrained to technical specialists or through fee-for-service commercial entities. The NKI- Rockland Sample provides a large- scale lifespan data sample featuring DCAs, and also openly shares its DCA tasks through the open-source MindLogger platform along with pipelines for feature extraction and analyses. Here we present normative performance from a digital version of Archimedes Spiral Drawing.

NKI-RS2 participants were largely drawn from the existing NKI-RS participant pool (n= 1,500), aged 885. The NKI-RS2 is in year 1 of data collection; here, we report on a subset of participants (n= 9) who performed a digitized version of the Archimedes Spiral Drawing task. This graphomotor task with well-established research and clinical utility in movement disordered populations was adapted for use for off-the-shelf tablet devices. The NKI-RS2 implements these tasks on an Apple iPad Pro2, sampling participant drawing at 120Hz, and featuring pixel- and millisecond- level resolution for all tasks. On the Spiral Drawing and Recall Tests participants traced five Archimedes spirals from the center outward through four windings presented on the iPad. They were then asked to replicate the spiral freehand three times. From these spiral drawings, we extracted time to completion, distance covered, speed/ speed variability, rotational smoothness, number of crossings, mean absolute error, bias, and goodness of fit to the ideal Archimedes spiral.

Comparing the tracing and recall conditions, participants showed significantly faster drawing speed (t[8]=5.32, p< .001), more variable drawing speed (t[8]=5.93, p< .001), reduced goodness of fit to the template t[8]=4.99, p< .002, and reduced rotational smoothness (t[8]=7.43, p< .0003) in the recall conditions. Collapsing across conditions, age predicted more variable drawing speed: t[8]= 2.77 p< .019, greater tracing error (t[8] = 2.69, p< .0227), and reduced rotational smoothness (t[8] = 2.67, p< .024). Between conditions, age predicts a greater increase in drawing speed variability (t[8] = 9.76, p< .0006).

Using the open source MindLogger platform and off-the-shelf digital tablets, we were able to replicate classic paper and pen neuropsychological tests. By adapting these tasks to DCA, we were able to extract meaningful features that are not otherwise accessible (drawing speed, variability, etc.), or that would require additional hardware solutions (e.g., dwell time). By making these tasks and their processing pipelines available, the NKI-RS2 can facilitate the democratization of DCA and DCA analysis to a broader range of researchers and clinicians.

Following the start of the SARS-COV-2 (COVID-19) pandemic there was a rapid uptake in teleneuropsychology (TeleNP). Many clinicians and researchers used videoconferencing technologies (e.g., Zoom®) to conduct remote neuropsychological assessments. Prior reviews (e.g., Marra et al., 2020) have indicated promise for the use of videoconference-based approaches to cognitive assessment under certain circumstances, though arguably nobody foresaw the widespread use of teleNP during the pandemic. Given the rapid expansion in the teleNP literature in the past couple of years, in this scoping review we specifically discuss research updates made during the COVID-19 pandemic pertaining to teleNP assessment of adults conducted via videoconferencing and their potential clinical applications.

GoogleScholar and PubMed were used to search for peer-reviewed original research articles published between January 1, 2020 (i.e., the approximate beginning of the COVID-19 pandemic) and August 1, 2022. Broad search terms were used pertaining to teleNP, remote cognitive assessment, videoconferencing, and neuropsychological assessment, resulting in 16 articles.

Though most of the included studies were based in the United States (n=5), there was international representation across studies (Chile=1; United Kingdom=1; Australia=2; New Zealand=1; France=2; Greece=1; Japan=2, Singapore=1). All of the identified articles examined TeleNP-related research questions using cognitive tests administered via videoconferencing that have been previously studied in-person to varying degrees. Several of the studies focused on psychometric characterization (i.e., reliability and validity) of the examined tests when delivered via videoconferencing, whereas others focused on demonstrating the relative equivalence of neuropsychological scores obtained via videoconferencing versus in-person evaluations.

Formal psychometric studies of traditional in-person neuropsychological tests delivered via videoconferencing since the start of the COVID-19 pandemic suggest that this remote modality of assessment is generally reliable and valid. Moreover, multiple recent studies have demonstrated relative equivalence of neuropsychological scores obtained via videoconferencing versus neuropsychological test scores obtained in-person. When considered alongside teleNP research conducted prior to the COVID-19 pandemic (e.g. Cullum et al., 2014), recent studies on videoconference-based neuropsychological assessment indicate that videoconferencing may not necessarily be a complete substitute for an in-person comprehensive evaluation given the inherent limitations of the procedure. However, teleNP via videoconferencing may be a promising tool in the neuropsychologist’s toolbox because it can help reduce common barriers to in-person neuropsychological assessment (e.g., travel time to clinics). Additional research on videoconferencing-based cognitive assessment is needed, especially in low-and-middle income countries (LMIC) and diverse populations where there may be more economic barriers to remote neuropsychological assessment relative to more economically-developed countries. Notably it is possible that research from LMIC may have been missed through the screening processes used in this review (e.g., inclusion of articles written in English).

Mobile, valid, and engaging cognitive assessments are essential for detecting and tracking change in research participants and patients at risk for Alzheimer’s Disease and Related Dementias (ADRDs). This pilot study aims to determine the feasibility and performance of app-based memory and executive functioning tasks included in the mobile cognitive app performance platform (mCAPP), to remotely detect cognitive changes associated with aging and preclinical Alzheimer’s Disease (AD).

The mCAPP includes three gamified tasks: (1) a memory task that includes learning and matching hidden card pairs and incorporates increasing memory load, pattern separation features (lure vs. non-lure), and spatial memory (2) a stroop-like task (“brick drop”) with speeded word and color identification and response inhibition components and (3) a digit-symbol coding-like task (“space imposters”) with increasing pairs and incidental learning components. The cohort completed the NACC UDS3 neuropsychological battery, selected NIH Toolbox tasks, and additional cognitive testing sensitive to pre-clinical AD, within six months of the mCAPP testing. Participants included thirty-seven older adults (60% female; age=72±4.4, years of education=17±2.5; 67% Caucasian, 30% Black/AA, 3% Multiracial) with normal cognition who are enrolled in the Penn Alzheimer’s Disease Research Center (ADRC) cohort. Participants completed one in-person session and two weeks of at-home testing, with eight scheduled sessions, four in the morning and four in the afternoon. Participants also completed questionnaires and an interview about technology use and wore activity trackers to collect daily step and sleep data and answered questions about mood, anxiety, and fatigue throughout the two weeks of at-home data collection.

The participants completed an average of 11 at-home sessions, with the majority choosing to play extra sessions. Participants reported high usability ratings for all tasks and the majority rated the task difficulty as acceptable. On all mCAPP tasks, participant performance declined in accuracy and speed with increasing memory load and task complexity. mCAPP tasks correlated significantly with paper and pencil measures and several NIH Toolbox tasks (p<0.05). Examination of performance trends over multiple sessions indicates stabilization of performance within 4-6 sessions on memory mCAPP measures and 5-7 sessions on executive functioning mCAPP measures. Preliminary analyses indicate differences in mCAPP measures and imaging biomarkers.

Participants were willing and able to complete at-home cognitive testing and most chose to complete more than the assigned sessions. Remote data collection is feasible and well-tolerated. We show preliminary construct validity with the UDS3 and NIH Toolbox and test-retest reliability following a period of task learning and performance improvement and stabilization. This work will help to advance remote detection and monitoring of early cognitive changes associated with preclinical AD. Future directions will include further evaluation of the relationships between mCAPP performance, behavioral states, and neuroimaging biomarkers as well as the utility of detection of practice effects in identifying longitudinal change and risk for ADRD-related cognitive decline.