Determine the classification concordance between a standalone performance validity test (PVT) and embedded PVTs from multiple cognitive domains.

Participants were 106 patients (49.1% female; 69% white) that underwent neuropsychological evaluation at an outpatient university doctoral clinical psychology training and research clinic (M/SD: age = 32.38/11.95; education = 13.7/2.75). A comprehensive neuropsychological battery included the Medical Symptom Validity Test (MSVT) and embedded PVTs from different cognitive domains: attention - Wechsler Adult Intelligence Scale - Fourth Edition Reliable Digit Span and Digit Span age-corrected scaled score (DS ACSS); memory - California Verbal Learning Test, 3rd Edition (CVLT-3) Forced-Choice Recognition (FCR), executive functions -Wisconsin Card Sorting Test (WCST) Failure to Maintain Set (FMS); visual-spatial/construction -Rey Complex Figure Test (RCFT) Copy raw score; language - Boston Diagnostic Aphasia Examination Complex Ideation Material (CIM); and motor functions - Finger Tapping Test (FTT). All participants were administered the MSVT but not all participants were administered all seven embedded PVTs. Credible/noncredible classification concordance rates and kappa correlations (i.e., percentage of agreement) were computed for each pairwise PVT combination.

Twenty-two percent (n = 23) of the sample failed at least one PVT, with 17.0% (n = 18) failing at least two. DS ACSS was the embedded PVT with the highest MSVT concordance rate at 92.4% and a fair kappa coefficient of .39; WCST FMS had the lowest concordance with MSVT at 82.9% and a slight kappa coefficient of .19. The highest concordance among embedded PVTs from different cognitive domains was CVLT-3 FCR and RCFT Copy raw score at 89.7% with a fair kappa coefficient of .35; the lowest agreement among embedded PVTs was WCST FMS and FTT at 74.0% with a kappa coefficient of -.02. More conservative kappa coefficients among all pair-wise embedded PVT combinations from different cognitive domains ranged from -.02 to .36. For all standalone and embedded PVT pairwise concordance rates, only two fell below the recommended minimum agreement of 80%: FCR vs. FMS = 79.3% and FMS vs. FTT = 74.0%.

Embedded PVTs across various cognitive domains have high agreement with a standalone PVT to aid in classifying noncredible performance, in the 83-92% range. Embedded PVTs from different cognitive domains also have mostly high agreement classification rates amongst themselves in aiding to determine noncredible performance, in the 74-90% range, with the lowest agreement rate between executive function and motor tests at 74%. More conservative kappa-based agreements between PVT pair-wise combinations were fairly consistent with other studies, with most being in the fair range. Finally, these findings indicate about a 17% base rate of noncredible cognitive performance in an outpatient university-based clinic.