A short yet reliable cognitive measure is needed that separates treatment and placebo for treatment trials for Alzheimer’s disease. Hence, we aimed to shorten the Alzheimer’s Disease Assessment Scale Cognitive Subscale (ADAS-Cog) and test its use as an efficacy measure.

Secondary data analysis of participant-level data from five pivotal clinical trials of donepezil compared with placebo for Alzheimer’s disease (N = 2,198). Across all five trials, cognition was appraised using the original 11-item ADAS-Cog. Statistical analysis consisted of sample characterization, item response theory (IRT) to identify an ADAS-Cog short version, and mixed models for repeated-measures analysis to examine the effect sizes of ADAS-Cog change on the original and short versions in the placebo versus donepezil groups.

Based on IRT, a short ADAS-Cog was developed with seven items and two response options. The original and short ADAS-Cog correlated at baseline and at weeks 12 and 24 at 0.7. Effect sizes based on mixed modeling showed that the short and original ADAS-Cog separated placebo and donepezil comparably (ADAS-Cog original ES = 0.33, 95% CI = 0.29, 0.40, ADAS-Cog short ES = 0.25, 95% CI =0.23, 0.34).

IRT identified a short ADAS-cog version that separated donepezil and placebo, suggesting its clinical potential for assessment and treatment monitoring.

Data from neurocognitive assessments may not be accurate in the context of factors impacting validity, such as disengagement, unmotivated responding, or intentional underperformance. Performance validity tests (PVTs) were developed to address these phenomena and assess underperformance on neurocognitive tests. However, PVTs can be burdensome, rely on cutoff scores that reduce information, do not examine potential variations in task engagement across a battery, and are typically not well-suited to acquisition of large cognitive datasets. Here we describe the development of novel performance validity measures that could address some of these limitations by leveraging psychometric concepts using data embedded within the Penn Computerized Neurocognitive Battery (PennCNB).

We first developed these validity measures using simulations of invalid response patterns with parameters drawn from real data. Next, we examined their application in two large, independent samples: 1) children and adolescents from the Philadelphia Neurodevelopmental Cohort (n = 9498); and 2) adult servicemembers from the Marine Resiliency Study-II (n = 1444).

Our performance validity metrics detected patterns of invalid responding in simulated data, even at subtle levels. Furthermore, a combination of these metrics significantly predicted previously established validity rules for these tests in both developmental and adult datasets. Moreover, most clinical diagnostic groups did not show reduced validity estimates.

These results provide proof-of-concept evidence for multivariate, data-driven performance validity metrics. These metrics offer a novel method for determining the performance validity for individual neurocognitive tests that is scalable, applicable across different tests, less burdensome, and dimensional. However, more research is needed into their application.

Although the psychometric properties of the Family Satisfaction with End-of-Life Care measure have been examined in diverse settings internationally; little evidence exists regarding measurement equivalence in Hispanic caregivers. The aim was to examine the psychometric properties of a short-form of the FAMCARE in Hispanics using latent variable models and place information on differential item functioning (DIF) in an existing family satisfaction item bank.

The graded form of the item response theory model was used for the analyses of DIF; sensitivity analyses were performed using a latent variable logistic regression approach. Exploratory and confirmatory factor analyses to examine dimensionality were performed within each subgroup studied. The sample included 1,834 respondents: 317 Hispanic and 1,517 non-Hispanic White caregivers of patients with Alzheimer's disease and cancer, respectively.

There was strong support for essential unidimensionality for both Hispanic and non-Hispanic White subgroups. Modest DIF of low magnitude and impact was observed; flagged items related to information sharing. Only 1 item was flagged with significant DIF by both a primary and sensitivity method after correction for multiple comparisons: “The way the family is included in treatment and care decisions.” This item was more discriminating for the non-Hispanic, White responders than for the Hispanic subsample, and was also a more severe indicator at some levels of the trait; the Hispanic respondents located at higher satisfaction levels were more likely than White non-Hispanic respondents to report satisfaction.

The magnitude of DIF was below the salience threshold for all items. Evidence supported the measurement equivalence and use for cross-cultural comparisons of the short-form FAMCARE among Hispanic caregivers, including those interviewed in Spanish.

There is a need for accurate and efficient assessment tools that cover a range of mental health and psychosocial problems. Existing, lengthy self-report assessments may reduce accuracy due to respondent fatigue. Using data from a sample of adults enrolled in a psychotherapy randomized trial in Thailand and a cross-sectional sample of adolescents in Zambia, we leveraged Item Response Theory (IRT) methods to create brief, psychometrically sound, mental health measures.

We used graded-response models to refine scales by identifying and removing poor performing items that were not well correlated with the underlying trait, and by identifying well-performing items at varying levels of a latent trait to assist in screening or monitoring purposes.

In Thailand, the original 17-item depression scale was shortened to seven items and the 30-item Posttraumatic Stress Scale (PTS) was shortened to 10. In Zambia, the Child Posttraumatic Stress Scale (CPSS) was shortened from 17 items to six. Shortened scales in both settings retained the strength of their psychometric properties. When examining longitudinal intervention effects in Thailand, effect sizes were comparable in magnitude for the shortened and standard versions.

Using Item Response Theory (IRT) we created shortened valid measures that can be used to help guide clinical decisions and function as longitudinal research tools. The results of this analysis demonstrate the reliability and validity of shortened scales in each of the two settings and an approach that can be generalized more broadly to help improve screening, monitoring, and evaluation of mental health and psychosocial programs globally.

To conduct advanced psychometric analysis of Primary Care Assessment Tool (PCAT) in Tibet and identify avenues for metric performance improvement.

Measuring progress toward high-performing primary health care can contribute to the achievement of sustainable development goals. The adult version of PCAT is an instrument for measuring patient experience, with key elements of primary care. It has been extensively used and validated internationally. However, only little information is available regarding its psychometric properties obtained based on advanced analysis.

We used data collected from 1386 primary care users in two prefectures in Tibet. First, iterative confirmatory factor analysis examined the fit of the primary care construct in the original tool. Then item response theory analysis evaluated how well the questions and individual response options perform at different levels of patient experience. Finally, multiple logistic regression modeling examined the predicative validity of primary care domains against patient satisfaction.

A best final structure for the PCAT-Tibetan includes 7 domains and 27 items. Confirmatory factor analysis suggests good fit for a unidimensional model for items within each domain but doesn’t support a unidimensional model for the entire instrument with all domains. Non-parametric and parametric item response theory analysis models show that for most items, the favorable response option (4 = definitely) is overwhelmingly endorsed, the discriminability parameter is over 1, and the difficulty parameters are all negative, suggesting that the items are most sensitive and specific for patients with poor primary care experience. Ongoing care is the strongest predictor of patient satisfaction. These findings suggest the need for some principles in adapting the tool to different health system contexts, more items measuring excellent primary care experience, and update of the four-point response options.