The psychometric rigor of unsupervised, smartphone-based assessments and factors that impact remote protocol engagement is critical to evaluate prior to the use of such methods in clinical contexts. We evaluated the validity of a high-frequency, smartphone-based cognitive assessment protocol, including examining convergence and divergence with standard cognitive tests, and investigating factors that may impact adherence and performance (i.e., time of day and anticipated receipt of feedback vs. no feedback).

Cognitively unimpaired participants (N = 120, Mage = 68.8, 68.3% female, 87% White, Meducation = 16.5 years) completed 8 consecutive days of the Mobile Monitoring of Cognitive Change (M2C2), a mobile app-based testing platform, with brief morning, afternoon, and evening sessions. Tasks included measures of working memory, processing speed, and episodic memory. Traditional neuropsychological assessments included measures from the Preclinical Alzheimer’s Cognitive Composite battery.

Findings showed overall high compliance (89.3%) across M2C2 sessions. Average compliance by time of day ranged from 90.2% for morning sessions, to 77.9% for afternoon sessions, and 84.4% for evening sessions. There was evidence of faster reaction time and among participants who expected to receive performance feedback. We observed excellent convergent and divergent validity in our comparison of M2C2 tasks and traditional neuropsychological assessments.

This study supports the validity and reliability of self-administered, high-frequency cognitive assessment via smartphones in older adults. Insights into factors affecting adherence, performance, and protocol implementation are discussed.

Remote assessment for cognitive screening and monitoring in the elderly has many potential advantages, including improved convenience/access and ease of repeat testing. As remote testing becomes more feasible and common, it is important to examine what factors might influence performance and adherence with these new methods. Personal beliefs about one’s ability to remember effectively have been shown to impact memory performance, especially in older adults (Lineweaver & Hertzog, 1998). The perception of a low level of personal control over memory may impact a person’s use of memory strategies which might otherwise enhance performance, as well as their beliefs about the efficacy of those strategies (Lineweaver et al., 2021). The present study examined the relationship between perceived memory self-efficacy and performance and adherence on self-administered, smartphonebased remote cognitive assessments.

Participants were 123 cognitively unimpaired adults (ages 55-80, 68.3% female, 87% White, M= 16.5 years of education) recruited from the Butler Hospital Alzheimer’s Prevention Registry as part of an ongoing study evaluating novel cognitive assessment methods. A cutoff of score of ≥34 on the modified Telephone Interview for Cognitive Status (TICSm) was required for enrollment. Perceived memory self-efficacy was assessed using two subscales of the Personal Beliefs about Memory Instrument (PBMI; Lineweaver et al., 1998): “prospective control”, the perception of control one currently has to influence future memory functioning, and “future control”, the perception of the amount of control over memory function one will have in the future. Participants completed three brief self-administered cognitive testing sessions per day for 8 consecutive days using a mobile app-based platform developed as part of the National Institute of Aging’s Mobile Toolbox initiative. Cognitive tasks assessed visual working memory (WM), processing speed (PS), and episodic memory (EM)(see Thompson et al., 2022).

Statistical analyses were conducted using univariate ANOVA tests to look for main effects of each PBMI subscale score on remote assessment adherence and average performance on each task over 8 days. After adjusting for aging, we found a higher rate of false alarms (proportion of misidentified stimuli) on the WM task was associated with higher levels of both self-reported prospective control (F(2, 86) = 4.188, p = .018) and future control (F(2, 96) = 5.003, p = .009). Increased response time on the PS task was also associated with higher levels of future control when adjusted for aging (F(2, 96) = 6.075, p = .003). There was no main effect of memory self-efficacy ratings on EM. We found no main effects of memory self-efficacy ratings on assessment adherence.

These findings suggest perceptions of high prospective and future control are associated with positive response bias on a forced-choice WM task, and high perceptions of future control are also associated with slower response times on PS tasks. Future research should examine whether this is due to increased deliberation, cautiousness, or other factors. Limitations include the potentially limited generalizability of this largely White, highly educated, and motivated sample self-selected for AD research. Next steps for this research include comparing these results with the effects of perceived self-efficacy on in-person cognitive assessments.

Intraindividual variability (IIV) is defined as fluctuations in an individual’s cognitive performance over time1. IIV has been identified as a marker of neurobiological disturbance making it a useful method for detecting changes in cognition among cognitively healthy individuals as well as those with prodromal syndromes2. IIV on laboratory-based computerized tasks has been linked with cognitive decline and conversion to mild cognitive impairment (MCI) and/or dementia (Haynes et al., 2017). Associations between IIV and AD risk factors including apolipoprotein (APOE) ε4 carrier status, neurodegeneration seen on brain imaging, and amyloid (Aß) Positron emission tomography (PET) scan status have also been observed1. Recent studies have demonstrated that evaluating IIV on smartphone-based digital cognitive assessments is feasible, has the capacity to differentiate between cognitively normal (CN) and MCI individuals, and may reduce barriers to cognitive assessment3. This study sought to evaluate whether such differences could be detected in CN participants with and without elevated AD risk.

Participants (n=57) were cognitively normal older adults who previously received an Aß PET scan through the Butler Hospital Memory and Aging Program. The sample consisted of primarily non-Hispanic (n=49, 86.0%), White (n=52, 91.2%), college-educated (M=16.65 years), females (n=39, 68.4%). The average age of the sample was 68 years old. Approximately 42% of the sample (n=24) received a positive PET scan result. Participants completed brief cognitive assessments (i.e., 3-4 minutes) three times per day for eight days (i.e., 24 sessions) using the Mobile Monitoring of Cognitive Change (M2C2) application, a mobile app-based cognitive testing platform developed as part of the National Institute of Aging’s Mobile Toolbox initiative (Sliwinski et al., 2018). Participants completed visual working memory, processing speed, and episodic memory tasks on the M2C2 platform. Intraindividual standard deviations (ISDs) across trials were computed for each person at each time point (Hultsch et al., 2000). Higher ISD values indicate more variability in performance. Linear mixed effects models were utilized to examine whether differences in IIV existed based on PET scan status while controlling for age, sex at birth, and years of education.

n interaction between PET status and time was observed on the processing speed task such that Aß- individuals were less variable over the eight assessment days compared to Aß + individuals (B= -5.79, SE=2.67, p=.04). No main or interaction effects were observed on the visual working memory task or episodic memory task.

Our finding that Aß- individuals demonstrate less variability over time on a measure of processing speed is consistent with prior work. No associations were found between IIV in other cognitive domains and PET status. As noted by Allaire and Marsiske (2005), IIV is not a consistent phenomenon across different cognitive domains. Therefore, identifying which tests are the most sensitive to early change is crucial. Additional studies in larger, more diverse samples are needed prior to widespread clinical use for early detection of AD.

Routine cognitive screening in the elderly may facilitate earlier diagnosis of neurodegenerative diseases and access to care and resources for patients and families. However, despite growing rates of Alzheimer's and related disorders (ADRD), the availability and implementation of cognitive screening for older adults in the US remains quite limited. Remote cognitive assessment via smartphone app may reduce several barriers to more widespread screening. We examined the validity of a remote app-based cognitive screening protocol in healthy older adults by examining remote task convergence with standard-person assessments and cerebral amyloid (Aß) status as an AD biomarker.

Participants (N =117) were cognitively unimpaired adults aged 60-80 years (67.5% female, 88% White, 75% education > 16 years). A portion had Aß PET imaging results available from prior research participation [(Aß positive (Aß+) n =26, and Aß negative (Aß-) n = 44]. A modified Telephone Interview for Cognitive Status (TICSm) cutoff score of >34 was used to establish unimpaired cognition. Participants completed 8 consecutive assessment days using Mobile Monitoring of Cognitive Change (M2C2), a smartphone app-based testing platform developed as part of the National Institute of Aging's Mobile Toolbox initiative. Brief (i.e., 3-4 minute) M2C2 sessions were assigned daily within morning, afternoon, and evening time windows. Tasks included measures of visual working memory (WM), processing speed (PS), and episodic memory (EM) (see Thompson et al., 2022). Participants then completed a battery of standard neuropsychological assessments in-person at a follow-up visit.

Participants completed 22.6 (SD = 2.6) out of 24 assigned sessions (3 sessions x 8 days) on average. Performance on all M2C2 tasks decreased significantly with age. Women performed significantly better on WM and EM tasks relative to men. There were no detectable significant differences in performance by race or education. Shorter mean reaction time on M2C2 PS trials predicted faster Trails A and B completion (ß = .26, p < .01, 95% CI [3.8, 23.3] and ß = .20, p < .05, 95% CI [.23, 6.8], respectively). Greater mean M2C2 WM accuracy predicted longer maximum backward digital span (ß = .24, p = .01, 95% CI [.02, .16]). Greater mean M2C2 EM accuracy predicted stronger Logical Memory delayed recall (ß = .33, p < .001, 95% CI [.004, .012]) and total immediate recall on the Free and Cued Selective Reminding Test (ß = .19, p < .05, 95% CI [.000, .006]). Moreover, EM significantly distinguished Aß- and Aß+ individuals (t (68) = 3.0, p < .01) with fair accuracy (AUC = .72).

Mean performance across 8-days on each M2C2 task predicted same-domain cognitive task performance on a standard assessment battery, with medium effect sizes. Performance on the EM task was also sensitive to cerebral Aß status, consistent with subtle memory changes implicated in the preclinical stage of AD. These findings support the validity of this remote testing protocol in healthy older adults, with implications for future efforts to facilitate accessible and sensitive cognitive screening for early detection of ADRD. Limitations include the restricted generalizability of this primarily white and college educated sample.

Clinical and experimental neuropsychology patients are not always able to complete a given test due to limitations in their functioning and it can lead to frustration and time wasted, leading researchers to examine the value of metrics that can be derived earlier in a test so as to ascertain and salvage useful information. The Trail Making Test (TMT) is an oft-utilized test of executive function and has been the focus of such exploration (e.g., first error vs. time to complete Trails B which can be lengthy in dementia cases and lead to discontinuation and loss of scorable data; Christidi et al., 2013; Correia et al., 2015). The present retrospective study utilized archival chart review to examine the association between a patient's diagnosis and occurrence of the first error on Trails B (TB1err).

De-identified data was culled from adult private practice records (n=137) in the northeastern United States (the study was conducted in compliance with local IRB review). Trails A and B times, as well as Digit Span scores (for checking construct validity) were pulled from reports, and Trails B record forms were scored to extract the enumerated stimulus where any first error was observed in the patient's rendering of the trail connecting alternating numbers and letters. Paired t-tests compared the average TB1err of normative individuals (no diagnosis) with patients with a primary diagnosis of mood disorder, traumatic brain injury (TBI), mild cognitive impairment (MCI), or dementia. Additionally, Pearson's correlations were computed comparing TB1err with Trails B time, and another test of executive function (Digit Span backwards).

The order of diagnoses according to the average occurrence of the first error on Trails B (from later, to sooner occurrence) was as follows: normative (no diagnosis), mood disorder, TBI, MCI, and finally dementia. There was a significant difference on this first error metric (TB1err) when comparing normative and dementia patients (p = .03; 8.3 vs 4.2 for the average enumeration of 1st error on Trails B). Furthermore, significant correlations were found between this derived TB1err metric and Digit Span backwards (r = .31; p <.001) as well as overall TrailsB performance (r = -.39; p < .001).

The present study adds to a growing literature on the utility of deriving test metrics to maximize useful data for clinical and experimental neuropsychology. Results from this retrospective chart review indicate additional validity data to support the use of extracting the first error on Trails B as a way to salvage useful data even when a patient may not be able to complete the full TMT as designed. In this preliminary sample there was a significant difference found for normative vs. dementia patients on this derived TB1err metric and suggests it is worthy of additional research to see if it can reliably differentiate various diagnoses. We expect this finding will also be useful in experimental designs wherein time is often limited and loss of data due to incomplete testing might be avoided by extracting the first error on TrailsB.