The ability to remotely monitor cognitive skills is increasing with the ubiquity of smartphones. The Mobile Toolbox (MTB) is a new measurement system that includes measures assessing Executive Functioning (EF) and Processing Speed (PS): Arrow Matching, Shape-Color Sorting, and Number-Symbol Match. The purpose of this study was to assess their psychometric properties.

MTB measures were developed for smartphone administration based on constructs measured in the NIH Toolbox® (NIHTB). Psychometric properties of the resulting measures were evaluated in three studies with participants ages 18 to 90. In Study 1 (N = 92), participants completed MTB measures in the lab and were administered both equivalent NIH TB measures and other external measures of similar cognitive constructs. In Study 2 (N = 1,021), participants completed the equivalent NIHTB measures in the lab and then took the MTB measures on their own, remotely. In Study 3 (N = 168), participants completed MTB measures twice remotely, two weeks apart.

All three measures exhibited very high internal consistency and strong test-retest reliability, as well as moderately high correlations with comparable NIHTB tests and moderate correlations with external measures of similar constructs. Phone operating system (iOS vs. Android) had a significant impact on performance for Arrow Matching and Shape-Color Sorting, but no impact on either validity or reliability.

Results support the reliability and convergent validity of MTB EF and PS measures for use across the adult lifespan in remote, self-administered designs.

Children with CHD or born very preterm are at risk for brain dysmaturation and poor neurodevelopmental outcomes. Yet, studies have primarily investigated neurodevelopmental outcomes of these groups separately.

To compare neurodevelopmental outcomes and parent behaviour ratings of children born term with CHD to children born very preterm.

A clinical research sample of 181 children (CHD [n = 81]; very preterm [≤32 weeks; n = 100]) was assessed at 18 months.

Children with CHD and born very preterm did not differ on Bayley-III cognitive, language, or motor composite scores, or on expressive or receptive language, or on fine motor scaled scores. Children with CHD had lower ross motor scaled scores compared to children born very preterm (p = 0.047). More children with CHD had impaired scores (<70 SS) on language composite (17%), expressive language (16%), and gross motor (14%) indices compared to children born very preterm (6%; 7%; 3%; ps < 0.05). No group differences were found on behaviours rated by parents on the Child Behaviour Checklist (1.5–5 years) or the proportion of children with scores above the clinical cutoff. English as a first language was associated with higher cognitive (p = 0.004) and language composite scores (p < 0.001). Lower median household income and English as a second language were associated with higher total behaviour problems (ps < 0.05).

Children with CHD were more likely to display language and motor impairment compared to children born very preterm at 18 months. Outcomes were associated with language spoken in the home and household income.

In the context of primary care, cognitive screenings are brief, non-diagnostic tests that clinicians can administer in order to provide appropriate referrals to neuropsychologists. Annual cognitive screening for adults over age 65 (“older adults”) can help monitor cognitive functioning over time and ensure more patients with cognitive impairments receive neuropsychological assessment and care earlier. Unfortunately, time constraints and lack of training present major barriers to cognitive screening in primary care, and less than half of cognitive impairment cases are identified in these settings. A remote cognitive screening mobile app has the potential to save primary care clinics time, particularly for the majority of older adults who are cognitively healthy. Moreover, a screening app well-validated for remote clinical use can replace the inadequate or nonexistent screening practices currently employed by many primary care clinics. In order to achieve their potential, remote smartphone-enabled cognitive screening paradigms must be acceptable and feasible for both patients and clinical end users. With this goal in mind, we describe the collaborative, human-centered design process and proposed implementation of MyCog Mobile (MCM), a self-administered cognitive screening app based on well-validated NIH Toolbox measures.

We conducted foundational interviews with primary care clinicians (N=5) and clinic administrators (N=3) and created user journey maps of their existing and proposed cognitive screening workflows. We then conducted individual semi-structured interviews with healthy older adults (N=5) as well as participated in a community stakeholder panel of older adults and caregivers (N=11). Based on the data collected, we developed high-fidelity prototypes of the MCM app which we iteratively tested and refined with the older adult interview participants. Older adults rated the usability of the prototypes on the Simplified System Usability Scale (S-SUS) and After Scenario Questionnaire (ASQ).

Clinicians and administrators were eager to use a well-validated remote screening app if it saved them time in their workflows and were fully integrated into their EHR. Clinicians prioritized easily interpretable score reports tied to automated best practice guidelines. Findings from interviews and user journey mapping further informed the details of the proposed implementation and core functionality of MCM. Older adult participants were motivated to complete a remote cognitive screener to ensure they were cognitively healthy, save time during their in-person visit, and for privacy and comfort reasons. Older adults also identified several challenges to remote smartphone screening which informed the user experience design of the MCM app. The average rating across prototype versions was 91 (SD 5.18) on the SSUS and 6.13 (SD 8.40), indicating above average usability.

Through our iterative, humancentered design process, we were able to develop a viable remote cognitive screening app and proposed implementation for primary care settings optimized for multiple stakeholders. Next steps include validating MCM in clinical and healthy populations, collaboratively developing best practice alerts for primary care EHRs with neuropsychologists, and piloting the finalized app in a community clinic. We hope the finalized MCM app will promote broader screening practices within primary care and improve early assessment and diagnosis of cognitive impairment for older adults.