Study One: Collaborative Modeling

In Lin et al. 2016 [Reference Lin, Huang, Simon and Liu36], we analyzed patterns in the collected depression trajectories of a treatment population and compared several methods to predict individual trajectories for monitoring treatment outcomes. The data include longitudinal PHQ-9 scores over 4 years for assessing depression severity from the MHRN. We analyzed >3,000 patients with at least six PHQ-9 observations who received treatment longer than 6 months. We used smoothing splines to model individual depression trajectories. We then used K-means clustering and collaborative modeling (CM) to identify subgroup patterns. We found five broad trajectory patterns in the ongoing treatment population: stable high, stable low, fluctuating moderate, an increasing and a decreasing group (Figure 2.2a).

Figure 2.2 CM formulation and result [Reference Lin, Huang, Simon and Liu36]

The CM approach assumes that the heterogeneous progression dynamics are represented by a number of canonical models in the population, where each patient’s progression model is captured as variants of these canonical models [Reference Lin, Huang, Simon and Liu36–Reference Lin, Liu and Huang38]. CM considers the underlying cluster structure embedded in the population and the resemblance of the individuals to these clusters. We assign a membership vector, denoted as ${𝒄}_i={\left(c_{i1},\dots ,c_{\mathrm{iK}}\right)}^T$, to each patient $i$, while $K$ is the number of canonical pattern groups. Thus, $c_{\mathrm{ik}}$ denotes the probability of patient $i$ belonging to a group $k$. For each patient $i$, we assume that there are longitudinal measurements (i.e., PHQ-9 scores) at $n_i$ time points, denoted as ${𝒚}_i={\left(y_{i1},\dots ,y_{i{n}_i}\right)}^T\in {ℝ}^{n_i\times 1}$, and the longitudinal measurements of the risk factors, denoted as ${𝐗}_i={\left({𝐱}_{i1},\dots ,{𝐱}_{i{n}_i}\right)}^T\in {ℝ}^{n_i\times p}$. CM employs a canonical model $g^k\left(𝐗\right)$ for characterizing the trajectory of group $k$. The canonical model is flexible and can take the form of a B-spline model, a GPR, or a Markov model. Then, the progression model of patient $i$ can be characterized by a weighted combination of the canonical models of the $K$ latent subgroups, $f^i\left(𝑿\right)=\sum _{𝒌}{𝒄}_{\mathrm{ik}}{𝒈}^{𝒌}\left(𝑿\right)$. Furthermore, the similarity between two patients can be quantified by comparing their risk factors and past trajectory information. The similarity can be represented as a similarity matrix, $𝐖$, with each element, $w_{\mathrm{jl}}$, representing the similarity between a pair of patients, $j,l$, and reflecting how likely the patients’ progressions would be similar. To guide the learning of model parameters, we can minimize the least square loss function (Figure 2.2b) or maximize the log likelihood to measure the goodness-of-fit of the individual progression models.

Study Two: Artificial Neural Network

Depression is often accompanied by thoughts of self-harm, which are a strong predictor of subsequent suicide attempt and suicide death. Few empirical data are available regarding the temporal correlation between depression symptoms and suicidal ideation. In Gong et al. 2019 [Reference Gong, Simon and Liu35], we investigated the traditional concern that suicidal ideation may increase during a period of depression improvement using depression trajectory data. We analyzed a chronic depression treatment population’s EHR, which contained 610 patients’ longitudinal PHQ-9 scores within 20 two-week periods. We discovered patterns in trajectories of depressive symptoms using GPR and artificial neural networks. We also estimated correlations between symptomatology (PHQ-8) and suicide ideation (Item 9). We found five patterns in the PHQ-8 trajectories. PHQ-8 and Item 9 scores displayed strong temporal correlations. See Figure 2.3. We also found 8% to 13% of the patients have experienced an increase in suicidal ideation during improvement of their PHQ-8. We showed some evidence that subgroups of depressive patients are at increased risk of suicide ideation during PHQ-8 improvement.

Figure 2.3 Clustering analysis for PHQ-8 (PHQ-9 total score minus the 9th question) found five subgroups in the learned features. Graphs show the mean scores of each subgroup for PHQ-8 and Item 9 (9th question on suicide ideation) scores over 20 biweeks [Reference Gong, Simon and Liu35].

Study Three: Rule-Based model

In Lin et al. 2018 [Reference Lin, Huang, Simon and Liu39], we established a rule-based method to identify a set of risk predictive patterns from person-level longitudinal depression measurements by integrating three steps: data transformation, rule discovery, and rule evaluation. We further used the identified rules to create rule-based monitoring strategies to adaptively monitor patients. To evaluate the effectiveness of rule-based monitoring, we compared several monitoring strategies by estimating the number of depressive patients (PHQ-9 ≥ 10) in the next 6 months that are correctly monitored (true positives). We assumed under the status quo that all patients are monitored every 6 months, which may lead to unnecessary monitoring of low-risk patients. We also considered a PHQ-9-based strategy, which monitors the patient if his/her last-period PHQ-9 score is 10 or greater. Under rule-based monitoring, we considered both using individual rules and combining all top predictive rules.

We applied the rule-based method on the EHR data of a depression treatment population containing PHQ-9 scores. Twelve risk predictive rules were identified (Table 2.2). We found the rule-based prognostic model based on the identified rules enabled more accurate prediction of disease severity than other prognostic models, including RuleFit, logistic regression, and Support Vector Machine. Two rule-based monitoring strategies outperformed the latest PHQ-9-based monitoring strategy by providing higher sensitivity and specificity. We concluded that the rule-based method can lead to a better understanding of disease dynamics and achieve more accurate prognostics of disease progressions (Figure 2.4).

Figure 2.4 Comparison of monitoring accuracy for all strategies [Reference Lin, Huang, Simon and Liu39]

Study Four: Selective Sensing

In Lin et al. 2018b [Reference Lin, Liu and Huang38], the study’s objective was to build personalized trajectory models to proactively probe new trajectories during online adaptive assessment and schedule the next visit under a capacity constraint. To do so, we integrated prognosis and sensing methodologies. For prognosis, a CM approach was used to predict the future progression risk of individuals. For adaptive monitoring, a selective sensing (SS) approach was developed to allocate limited sensing resources to monitor the high-risk individuals. See Figure 2.5.

Figure 2.5 Overview: the CM-based prognosis and selective sensing for monitoring a heterogeneous patient population [Reference Lin, Liu and Huang38]

We briefly describe the selective sensing method here. SS is formulated as an integer programming problem: the goal is to optimally allocate the sensing (monitoring) resources to detect a subgroup of high-risk individuals at each monitoring period. A patient’s disease progression is modeled using the Markov model. Using a CM approach, we first estimated a transition matrix ${𝐏}_{\mathrm{it}}$ to predict the progression risk of each individual patient i at each monitoring/sensing period $t$. Healthcare systems are often constrained by providers’ capacity for mental health follow-up visits. For example, we learned that the demand for psychotherapy visit is much greater than supply at several health systems. We would prefer an algorithm that can optimally allocate the limited monitoring resources to detect high-risk individuals that are most likely to benefit from further interventions.

Suppose that there are $N$ individuals and we denote the measurements of these individuals at each monitoring period as ${𝒙}_t=\left(x_{1t},\dots ,x_{\mathrm{Nt}}\right)$, where each measurement is a PHQ-9 score of the patient. We were interested in detecting the high-risk patients. Due to the limited sensing resources, we can only observe $M$out of $N$ individuals at each period $\left(M<N\right)$. We introduced the binary decision variable ${\delta }_{\mathrm{it}}$ for each measurement $x_{\mathrm{it}}$ such that ${\delta }_{\mathrm{it}}=1$ if and only if $x_{\mathrm{it}}$ is observed at period $t$. Thus, the problem is how to choose ${\delta }_{\mathrm{it}}$ at each period such that the sensing constraint is satisfied and highest-risk individuals are detected. Detailed technical formulation can be found in Lin et al. 2018 [Reference Lin, Liu and Huang38].

New observations collected by the sensing strategy at the next period were further incorporated in the CM-based prognostic method to update the prognosis of all individuals, guiding the monitoring decision in the next period, i.e., the CM uses these observations to update the canonical models and membership vector ${𝒄}_i$ for all individual models. Adaptively monitoring the predicted high-risk individuals may lead to an increased number of missing values on predicted low-risk individuals. To guarantee an accurate estimation of the next-period major depression risk, we imputed the missing value before running risk prediction.

We applied the CM and SS methods on an EHR dataset of 610 patients that have at least 6 observations within 40 weeks under ongoing treatment [Reference Lin, Liu and Huang38]. We characterized patients’ depression progressions using Markov models and predicted the risk to severe depression using CM. We ran the selective sensing algorithm to adaptively monitor all patients over 15 time periods (each consists of two weeks) under a sensing capability of 100 patients in each period. Prediction performance is evaluated by the correlation between predicted risks of severe depression and ground truth risks (derived from observed depression onset of the patients). Detection performance is measured by the percentage of severe patients being detected and the average true risk among detected patients. For instance, when only 10% of the patients could be monitored each time, our results showed that the selective sensing algorithm outperformed the rank and selection method.

Study Five: Partially Observable CM and POMDP

The aforementioned studies assumed fully observable disease state and no treatment feedback. In Gong and Liu 2019 [Reference Gong and Liu44], we proposed a partially observable collaborative modeling (POCM) method. Depression is modeled using a Hidden Markov Model (HMM). In an HMM, disease progression is represented using transition probabilities between true disease states, and emission probabilities are probabilities of observing some measurements of the true disease states. Similar to the CM, each patient’s transition matrix and emission matrix in the HMM are assumed to be a linear combination of several canonical progression groups in the population. The weight of each patient belonging to each group is called the membership. We developed a POCM solution algorithm to estimate the parameters of the transition and emission matrices.

Next, we used a partially observable MDP (POMDP) to make a sequence of adaptive treatment decisions based on the estimated dynamics. The hidden states of the POMDP are depression severities, observations are depression assessment scores (i.e., PHQ-9), and decisions are treatment options. The objective is to optimally select between two types of treatments in each time period and maximize health outcome over time. For example, Treatment I is usual care under antidepressant medications, and Treatment II is an intensive depression management program with telephone-based treatment coordination. The objective is to maximize discounted total rewards measured using Net Monetary Benefit (defined as total health benefits × willingness-to-pay – total cost). Health benefits can be measured by quality-adjusted life years gained.

The process of creating the adaptive treatment framework includes three steps. See Figure 2.6. (i) In the learning step, the canonical transition and emission matrices for each progression pattern and patients’ memberships are learned from an existing data set of patients under Treatment I using POCM. The population average treatment effect for Treatment II is assumed to be known from clinical trials, and such knowledge is used to estimate the canonical groups’ parameters for Treatment II. (ii) In the experiment step, the personal dynamic for a new patient is initialized using model parameters estimated from the learning step and updated under both treatment options by running separate short trial periods; this is again accomplished by the POCM algorithm. The membership is then solved for each new patient belonging to each canonical group under either Treatment I or II. (iii) In the decision step, the optimal treatment strategy is obtained by solving a POMDP with the dynamic programming method (e.g., modified Monahan’s algorithm [Reference Monahan45]).

Figure 2.6 Overview of adaptive treatment design [Reference Gong and Liu44].

We compared the performance of the POMDP-based policies and several heuristic rule-based policies using a simulated depression treatment population. Results showed that the POCM can provide a better estimation of personal disease progression than the traditional method of solving an HMM when there are subgroup structures in the disease progression. We also demonstrated that the POCM-POMDP policies give the highest benefit for patients over the course of treatment. In addition, the POCM-based policies switch treatment less frequently than other policies. This research helps to advance the development of AI decision support tools for chronic disease care [Reference Gong and Liu44].

Study Six: Prognostic-Based CEA

Prognostic-based monitoring that stratifies the individual’s disease progression risk into different levels and adaptively allocates monitoring resource to high-risk individuals has the potential to improve patient health outcome and cost-effectiveness of the monitoring service. However, challenges include how to best apply prognostic models to inform the design of monitoring strategies and identify the cost-effective strategies.

In Lin et al. 2019 [Reference Lin, Huang, Simon and Liu48], we developed a decision support framework that integrated individual prognostics, monitoring strategy design and cost-effectiveness analysis (Figure 2.7). We applied the proposed framework to simulate the adaptive monitoring of a depression treatment population from EHR data. Several prediction algorithms with increasing complexity, including natural history matching, logistic regression, rule-based method, and Markov-based CM, were simulated to monitor the high-risk individuals for severe depression over time. We found six cost-effective monitoring strategies and demonstrated that the two routine monitoring strategies were dominated by the prognostic-based monitoring strategies (Figure 2.8). Methods from this research showed promise to implement prognostic-based monitoring of chronic conditions in clinical practice.

Figure 2.7 The framework of prognostic-based monitoring. Here, $r_{\mathrm{it}}$ denotes the risk score of individual $i$ in $t$th monitoring period, and $\theta$ represents the threshold for monitoring [Reference Lin, Huang, Simon and Liu48].

Figure 2.8 Cost-effectiveness frontier. The cost-effective strategies are represented by blue dots and the dominated strategies are denoted by red dots. SQ: status quo of routine monitoring under various frequencies [Reference Lin, Huang, Simon and Liu48].

Study Seven: Cohort-Based CEA

In Sun and Liu 2020 [Reference Sun and Liu49], we evaluated the cost-effectiveness of an adaptive remote monitoring technology for optimally switching between nine depression treatment lines. We used Markov-cohort models to simulate chronic depression patients’ disease progression under monitoring and treatment over two years. Cohorts are defined by age (base case, 45 years), and sex (base case, 69% female). In addition, we considered heterogeneous disease progression patterns and clustered patients into three groups including a high-risk, a medium-risk, and a low-risk group of major depression. We considered five strategies: an adaptive technology that schedules follow-up appointment for treatment switch based on remotely monitoring patients’ response to treatment with inaccuracy (i.e., imperfect sensitivity and specificity); a rule-based follow-up strategy that assigns the next follow-up time based on the patient’s health state observed at the current follow-up (similar to current practice); and fixed frequency follow-up at every two-month, four-month, and six-month period. In the base case, since the monitoring accuracy and usage cost are uncertain, we simulated more than 1,000 scenarios and investigated how sensitivity, specificity, and cost of the remote monitoring technology would affect its cost-effectiveness.

Results showed for an adaptive remote monitoring technology with 0.75 sensitivity and specificity, it is cost effective with an incremental cost-effectiveness ratio ranging from $52,600/QALYs to $63,800/QALYs (2019 USD) gained compared to the rule-based follow-up strategy. Sensitivity analyses indicated that the imperfect technology is 63–78% cost effective depending on the risk group. In summary, a combination of methods including clustering, Markov-cohort model, and treatment simulation can be generalized to evaluate the cost-effectiveness of adaptive monitoring technology in other disease applications.