A recent survey in the USA found that 95% of the general public believes that when a person with a mental disorder is predicted to be violent to others, legal intervention to avert the anticipated harm is justified (Reference Pescosolido, Monahan and LinkPescosolido et al, 1999). The pervasiveness of such belief throughout the world helps to explain the wide and growing variety of laws, including, in-patient and out-patient civil commitment, sexual predator commitment, tort liability and employment discrimination, that call upon psychiatrists and psychologists to assess the risk of violence (Reference ReedReed, 1997). A large body of research conducted in the 1970s and 1980s called into question the ability of clinicians to make accurate risk assessments of violence of the type that the public and the law demand. More recent research (Reference Lidz, Mulvey and GardnerLidz et al, 1993) has been only slightly more sanguine. One repeatedly suggested way to improve the accuracy of risk assessments of violence has been to use statistical or actuarial methods to inform clinical judgement (Reference BorumBorum, 1996). However, where clinically relevant actuarial tools have been available, their application has been sufficiently cumbersome and time-consuming that actuarial input into the risk assessment of violence has been impractical in most real-world clinical settings (Reference Gardner, Lidz and MulveyGardner et al, 1996).
ITERATIVE CLASSIFICATION TREE METHOD
We have recently developed an actuarial tool for assessing the risk of violence by people discharged from psychiatric facilities that we believe has greater potential for clinical application than existing actuarial methods (Reference Steadman, Silver and MonahanSteadman et al, 2000). We call this tool the Iterative Classification Tree (ICT). Classification trees (Reference Breiman, Freidman and OlshenBreiman et al, 1984) have long been used in a number of areas of medicine, including neurology (e.g. Reference Levy, Caronna and SingerLevy et al, 1985). A classification tree approach to the risk assessment of violence is predicated upon an interactive and contingent model of violence: one that allows many different combinations of risk factors to classify a person as high or low risk. Whether a particular question is asked in any clinical assessment grounded in this approach depends on the answers given to each prior question. Based on a sequence established by the classification tree, a first question is asked of all persons being assessed. Contingent on the answer to that question, one or another second question is posed, and so on, until each person is classified into a category on the basis of the risk of violence. This contrasts with the usual approach to actuarial risk assessment in which a common set of questions is asked of everyone being assessed and every answer is weighted and summed to produce a score that can be used for the purposes of categorisation.
In addition to its tree-based character, our approach acknowledges the practical impossibility of adequately classifying all persons into a high or a low violence risk group. Therefore, rather than relying on the standard single threshold for distinguishing among cases, our approach employs two thresholds: one for identifying high-risk cases and one for identifying low-risk cases. We assume that inevitably there will be cases that fall between these two thresholds, cases for which any prediction scheme is incapable of making an adequate assessment of high or low risk. Based on current knowledge, the aggregate degree of risk presented by these intermediate cases cannot be distinguished statistically from the base rate of the sample as a whole.
Our first test of the ICT method (Reference Steadman, Silver and MonahanSteadman et al, 2000) focused on how well the method performed in making risk assessments of violence under ideal conditions (i.e. with few constraints on the time or resources necessary to gather risk factors). For example, the risk factor that most clearly differentiated high-risk from low-risk groups was the Hare Psychopathy Checklist: Screening Version (Hare PCL-SV; Reference Hart, Cox and HareHart et al, 1995). Given that the full Hare PCL-R requires several hours to administer - the screening version alone takes over one hour to administer - and that it has to be administered by experienced clinicians whom Hare (Reference Hare1998) recommended should undergo three days of specialised training, resource constraints in many non-forensic clinical settings will preclude its use.
This article applies the ICT method to the sample of patients assessed in the MacArthur Violence Risk Assessment Study (Reference Steadman, Mulvey and MonahanSteadman et al, 1998). Our goal is to increase the clinical utility of this actuarial method by restricting the risk factors tested to those commonly available in hospital records or capable of being assessed routinely in clinical practice.
Admissions were sampled from acute psychiatric in-patient facilities at three sites: Western Psychiatric Institute and Clinic (Pittsburgh, PA); Western Missouri Mental Health Center (Kansas City, MO); Worcester State Hospital and the University of Massachusetts Medical Center (Worcester, MA). Selection criteria for research subjects were: civil admissions; between the ages of 18 and 40 years; English-speaking; White or African American ethnicity (or Hispanic in Worcester only); and a chart diagnosis of schizophrenia, schizophreniform disorder, schizoaffective disorder, depression, dysthymia, mania, brief reactive psychosis, delusional disorder, alcohol or drug abuse or dependence, or a personality disorder. After complete description of the study to the subjects, written informed consent was obtained.
We approached a quota sample (to ensure representativeness across sites on gender, race, and age) of 1695 to participate. The refusal rate was 29% (n=492). The final sample given a hospital interview was 1136. Differences between the eligible admissions and the follow-up sample (n=939) are discussed in detail elsewhere (Reference Steadman, Mulvey and MonahanSteadman et al, 1998). Males comprised 57.3% of the sample. Ethnically, 68.7% of the sample was White, 29.1% African American and 2.2% Hispanic. The mean age was 29.9 (s.d.=6.2) years. Depression was the most frequent primary research diagnosis on the DSM-III-R Checklist (Reference Janca and HelzerJanca & Helzer, 1990; 41.9%), followed by alcohol/drug abuse or dependence (21.8%), schizophrenia (17.0%), bipolar disorder (14.1%), personality disorder only (2.1%) and other psychotic disorder (3.1%). The proportion of all cases with a primary research diagnosis of major mental disorder that had a co-occurring diagnosis of substance abuse or dependence was as follows: depression, 49.6%; schizophrenia, 41%; bipolar disorder, 37.7%; and other psychotic disorder, 45%.
Hospital data collection
Hospital data collection was conducted in two parts: an interview by a research interviewer to obtain data on risk factors and violence; and an interview by a research clinician (PhD or MA/MSW in psychology or social work) to confirm the chart diagnosis using the DSM-III-R Checklist and to administer several clinical instruments.
The hospital data set assembled in the MacArthur Violence Risk Assessment Study consisted of 134 risk factors from four conceptual domains: dispositional or personal factors (e.g. age); historical or developmental factors (e.g. child abuse); contextual or situational factors (e.g. social networks); and clinical or symptom factors (e.g. delusions) (Reference Steadman, Monahan, Appelbaum, Monahan and SteadmanSteadman et al, 1994). For the present analysis, we eliminated 28 risk factors that would be the most difficult to obtain in clinical practice, restricting ourselves to the remaining 106. Two criteria were used to eliminate risk factors. The first was to eliminate information generally unavailable to mental health personnel in the context of brief hospitalisation (e.g. information in official arrest records, in distinction to self-report of prior arrests). The second was to eliminate information that required the administration of a lengthy (>12-item) instrument to obtain (e.g. a social network inventory (Reference Estroff, Zimmer, Monahan and SteadmanEstroff & Zimmer, 1994)). A list of all 134 risk factors, with their bivariate correlations with violence and with an indication of which were eliminated from these analyses, is provided in Table 1.
|Domain||Reference||Pearson coefficient R|
|Hare Psychopathy Cheklist: Screening Version >121||Reference Hart, Cox and HareHart et al, 1995||0.26***|
|Novaco anger: behaviour||Reference Novaco, Monahan and SteadmanNovaco, 1994||0.16***|
|Novaco anger: cognitive||0.11*|
|Novaco anger: arousal||0.09**|
|Novaco anger: intensity||0.08*|
|Barratt impulsiveness: motor||Reference Barratt, Monahan and SteadmanBarratt, 1994||0.07*|
|Barratt impulsiveness: non-planning||0.05|
|Barratt impulsiveness: cognitive||0.05|
|Years of education||-0.11***|
|Socio-economic status1||Reference Hollingshead and RedlichHollingshead & Redlich, 1958||0.05|
|Age at first hospitalisation||-0.04|
|No. of prior hospitalisations||-0.03|
|Involuntary legal status||0.11**|
|Recent violent behaviour||0.14***|
|Adult arrest: seriousness||0.25***|
|Adult arrest: frequency||0.24***|
|Any arrest: person crime1||Official Report||0.13***|
|Any arrest: other crime1||Official Report||0.11***|
|Sexually abused before age 20||-0.03|
|Seriousness of abuse as child||0.14***|
|Frequency of abuse as child||0.12***|
|Father ever used drugs||0.16***|
|Father ever arrested||0.15***|
|Father ever excess drinking||0.11**|
|Father ever admitted to psychiatric hospital||0.02|
|Lived with father to age 15||-0.09**|
|Mother ever used drugs||0.05|
|Mother ever arrested||0.05|
|Mother ever excess drinking||0.06|
|Mother ever admitted to psychiatric hospital||-0.02|
|Lived with mother to age 15||-0.06|
|Parents ever fought with each other||0.06|
|Parents ever fought with others||0.03|
|Any head injury: loss of consciousness||0.10**|
|Any head injury: no loss of consciousness||0.06|
|Attempt to kill self||0.01|
|Living in private residence||-0.05|
|Perceived stress1||Reference Cohen, Kamarck and MermelsteinCohen et al, 1983||0.08*|
|Social networks||Reference Estroff, Zimmer, Monahan and SteadmanEstroff & Zimmer, 1994|
|No. of people in social network1||-0.02|
|% Mental health professionals in social network1||-0.10**|
|% Family in social network1||0.01|
|No. of negative persons in social network1||0.07*|
|No. of positive and material supporters1||-0.07*|
|Average no. of mentions per negative person1||0.06|
|Average no. of mentions per positive material person1||-0.03|
|Frequency of social network contact1||-0.03|
|Duration of social network contact1||0.02|
|Chart antisocial personality disorder||0.19***|
|DSM-III-R Checklist||Reference Janca and HelzerJanca & Helzer, 1990|
|Major disorder, no substance||-0.19***|
|Major disorder and substance||0.08*|
|Substance, no major disorder||0.15***|
|Drug or alcohol||0.18***|
|Personality disorder only||0.02|
|Brief Psychiatric Rating Scale||Reference OverallOverall, 1988|
|Thought disturbance sub-scale1||-0.06*|
|Global Assessment of Functioning||American Psychiatric Association, 1989||-0.05|
|Activities of daily living||-0.01|
|Delusions||Reference Appelbaum, Robbins and RothAppelbaum et al, 1999|
|Influence on others||-0.03|
|Violent fantasies||Reference Grisso, Davis and VesselinovGrisso et al, 2000|
|Focus same person||0.10**|
|While with target||0.12***|
|Frequent, not escalating, not with target||-0.01|
|Frequent, escalating, not with target||0.09**|
|Frequent, not escalating, with target||0.08*|
|Frequent, escalating, with target||0.10**|
|Not frequent, not escalating, not with target||0.13***|
|Present at time of admission||Record Review|
|Unable to care for self||0.02|
|Mini-Mental State1||Reference Folstein, Folstein and McHughFolstein et al, 1975||0.02|
|Perceived coercion at admission1||Gardner et al, 1993||0.03|
Community data collection
Twenty weeks after hospital discharge was chosen as the time frame for the analysis here because this was the period during which the prevalence of violence by patients in the community was at its highest (Reference Steadman, Mulvey and MonahanSteadman et al, 1998). Research interviewers attempted two follow-up interviews with enrolled patients in the community during this period, approximately 10 weeks apart. A collateral informant who knew of the patient's behaviour in the community during the follow-up period - usually, but not always, a family member - was also interviewed on the same schedule. Arrest and re-hospitalisation records provided the third source of information about the patients' behaviour in the community.
Patients and collaterals independently were asked whether the patient had been involved in any of several categories of violent behaviour in the past 10 weeks (Reference Lidz, Mulvey and GardnerLidz et al, 1993). Only the most serious act for each discrete incident was coded. Violence to others was defined to include the following: acts of battery that resulted in physical injury; sexual assaults; assaultive acts that involved the use of a weapon; or threats made with a weapon in hand. (Battery that did not result in injury was defined as ‘other aggressive act’ (Reference Steadman, Mulvey and MonahanSteadman et al, 1998) and is not considered in the analyses reported here.) Violence reported by any of the three data sources - subject self-report, collateral report, or official records - was reviewed by a team of trained coders. Ethical and legal issues encountered in conducting this research are discussed elsewhere (Reference Monahan, Appelbaum and MulveyMonahan et al, 1994).
Developing the classification tree
To develop the ICT model, we used CHAID (chi-squared automatic interaction detector) software (SPSS, 1993). Specifically, the CHAID algorithm was used to assess the statistical significance of the bivariate association between each of the 106 eligible risk factors and the dichotomous outcome measure - violence in the community - until the most statistically significant value of χ2 was identified, with P < 0.05 a necessary condition for risk factor selection. Once a risk factor was selected, the sample was partitioned according to the values of that risk factor. This selection procedure was then repeated for each of the sample partitions, thus further partitioning the sample. The result of the partitioning process was to identify groups of cases that shared the same risk factors and that also shared the same values on the outcome measure of violence.
Iterating the classification tree
We then extended this recursive partitioning approach in an iterative fashion. That is, all subjects not classified into groups designated as either high risk or low risk in the first iteration of CHAID were pooled together and re-analysed in a second iteration of CHAID. This iterative process continued until it was not possible to classify any additional groups of subjects as either high or low risk (with no group allowed to contain fewer than 50 cases).
Choosing two cut-offs
The choice of cut-off scores for high-risk and low-risk categories must be made in the context of legal or policy values external to the methodology chosen for assessing risk. Here, for illustrative purposes, we defined any group of patients with a rate of violence that was less than half the base prevalence rate of the total sample, as in the low-risk category, and any group of patients whose rate of violence was greater than twice the base prevalence rate of the total sample, as in the high-risk category. Because the base prevalence rate of violence during the first 20 weeks after hospital discharge for the total sample was 18.7% (i.e. 18.7% of the patients committed at least one violent act during either the first or second 10-week follow-up period), this meant that the cut-off for the low-risk category was 9% violent and the cut-off for the high-risk category was 37% violent.
The ICT contained three iterations (Fig. 1). In the first iteration, the tree classified 429 of the 939 subjects (45.7%) into either the high- or the low-risk categories. In the second iteration, the tree classified as high- or low-risk 167 (32.7%) of the 510 subjects who were not classified into either high- or low-risk groups at the end of iteration 1. In the third iteration, the tree classified as high- or low-risk 86 of the 343 subjects (25.1%) who were unclassified at the end of iteration 2. At the end of iteration 3, no further groups could be classified as high- or low-risk, given the parameters of the model we had set (e.g. no group with fewer than 50 cases); 257 subjects (27.4% of the total sample) remained unclassified. The final ICT contained 15 contingent risk factors that formed 11 risk groups (four low-risk groups, accounting for 50.9% of the total sample; three high-risk groups, accounting for 21.7% of the total sample; and four unclassified risk groups, accounting for 27.4% of the sample).
The risk factors displayed in Fig. 1 are defined as follows. Seriousness of prior arrests was a patient's self-report of the seriousness of arrests since age 15 years. Motor impulsiveness was measured from the motor sub-scale of the Barratt Impulsiveness Scale (Reference Barratt, Monahan and SteadmanBarratt, 1994). Father used drugs was a self-report question on whether the patient's father ever used drugs excessively. Recent violent fantasies was measured by the Schedule of Imagined Violence (Reference Grisso, Davis and VesselinovGrisso et al, 2000). Major disorder without substance abuse refers to a diagnosis of any major mental disorder without any co-occurring substance abuse diagnosis, as reached by research clinicians using the DSM-III-R Checklist. Legal status was the initial status for the baseline hospitalisation, as recorded in hospital admission records. Schizophrenia was the diagnosis reached by research clinicians using the DSM-III-R Checklist. Anger reaction was measured by a short version of the Behavioural Subscale of the Novaco Anger Scale (Reference Novaco, Monahan and SteadmanNovaco, 1994). Employed was a self-report question regarding the patient's paid full- or part-time employment status in the two months prior to hospital admission. Recent violence was a self-report of violence in the two months prior to hospital admission. Loss of consciousness referred to a self-report of any loss of consciousness due to head injury. Parents fought was a self-report by the patient that his or her parents engaged in physical fights with one another when the patient was growing up. (A complete list of the questions comprising these risk factors is available from the first author upon request.)
Receiver operating characteristic
To assess the predictive accuracy of the actuarial model produced by this method and to facilitate further comparisons of our results with other research on violence risk assessment, we used a receiver operating characteristic (ROC) analysis (Reference Gardner, Lidz and MulveyGardner et al, 1996; Reference Quinsey, Harris and RiceQuinsey et al, 1998). The statistic used to summarise the analysis is the area under the ROC curve, which corresponds to the probability that a randomly selected violent patient will have been assessed by the risk assessment tool as higher risk than a randomly selected non-violent patient (Reference SwetsSwets, 1988). The area under the ROC curve for the 11 risk groups presented in Fig. 1 is 0.80 (P<0.001). The distribution of cases that were violent or not violent during the follow-up as a function of the low- and high-risk cut-offs used to generate the ICT is presented in Table 2.
|Low <9%||Unclassified||High >37%||Total|
We did not cross-validate the ICT. Cross-validation of a risk assessment model requires estimating the model on a subset of the data and validating the model on the rest. As noted by Gardner et al (Reference Gardner, Lidz and Mulvey1996), however, cross-validation “wastes information that ought to be used estimating the model” (p 43). For this reason, bootstrapping (Reference Mooney and DuvalMooney & Duval, 1993) has become a widely used analytical strategy for estimating the shrinkage to be expected when a model is generalised to a sample other than the one on which it was estimated. In conducting such an analysis, 1000 bootstrapped samples were drawn from the original data set. Table 3 presents the 95% confidence intervals for each of the 11 risk groups in the ICT, in order of decreasing risk. The ranges of these intervals indicate how the ICT is likely to perform on other similar samples.
|Risk group||% violent in risk group||95% confidence interval|
We have sought to increase the utility of an actuarial method for real-world clinical decision-making by applying the newly developed ICT method (Reference Steadman, Silver and MonahanSteadman et al, 2000) to a set of violence risk factors commonly available in clinical records or capable of being assessed routinely in clinical practice. We have shown that the ICT partitioned 72.6% of a sample of discharged psychiatric patients into one of two categories with regard to their risk of violence to others during the first 20 weeks after discharge. One category consisted of groups whose rates of violence were no more than half the base rate of the total patient sample (i.e. ≤9% violent). The other category consisted of groups whose rates of violence were at least twice the base rate of the total patient sample (i.e. ≥37% violent). The actually observed rates of violence in the low- and high-risk categories were 5% and 45%, respectively. The prevalence of violence within individual risk groups within the low- and high-risk categories varied from 2.7% to 52.7% (Table 3).
The ICT left 27.4% of the total sample unclassified, meaning that it could find no combination of risk factors that allowed these patients to be classified into either a low- or a high-risk group. The violence rate for the unclassified category was 24.1%.
Illustrating the use of the ICT may be helpful. A clinician evaluating a patient's risk of violence using the ICT presented in Fig. 1. would first ask the patient about the seriousness of his or her prior arrests. If the patient stated that he or she had previously been arrested for a violent crime, the clinician would then inquire into whether the patient recently had been fantasising about being violent. If the patient responded affirmatively to this second question, he or she at that point would be placed in the high violence risk category. More specifically, the patient would be placed in risk group B, a group in which approximately 53% of the patients are expected to commit a violent act in the next several months.
If, on the other hand, the patient denied having violent fantasies, the clinician would then indicate whether the patient had a diagnosis of schizophrenia. If the patient did have such a diagnosis, he or she at that point would be placed in the low violence risk category. More specifically, the patient would be placed in risk group E, a group in which approximately 7% of the patients are expected to commit a violent act in the next several months. (For other studies finding rates of violence to be lower among patients with schizophrenia than among patients with other, primarily personality disorder, diagnoses, see: Reference Gardner, Lidz and MulveyGardner et al, 1996; Reference Quinsey, Harris and RiceQuinsey et al, 1998; Reference Wallace, Mullen and BurgessWallace et al, 1998.)
Comparative predictive accuracy
We have demonstrated here that the ICT method may be adapted for clinical use. The method does not require unavailable or costly-to-gather data in order to characterise the risk of violence. Rather, risk factors usually found in patient files, or capable of routine assessment, are all that are required for the ICT to function. The predictive accuracy of the ICT using a reduced set of 106 clinically feasible risk factors from the MacArthur Violence Risk Assessment Study (an area under the ROC curve of 0.80) is comparable to the predictive accuracy that we reported (Reference Steadman, Silver and MonahanSteadman et al, 2000) for risk assessment using the expanded set of 134 risk factors (an area under the ROC curve of 0.82).
Violence risk assessment software
Although the contingent nature of the risk factors identified in Fig. 1 may appear too intricate for use in clinical practice, the utility of the ICT model would be enhanced greatly with the aid of software. Software would facilitate the assessment of an individual patient by guiding the clinician to ask only those questions required to assess risk. We are in the process of developing such software.
Clinical Implications and Limitations
▪ The classification tree approach to violence risk assessment proposed here improves on traditional approaches by explicitly acknowledging that violence is an outcome reached by multiple routes.
▪ Employing two cut-off scores - one for identifying high-risk cases and one for identifying low-risk cases - is more realistic than attempting to classify all patients as to the risk of violence.
▪ A clinically useful actuarial approach for assessing the risk of violence among acute psychiatric patients now exists.
▪ This is a clinical study of violence among people hospitalised for mental disorders, not an epidemiological study of violence among people with mental disorders in the general population.
▪ The extent to which the accuracy of the actuarial tool developed here generalises to other types of clinical setting (e.g. forensic hospitals) is unknown.
▪ The proposed tool is specifically designed for assessing the risk of violence; efforts to manage the risk of violence will require additional data.
This research was supported by the Research Network on Mental Health and the Law of the John D. and Catherine T. MacArthur Foundation and by NIMH grant R01 49696. We are indebted to the members of the Network (Shirley S. Abrahamson, Richard J. Bonnie, Pamela S. Hyde, Stephen J. Morse, Paul Slovic and David B. Wexler) and to Steven Banks and Roumen Vesselinov.