Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-27T02:08:29.606Z Has data issue: false hasContentIssue false

Look back in anger: flaws in the retrospective evaluation of risk assessment

Published online by Cambridge University Press:  02 January 2018

Matthew Large
Affiliation:
School of Psychiatry, University of New South Wales, Sydney, Australia, email: mmbl@bigpond.com
Olav Nielssen
Affiliation:
School of Psychiatry, University of New South Wales, Sydney, Australia, email: mmbl@bigpond.com
Rights & Permissions [Opens in a new window]

Abstract

Type
Columns
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2013

The team from the National Confidential Inquiry into Suicide and Homicide by People with Mental Illness recently posted a report on quality of risk assessment prior to suicide and homicide. Reference Rahman, Gupta, While, Rodway, Ibrahim and Bickley1 The report describes an un-masked retrospective survey of the risk assessments found in the case notes of 42 suicide victims and 39 homicide perpetrators. The authors suggest that in about a third of cases poor risk assessment might have contributed in some way to those deaths. However, the assumptions made in the report and the interpretation of the results raise serious concerns.

First, the study did not examine whether risk assessments that were classified as inadequate were more common in the notes of suicide or homicide cases than in the notes of other comparable patients with a non-fatal outcome. In fact, there is no evidence to show that raters who are masked to the eventual outcome can correctly identify the notes of patients who have died by suicide Reference Conlon, Garland, Prescott, Mannion, Leonard and Fahy2 or perpetrated homicide.

Second, there is no empirical evidence to suggest that risk assessment is of any use in preventing rare events such as suicide and homicide. Reference Wand3 The low base rate of these events means that for every correct prediction there are inevitably a very large number of false positive predictions, reducing the possibility of arranging any practical intervention to prevent the adverse outcome. The low proportion of true positives means that any intervention that follows from a high-risk categorisation with the aim of preventing a rare outcome must be sufficiently effective and benign to warrant treating so many false positives, and must be efficient enough so as not to result in the excessive diversion of healthcare resources from low-risk patients, including the proportion of false negative categorisations. The lack of sensitivity of risk assessments means that they miss about half of all homicides Reference Fazel, Buxrud, Ruchkin and Grann4 and as many as 90% of all suicides. Reference Madsen and Nordentoft5 Hence, if there are benign life-saving interventions that are suitable for high-risk patients, they should be offered to so-called low-risk patients as well, obviating the need for a risk assessment.

Hindsight will always allow us to identify clinical decisions that might have prevented an adverse event, like the action replays of goals in football matches. However, when viewed prospectively, clinical decisions involve a level of uncertainty and the requirement that the clinician accepts a level of risk in order to respect the patient’s wishes and ration the resources that are available. Perhaps the reason that a third of risk assessments were thought to be of poor quality was because those clinicians were aware of the futility of trying to predict a rare event, and were just getting on with doing what is possible, which is a comprehensive assessment of the individual patient’s treatment needs, followed by ethical and compassionate evidence-based treatment of every patient.

Footnotes

Declaration of interest

M.L. and O.N. have both received speakers fees from AstraZeneca.

References

1 Rahman, RS, Gupta, S, While, D, Rodway, R, Ibrahim, S, Bickley, H, et al. Quality of Risk Assessment Prior to Suicide And Homicide: A Pilot Study. National Confidential Inquiry into Suicide and Homicide by People with Mental Illness, University of Manchester, 2013.Google Scholar
2 Conlon, L, Garland, M, Prescott, P, Mannion, L, Leonard, M, Fahy, TJ. Psychiatric aftercare and suicide risk: a case-control study using blind rating. Arch Suicide Res 2007; 11: 291–5.Google Scholar
3 Wand, T. Investigating the evidence for the effectiveness of risk assessment in mental health care. Issues Ment Health Nurs 2012; 33: 27.Google Scholar
4 Fazel, S, Buxrud, P, Ruchkin, V, Grann, M. Homicide in discharged patients with schizophrenia and other psychoses: a national case-control study. Schizophr Res 2010; 123: 263–9.Google Scholar
5 Madsen, T, Nordentoft, M. Drs Madsen and Nordentoft reply. J Clin Psychiatry 2012; 73: 1034–5.Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.