The study of safety and tolerability outcomes associated with exposure to psychotropic medicines remains a challenging issue. While for efficacy outcomes it is widely accepted that randomised controlled trials (RCTs) represent the most reliable and appropriate reference standard, for safety and tolerability outcomes individual RCTs may not provide satisfactory information (Barbui et al., Reference Barbui, Gastaldon, Papola and Ostuzzi2017a). This is especially the case for psychotropic medicines: for uncommon or rare safety outcomes, RCTs are usually underpowered to establish associations; for unexpected safety outcomes, RCTs may not have planned to systematically collect information on that outcome; for safety outcomes not occurring immediately after the intervention is provided, RTCs may be too short in duration. Additionally, some safety outcomes may also be outcomes of the underlying condition: for example, suicidality or worsening of cognitive functioning may be adverse effects of some psychotropic medicines, but these are also outcomes of many mental health conditions. Therefore, detecting these outcomes in RCTs is expected and considered part of the natural history of the condition being studied, while a potential association with the investigated medicine is rarely suspected. Moreover, the interest for safety outcomes of psychotropic medicines is often comparative, that is, for medicines that are similar in efficacy, assessing if there are clinically relevant differences in terms of safety or tolerability outcomes would be extremely important. RCTs, however, are almost never designed to detect such differences. More generally, RCTs often exclude patients at risk of safety outcomes, such as for example patients with cardiovascular comorbidities or pregnant women.

Based on these arguments, in addition to the information provided by individual RCTs some other study designs may be used to expand knowledge on safety outcomes. In this Issue of Epidemiology and Psychiatric Sciences Hélène Verdoux critically describes the pros and cons of observational studies (Verdoux, Reference Verdoux2019). Observational studies, by enrolling large samples of unselected participants treated in real-life conditions and followed over long periods, may offer a different perspective on the study of safety outcomes, being able to detect serious unexpected adverse effects, negative outcomes occurring after long-term exposure to psychotropic medicines, and rare events, such as those occurring to pregnant women exposed to psychotropic medicines. Of course, observational studies have intrinsic limitations related to the lack of random allocation of the exposure variable and this may introduce a risk of bias, confounding by indication and severity, reverse causality and other biases that Hélène Verdoux brilliantly describes using practical examples (Verdoux, Reference Verdoux2019).

Another strategy that may be employed to investigate the safety of psychotropic medicines is to re-analyse findings from RCTs or observational studies using meta-analytical techniques. Systematic reviews (SRs) may advance knowledge as primary clinical research does. The relationship between antidepressant drug (AD) exposure and suicide symptoms is a paradigmatic example of this (Barbui et al., Reference Barbui, Addis, Amato, Traversa and Garattini2017b). In 2006 the FDA carried out a SR and meta-analysis of 372 placebo-controlled AD trials with nearly 1 00 000 patients (Stone et al., Reference Stone, Laughren, Jones, Levenson, Holland, Hughes, Hammad, Temple and Rochester2009). On the basis of this analysis the relationship between AD drug treatment and the incidence of reported suicidal behaviour in clinical trials was found to be strongly related to age: the risk associated with drug treatment relative to placebo was elevated in subjects under age 25, neutral in subjects aged 25–64, and reduced in subjects aged 65 and older (Stone et al., Reference Stone, Laughren, Jones, Levenson, Holland, Hughes, Hammad, Temple and Rochester2009). This knowledge was new, in the sense that before the FDA analysis the effect of age as a modifier of this effect was unknown.

SRs of observational studies may be similarly relevant, as the unintended consequences of exposure to psychotropic medicines, not captured by RCTs, may be described and quantified. For example, a recent SR of observational studies investigated whether exposure to first-generation and second-generation antipsychotics (APs) is associated with an increased risk of fractures (Papola et al., Reference Papola, Ostuzzi, Thabane, Guyatt and Barbui2018). Interestingly, this negative outcome has never been studied by means of experimental studies, so RCTs cannot provide useful information on this risk. The analysis was able to show that AP exposure in unselected populations was associated with a 57% increase in the risk of hip fractures and a 17% increase in the risk of any fractures, with some differences between individual medicines (Papola et al., Reference Papola, Ostuzzi, Thabane, Guyatt and Barbui2018).

This type of risk quantification is possible only when meta-analytical approaches are applied to systematically selected studies. However, the SR methodology has a number of limitations, as comprehensively described in this Issue of Epidemiology and Psychiatric Sciences by Marco Solmi and colleagues who pointed out that, beyond SRs of randomised or observational evidence, a new methodology, called ‘umbrella review’, can be used to systematically assess the risk of bias (quality) of SRs, and to grade the credibility of evidence (Solmi et al., Reference Solmi, Correll, Carvalho and Ioannidis2019). Umbrella reviews are overviews of SRs of randomised trials, observational studies, or both (Fig. 1). Solmi and colleagues critically analysed a number of parameters that should be accounted for when assessing the evidence from SRs included in umbrella reviews, such as research design, statistical features, quality of single studies and meta-analyses, and the reproducibility and transparency of the evidence (Solmi et al., Reference Solmi, Correll, Carvalho and Ioannidis2019). This approach is particularly relevant when applied to psychotropic medicines, as clinicians, patients and other interested stakeholders need to make an informed judgment on a number of aspects including the clinical relevance of safety findings, as part of any shared decision-making process.

Fig. 1. Findings from randomised or observational studies can be summarised, using meta-analytical strategies, to produce systematic reviews (SRs). Findings from SRs of both randomised and observational studies can be summarised, using newly developed analytical approaches, to produce umbrella reviews (overviews of SRs).

Both the Editorial of Hélène Verdoux and that of Marco Solmi and colleagues seem to suggest that each of these methodological approaches, including RCTs, observational studies, SRs and umbrella reviews, should not be considered as the exclusive or ideal research approach to provide the most reliable answers to questions related to the safety of psychotropic medicines. It is clear that the strengths of RCTs mirror the limitations of observational studies, and vice-versa and that the usefulness of SRs and umbrella reviews is highly dependent on the quantity and quality of primary experimental and observational research. Since no single approach can fully resolve these complex issues, research involving randomised and observational methods will continue to be necessary, as will continuing methodological advances. Epidemiology and Psychiatric Sciences will continue to be a leading voice for the advancement of science, practice and policy in this area.