Skip to main content Accessibility help
×
Home

Information:

  • Access
  • Cited by 31

Figures:

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Patient experience of negative effects of psychological treatment: results of a national survey
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Patient experience of negative effects of psychological treatment: results of a national survey
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Patient experience of negative effects of psychological treatment: results of a national survey
        Available formats
        ×
Export citation

Abstract

Background

To make informed choices, patients need information about negative as well as positive effects of treatments. There is little information about negative effects of psychological interventions.

Aims

To determine the prevalence of and risk factors for perceived negative effects of psychological treatment for common mental disorders.

Method

Cross-sectional survey of people receiving psychological treatment from 184 services in England and Wales. Respondents were asked whether they had experienced lasting bad effects from the treatment they received.

Results

Of 14 587 respondents, 763 (5.2%) reported experiencing lasting bad effects. People aged over 65 were less likely to report such effects and sexual and ethnic minorities were more likely to report them. People who were unsure what type of therapy they received were more likely to report negative effects (odds ratio (OR) = 1.51, 95% CI 1.22–1.87), and those that stated that they were given enough information about therapy before it started were less likely to report them (OR = 0.65, 95% CI 0.54–0.79).

Conclusions

One in 20 people responding to this survey reported lasting bad effects from psychological treatment. Clinicians should discuss the potential for both the positive and negative effects of therapy before it starts.

Footnotes

See editorials, pp. 208–209 and 210–212, this issue.

The National Audit of Psychological Therapies (NAPT) is managed by the Royal College of Psychiatrists' College Centre for Quality Improvement (CCQI). It is commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit and Patient Outcomes Programme (NCAPOP). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

Declaration of interest

G.P. was chief investigator of an NIMH-funded project that led to the development of the Supporting Safe Therapy information resource (www.supportingsafetherapy.org).

A number of psychological and pharmacological interventions have been shown to be effective for the treatment of mental disorders. Although both the effects and side-effects of pharmacological treatments have been widely investigated, 1 there is less information about negative effects of psychological treatments. 24 Research trials of psychological therapies do not monitor or report negative effects adequately compared with pharmacological trials. 5 It is estimated that between 5 and 10% of people have higher levels of psychiatric symptoms following psychological treatment than before they start it, 6 but it is not possible to attribute this to the treatment itself. Very few studies have examined patient experiences of negative effects of psychological treatments, and those that have, have been small and restricted to selected groups of patients. 7,8

The National Audit of Psychological Therapies is a large-scale examination of state-funded psychological therapy services for adults with depression and anxiety in England and Wales. 9 The audit comprises an examination of routine clinical records and a survey of people using a wide range of primary and secondary care services to evaluate their performance against agreed standards of care. We analysed data from the audit to determine the prevalence of patient-reported negative effects of psychological treatments and to identify factors that may influence the likelihood that patients experience these.

Method

Setting and participants

Data for the study were collected as part of the second round of the audit in 2012–2013. To identify eligible services we contacted medical directors and chief executives of all National Health Service (NHS) providers in England and Wales asking them to submit contact details for the psychological treatment services they provide. We supplemented this with data from the national Improving Access to Psychological Therapies (IAPT) programme in England, and contact details of services that participated in an earlier round of the audit. 10 A total of 220 services took part (approximately 60% of the 350–380 services that we estimate were eligible to take part in the audit at that time). All those aged 18 years or older who were on the case-load of participating services and receiving out-patient treatment for anxiety and/or depression on an agreed census date within the period 1 July to 31 October 2012 were invited to complete an anonymous service-user questionnaire that examined people's experience of the process and outcomes of treatment. All participants were given written information about the audit and invited to complete a paper or web-based questionnaire. Those who opted to complete a paper version were given a pre-paid envelope to return the questionnaire directly to the audit team. Demographic data were not collected from people who did not participate in the survey, but were available from the audit of clinical records that was conducted along with the survey.

Main outcome measure and covariates

As part of a parallel qualitative study examining patient experiences of negative effects of psychological therapies, we recruited patients who may have had these experiences through adverts in local newspapers and online fora. Patients who were interviewed made a distinction between short-lived unsettling or upsetting experiences that occurred during therapy, and longer-lasting negative effects. Some patients felt the term ‘negative’ did not properly capture how difficult their experience had been. Based on these accounts we asked all patients who took part in the survey to indicate whether they had experienced ‘lasting bad effects from the treatment’. People were asked whether they strongly or slightly agreed with this statement, whether they were not sure, or slightly or strongly disagreed with it.

The survey also included a series of questions on demographic factors (age, gender, sexual orientation and ethnicity), type of therapy received, and the person's experience of the process of care. The latter included questions on how long people had to wait before the start of treatment, the number of sessions they received, whether they thought they were referred at the right time, whether the time they waited was reasonable, whether they thought they had received enough information about treatment before it began and whether they were asked to give feedback on their progress in treatment. These items were based on guidelines for delivering psychological treatments and feedback from an expert group of service users and providers.

Statistical methods

The primary outcome was a self-report of having experienced ‘lasting bad effects from the treatment’. Having calculated the prevalence of those who agreed, disagreed or were neutral about whether they had experienced lasting bad effects from treatment, this item was converted into a dichotomous variable: whether people strongly or slightly agreed that they had experienced lasting bad effects or disagreed or were unsure whether this was the case. We then examined univariate associations between this variable and demographic factors (age, gender, ethnicity and sexual orientation) and factors associated with the process of care. Differences in levels of self-reported negative effects were examined between those receiving different forms of therapy. A multivariate analysis of factors associated with the likelihood of having experienced negative effects of therapy was subsequently conducted. To restrict the number of factors in this analysis, only variables showing some evidence of an association with the outcome in the univariate analysis (P<0.1) were included in the multivariate analysis. A backwards selection procedure was used to retain only the statistically significant variables.

A feature of the data was that patients were clustered within different services. Outcomes from patients from the same service may be more similar than outcomes from patients from different services. Therefore, to allow for this data structure, and the dichotomous nature of the outcome, all analysis was performed using multilevel logistic regression. Two-level models were used with patients nested within services. The analyses were implemented using the software package Stata (version 12.1).

Results

Of 220 psychological treatment services that took part in the audit, 184 (83.6%) collected data for the patient survey. Patient questionnaires were sent out to 76 950 people who were either receiving therapy or had recently completed it and 15 078 (19.6%) responded. Returns from ineligible patients who were aged under 18 or had not started therapy at the time of the survey were removed and data from 14 587 (19.0%) were included in this analysis. Characteristics of those who took part in the study are presented in Table 1 together with aggregate data from the audit of clinical records of people using the 220 treatment services during this period. Types of therapy received by patients are listed in Table 2. Among the 14 384 who provided information on the type of therapy they received, the most commonly reported treatment was cognitive–behavioural therapy (CBT, n = 7 340, 51.03%). Whereas most people reported receiving one therapy, 1208 (8.40%) reported receiving two or more, usually a low-intensity psychological treatment together with CBT, and 163 (1.13%) reported receiving three or more therapies. Most people received individual therapy, but 66 (4.59%) reported receiving group-based treatment either alone or combined with an individual therapy. A minority of patients indicated that they received another type of therapy not featured in the list they were presented with (n = 563, 3.77%) or reported that they were unsure what type of therapy they received (n = 161, 1.11%).

Table 1 Demographic characteristics of study participants and comparative data from the case-note audit

Demographic characteristics Study sample, n (%) Sample included in the case-note audit, n (%) Difference in proportions (95% CI)
Age, years 14 148 122 740
    18–24 1088 (7.69) 16405 (13.37) −5.68 (−5.18 to −6.15)
    25–34 2513 (17.76) 30117 (24.54) −6.78 (−6.09 to −7.44)
    35–44 3287 (23.23) 28796 (23.46) −0.23 (−0.51 to 0.96)
    45–54 3519 (24.87) 25359 (20.66) 2.57 (1.84 to 3.31)
    55–64 2474 (17.49) 14269 (11.63) 5.86 (5.22 to 6.53)
    65–74 980 (6.93) 5617 (4.58) 2.35 (1.92 to 2.90)
    75+ 287 (2.03) 2177 (1.77) 0.25 (0.02 to 0.51)
Gender 13 954 122 585
    Female 9656 (69.20) 79157 (64.57) 4.63 (3.81 to 5.43)
    Male 4298 (30.80) 43 428 (35.43)
Ethnicity 14 004 101 550
    White 13 134 (93.79) 90 769 (89.38) 4.41 (3.95 to 4.84)
    Asian 348 (2.49) 3736 (3.68) −1.19 (−0.9 to −1.47)
    Black 159 (1.14) 2788 (2.75) −1.61 (−1.40 to −1.80)
    Mixed 219 (1.56) 2181 (2.15) −0.58 (−0.35 to −0.80)
    Chinese/other 144 (1.03) 2078 (2.05) −1.02 (−0.81 to −1.20)

Table 2 Likelihood of experiencing negative effects of treatment, type and form of therapy, taking into account clustering by service

Reporting negative effect, n/N (%) OR (95% CI) P
Type of therapy
Cognitive–behavioural therapy
    No 441/6840 (6) 1 <0.001
    Yes 311/7340 (4) 0.64 (0.56–0.75)
Counselling
    Yes 522/10136 (5) 1 0.18
    No 230/4044 (6) 1.12 (0.95–1.32)
Psychodynamic therapy
    No 720/13834 (5) 1 0.004
    Yes 32/346 (9) 1.75 (1.19–2.58)
Cognitive analytical therapy
    No 741/14011 (5) 1 0.56
    Yes 11/169 (7) 1.20 (0.64–2.24)
Low-intensity therapy
    No 724/13679 (5) 1 0.73
    Yes 28/501 (6) 1.07 (0.73–1.59)
Humanistic therapy
    No 743/13901 (5) 1 0.11
    Yes 9/279 (3) 0.58 (0.30–1.13)
Solution-focused therapy
    No 743/13931 (5) 1 0.21
    Yes 9/249 (4) 0.65 (0.33–1.27)
Other
    No 708/13661 (5) 1 0.16
    Yes 44/519 (8) 1.64 (1.19–2.26)
Unsure
    No 591/12198 (5) 1 <0.001
    Yes 161/1982 (8) 1.71 (1.42–2.05)
Form of therapy
    Individual 670/10136 (7) 1 0.88
    Group 42/814 (5) 0.97 (0.70–1.34)
    Both 24/500 (5) 0.90 (0.59–1.38)

Regarding treatment process, most people were referred to treatment by a family doctor or other healthcare professional, but 2041 (15.66%) referred themselves to the service (Table 3). Most people were seen by a therapist within 3 months of referral to the service (n = 10 114, 73.33%), and received fewer than 10 sessions of treatment (n = 10 229, 74.35%). Patient views about the process of treatment they received are presented in Table 4; most patients reported being referred at the right time, being offered the right number of sessions and being given sufficient information about treatment before it started.

Table 3 Likelihood of reporting negative effects of treatment, treatment processes, taking into account clustering by service

Negative effect, n/N (%) OR (95% CI) P
Source of referral
    Self-referral 95/2041 (5) 1 0.30
    Someone else 572/10988 (5) 1.13 (0.90–1.41)
Time taken for treatment to start
    Less than 1 month 203/4082 (5) 1 0.002
    1–3 months 284/6032 (5) 0.94 (0.78–1.13)
    4–6 months 119/2179 (5) 1.10 (0.87–1.39)
    7–9 months 44/761 (6) 1.16 (0.82–1.63)
    10–12 months 31/327 (9) 1.95 (1.30–2.91)
    More than 12 months 34/430 (8) 1.58 (1.07–2.33)
Number of sessions
    1–5 292/5442 (5) 1 <0.001
    6–10 214/4787 (4) 0.81 (0.68–0.98)
    11–15 89/1739 (5) 0.94 (0.73–1.20)
    16–20 54/960 (6) 1.03 (0.76–1.40)
    21–25 9/295 (3) 0.54 (0.38–1.97)
    26 or more 49/535 (9) 1.65 (1.18–2.29)

Table 4 Likelihood of reporting negative effects of treatment, experiences of treatment, taking into account clustering by service

Experience of treatment Negative effect, n/N (%) OR (95% CI) P
I was referred at right time
    Unsure/disagree 225/3220 (7) 1 <0.001
    Agree 525/10 750 (5) 0.71 (0.61–0.84)
The waiting time was reasonable
    Unsure/disagree 270/4702 (6) 1 0.18
    Agree 486/9417 (5) 0.90 (0.77–1.06)
Appointments scheduled at a convenient time
    Unsure/disagree 100/1008 (10) 1 <0.001
    Agree 655/13 136 (5) 0.48 (0.38–0.60)
Able to get there without difficulty
    Unsure/disagree 83/863 (10) 1 <0.001
    Agree 664/13 175 (5) 0.51 (0.40–0.65)
Received enough information about treatment before it began
    Unsure/disagree 250/3118 (8) 1 <0.001
    Agree 506/11 018 (5) 0.56 (0.47–0.65)
Receiving the right number of sessions
    Unsure/disagree 293/4577 (6) 1 <0.001
    Agree 451/9483 (5) 0.73 (0.63–0.85)
I am asked to give feedback on how helpful I am finding treatment
    Unsure/disagree 240/4137 (6) 1 0.16
    Agree 515/9983 (5) 0.89 (0.76–1.05)

Of 14 270 people who provided information about the impact of therapy, 763 (5.23%, 95% CI 4.88–5.60) strongly or slightly agreed that it had resulted in lasting bad effects, and an additional 1099 (7.70%, 95 CI 7.27–8.15) reported that they were unsure whether therapy had resulted in lasting bad effects. Relationships between demographic factors, type of therapy, process of care and the likelihood of reporting lasting bad effects taking into account clustering by service, are presented in Tables 2, 3, 4 and 5.

Table 5 Likelihood of experiencing negative effects of treatment, demographic factors, taking into account clustering by service

Reporting negative effect, n/N (%) OR (95% CI) P
Age, years
    18–24 68/1088 (6) 1 0.001
    25–34 124/2513 (5) 0.77 (0.57–1.05)
    35–44 167/3287 (5) 0.80 (0.60–1.08)
    45–54 226/3519 (6) 1.03 (0.78–1.37)
    55–64 128/2474 (5) 0.82 (0.60–1.11)
    65–74 38/980 (4) 0.60 (0.40–0.91)
    75+ 5/287 (2) 0.26 (0.10–0.66)
Gender
    Male 256/4298 (6) 1 0.01
    Female 478/9656 (5) 0.82 (0.70–0.96)
Sexual orientation
    Heterosexual 632/12874 (5) 1 0.002
    Lesbian/gay 23/365 (6) 1.28 (0.83–1.97)
    Bisexual/other 30/320 (9) 1.98 (1.35–2.92)
Ethnicity
    White 649/13 134 (5) 1 <0.001
    Asian 41/348 (12) 2.61 (1.85–3.67)
    Black 16/159 (10) 2.16 (1.27–3.67)
    Mixed 17/219 (8) 1.65 (1.00–2.74)
    Chinese/other 19/144 (13) 2.86 (1.73–4.69)

In the multivariate analysis, factors associated with the likelihood of negative effects included some patient characteristics, types of therapy and aspects of the process of care (Table 6). Likelihood of reporting lasting bad effects of therapy was associated with receiving ‘other’ forms of treatment or being unsure what type of therapy the person had received.

Table 6 Multivariate model of factors associated with likelihood that patients reported lasting bad effects of treatment, taking into account clustering by service

Category OR (95% CI) P
Age, years
    18–24 1 0.02
    25–34 0.75 (0.54–1.04)
    35–44 0.78 (0.57–1.07)
    45–54 0.98 (0.72–1.33)
    55–64 0.82 (0.59–1.15)
    65–74 0.61 (0.39–0.96)
    75+ 0.28 (0.10–0.78)
Sexual orientation
    Heterosexual 1 0.003
    Lesbian/gay 1.31 (0.84–2.05)
    Bisexual/other 1.97 (1.31–2.99)
Ethnicity
    White 1 <0.001
    Asian 2.07 (1.34–3.18)
    Black 2.50 (1.41–4.42)
    Mixed 1.54 (0.88–2.70)
    Chinese/other 3.30 (1.93–5.63)
Appointments scheduled at a convenient time
    Unsure/disagree 1 <0.001
    Agree 0.61 (0.46–0.79)
Able to get there without difficulty
    Unsure/disagree 1 0.05
    Agree 0.74 (0.55–0.99)
Received enough information about

treatment before it began
    Unsure/disagree 1 <0.001
    Agree 0.65 (0.54–0.79)
Time taken for treatment to start
    <1 month 1 0.02
    1–3 months 0.86 (0.71–1.06)
    4–6 months 0.97 (0.75–1.25)
    7–9 months 1.03 (0.71–1.49)
    10–12 months 1.76 (1.15–2.69)
    >12 months 1.18 (0.77–1.80)
Type of therapy: other
    No 1 0.001
    Yes 1.84 (1.29–2.63)
Type of therapy: not sure
    No 1 <0.001
    Yes 1.51 (1.22–1.87)

Discussion

Data from almost 15 000 people receiving psychological treatment for anxiety and depression in England in Wales suggest that about 1 in 20 think that it had a lasting bad effect. The likelihood of patients reporting bad effects from treatment varied according to both demographic and clinical factors. People over the age of 65 were less likely to report negative effects than younger patients. People from Black and minority ethnic groups and non-heterosexuals were more likely to report such problems. Patients were less likely to report lasting bad effects of treatment if they felt they had been given sufficient information about therapy before it started. Whereas the results of the multilevel univariate analysis suggested different levels of patient experience of bad effects among different types of therapy, multivariate analysis suggested that it is only those offered ‘other’ therapies or those who were unsure what type of therapy they received that were more likely to report this type of negative experience.

Strengths and limitations

Strengths of the study are that it is based on a large sample recruited from a broad geographical spread of services that included a wide variation of treatment modalities and settings. Our primary outcome was based on a question that was developed following in-depth interviews with patients who reported negative experiences of psychological treatments. However, the study has a number of limitations that need be taken into account when considering the results. These include a low response rate to the survey and a reliance on patient recall of information about the type and duration of treatment that they received. Although we do not have demographic data from those who did not respond to the survey, comparative data from the case-note audit suggests that the response rate may have been different in different groups of patients. Nonetheless, differences in the proportion of patients in different age groups and from different ethnic backgrounds who reported experiencing negative effects are greater than differences in the proportion of people from these groups who responded to the survey. Another limitation is we do not have information about diagnoses or other clinical details. Previous studies examining outcomes of in-patient mental healthcare indicate that people with some conditions, such as somatoform or personality disorders, may be more likely to deteriorate during treatment than others. 11 Qualitative data about negative effects were not collected in the survey, but data from an ongoing analysis of in-depth interviews of people who report these experiences suggest that these include exacerbations of existing symptoms and emergence of new ones including anxiety, anger and loss of self-esteem. Finally, it is important to note that survey data were collected from people who were in treatment or had recently completed it and we do not know the extent to which these negative experiences subsequently resolved. However, even when negative experiences do not turn out to be lasting, they are unpleasant for the patient and have the potential to erode the patient's confidence in the therapist or therapy process and limit further engagement with the treatment.

Implications

A substantial minority of patients who responded to the survey reported that they experienced lasting bad effects from their treatment. To give informed consent to treatment it is important that patients are informed about possible costs and benefits. 12 Although far more people reported beneficial effects than reported harm, it is important that people being referred to psychological therapy services are informed that a minority of people experience negative effects from treatment. With well over a million people receiving psychological treatment for common mental disorders in England alone over recent years, 13 these data imply that many thousands of patients could have experienced negative effects from treatment. Our finding that people who did not know what type of therapy they received or stated that they were not given enough information about treatment before it started were more likely to report lasting bad effects is noteworthy. It is possible that people who were properly informed about treatment before it started had a better sense of what was involved and more realistic expectations of what it might achieve.

The finding that ethnic and sexual minority groups are more likely to report negative effects is of concern. Although available evidence suggests psychological therapies are at least as effective among people from minority ethnic groups as they are among White patients in Western countries, 14 we are not aware of any previous research that has examined the likelihood of negative effects. Previous reports have highlighted the role that attitudes to homosexuality can have on patient experience of psychological therapies among gay people. 15 Our findings may indicate a need to place greater emphasis on the development of therapists' cultural competence during initial training and subsequent professional development activities. 16

There is some evidence that helping therapists become aware of poor response to treatment can help avoid negative treatment outcomes. 17,18 More research is needed to establish whether this can reduce the incidence of negative experiences of treatment and it is clearly important for therapists and patients to discuss both the positive and negative effects of treatment during therapy. Future research should also include longitudinal studies that examine the course of negative effects of treatment and what can be done to help people who experience them.

Meanwhile, clinicians delivering psychological therapies should ensure that people feel that they have sufficient information about treatment before it starts and obtain informed consent to treatment by ensuring that people considering psychological treatment for their condition are aware that there is the potential for both positive and negative effects.

References

1 Nutt, D, Sharpe, M. Uncritical positive regard? Issues in the efficacy and safety of psychotherapy. J Psychopharmacol 2008; 22: 36.
2 Dimidjian, S, Hollon, SD. How would we know if psychotherapy were harmful? Am Psychol 2010; 65: 2133.
3 Lilienfeld, SO. Psychological treatments that cause harm. Perspect Psychol Sci 2007; 2: 5370.
4 Mohr, DC. Negative outcome in psychotherapy: a critical review. Clin Psychol 1995; 2: 127.
5 Duggan, CP, Parry, G, McMurran, M, Davidson, K, Dennis, J. The recording of adverse events from psychological treatments in clinical trials: evidence from a review of NIHR-funded trials. Trials 2014; 15: 335.
6 Lambert, MJ. The efficacy and effectiveness of psychotherapy. In Handbook of Psychotherapy and Behavior Change (6th edn) (ed. Lambert, MJ): 169218. Wiley, 2013.
7 Beutel, ME, Hoflich, A, Kurth, RA, Reimer, C. Who benefits from inpatient short-term psychotherapy in the long run? Patients' evaluations, outpatient aftercare and determinants of outcome. Psychol Psychother 2005; 78: 219–34.
8 Coursey, RD, Keller, AB, Farrell, EW. Individual psychotherapy and persons with serious mental illness: the clients' perspective. Schizophr Bull 1995; 21: 283301.
9 Royal College of Psychiatrists. Second Round of the National Audit of Psychological Therapies for Anxiety and Depression (NAPT). National Report 2013. Royal College of Psychiatrists, 2013.
10 Royal College of Psychiatrists. National Audit of Psychological Therapies for Anxiety and Depression. National Report 2011. Royal College of Psychiatrists, 2011.
11 Reuter, L, Bengel, J, Scheidt, CE. Non-response to therapy in acute and rehabilitative psychosomatic inpatient care – a systematic review Z Psychosom Med Psychother 2014; 60: 121–45.
12 Boisvert, CM. Negative treatment effects: is it time for a black box warning? Am Psychol 2010; 65: 680–1.
13 Department of Health. IAPT Three-Year Report: The First Million Patients. Department of Health, 2012.
14 Miranda, J, Bernal, G, Lau, A, Kohn, L, Hwang, WC, LaFromboise, T. State of the science on psychosocial interventions for ethnic minorities. Ann Rev Clin Psychol 2005; 1: 113–42.
15 King, M, Semlyen, J, Killaspy, H, Nazareth, I, Osborn, D. A Systematic Review of Research on Counselling and Psychotherapy for Lesbian, Gay, Bisexual and Transgender People. British Association for Counselling and Psychotherapy, 2007.
16 Bhui, K, Warfa, N, Edonya, P, McKenzie, K, Bhugra, D. Cultural competence in mental health care: a review of model evaluations. BMC Health Serv Res 2007; 7: 15.
17 Lambert, MJ, Whipple, JL, Smart, DW, Vermeersch, DA, Nielsen, SL, Hawkins, EJ. The effects of providing therapists with feedback on patient progress during psychotherapy: are outcomes enhanced? Psychother Res 2001; 11: 4968.
18 Lambert, MJ, Whipple, JL, Vermeersch, DA, Smart, DW, Hawkins, EJ, Nielsen, SL, et al. Enhancing psychotherapy outcomes via providing feedback on client progress: a replication. Clin PsycholPsychother 2002; 9: 91103.