Work represents an important goal for many people with severe mental illnesses, Reference Lehman1 but in many economically developed countries rates of unemployment among people with severe mental illness often exceed 90%. Reference Thornicroft, Tansella, Becker, Knapp, Leese and Schene2,Reference Marwaha, Johnson, Bebbington, Stafford, Angermeyer and Brugha3 However, several randomised controlled trials (RCTs), predominantly in North America, have found that supported employment using the individual placement and support (IPS) model can increase rates of competitive employment to 30–60%. Reference Bond, Drake and Becker4–Reference Rinaldi and Perkins6 It is less clear whether the IPS model is effective in populations with high rates of unemployment and relatively generous state welfare benefits for unemployed people with severe mental illness. Only one European study has been published to date investigating the IPS model in countries with diverse labour markets and different welfare systems Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7 and in this study the IPS model was not effective in all countries included. We therefore aimed to assess the effectiveness and cost-effectiveness of the IPS model in the UK. Our primary hypothesis was that at 1-year follow-up a significantly greater percentage of individuals who received IPS would be in competitive employment compared with those receiving usual services. Secondary hypotheses were that participants allocated to IPS would be: working significantly more hours each week; have significantly longer job tenure; earn a significantly higher net income; have significantly higher job satisfaction; make significantly greater (net) contributions to the national economy; have a significantly higher quality of life in the domains of work, finance, leisure and social life; have more of their needs met; and have significantly improved self-esteem.
Study design and sample characteristics
The SWAN (Supported Work and Needs) study is a pragmatic RCT of the IPS model. Participants were recruited from community mental health teams in two boroughs of South London (ISRCTN96677673). Borough A has an average index of deprivation score of 21.31 (a rank average score of 125 out of 354 for the UK), a 6.3% unemployment rate (compared with 5.3% nationally) and incapacity benefit (a benefit received by people unable to work because of ill health) is received by 5.6% of working-age residents (compared with 7.2% nationally). Borough B has an average index of deprivation score of 34.95 (a rank average score of 19 out of 354, i.e. a high level of deprivation), a 9.5% unemployment rate and incapacity benefit received by 7%.
Inclusion criteria were that participants should be receiving out-patient or community psychiatric care from local mental health services, have severe mental illness (duration of illness over 2 years, global asssessment of functioning (GAF) Reference Endicott, Spitzer, Fleiss and Cohen8 score of 60 or less, and a diagnosis of a psychotic or chronic affective disorder), aged 18–65, able to read and speak English to a high enough standard to give informed written consent, to have been unemployed for at least 3 months and want to obtain competitive employment. Participants were excluded if they had been referred for IPS in the previous 6 months. Mental health professionals were asked to identify potential participants and ask them if they agreed to being approached by a researcher. After participants were given a complete description of the study by the researcher, written informed consent was obtained. Ethical approval was obtained from the Joint Institute of Psychiatry/South London and Maudsley Research Ethics Committee (reference no 319/03).
Participants were randomly allocated to the intervention group or the control group. Treatment allocation was stratified by gender and age (10-year bands). Randomisation with minimisation was used, performed by the Institute of Psychiatry Mental Health and Neuroscience Clinical Trials Unit, a unit independent of the study to maintain concealment. The intervention condition was an IPS programme integrated within community mental health teams provided by a well-established not-for-profit non-governmental supported employment agency. The intervention involved linking four experienced employment specialists (two for each borough) with community mental health teams. In addition to seeing clients, the employment specialists attended team meetings and multidisciplinary care planning meetings, and met with care coordinators whenever appropriate. They focused on rapid placement with continued follow-up support and sought to find employment opportunities that were consistent with participants' preferences, skills and abilities. However, the integration of the IPS programme was not structural or managerial, reflecting supported employment as it is provided in the vast majority of settings in the UK, which is the current provision referred to by the UK government in its implementation of IPS. 9
The control condition (treatment as usual, TAU) consisted of existing psychosocial rehabilitation and day care programmes available in the local area. There were 33 such services in the catchment areas of the RCT; these were visited or telephoned and a senior manager was asked about their operational characteristics. Each service employed, on average, 7 vocational staff, had a median of 58 places available to clients (range 6–3000) and 79% of services received referrals from clinical teams. A range of courses were offered, most commonly pre-employment preparation (e.g. interview skills, curriculum vitae coaching and application form practise), computers/information technology and confidence building/motivation.
The participating community mental health teams were given presentations on the study with opportunities for questions and answers and written information on the study, and employment consultants working with the community mental health teams would regularly explain the study when they attended team meetings and when addressing individual questions and concerns. Teams were also regularly sent newsletters with updates on study progress and examples of success in obtaining employment. Written information on the study was revised when the study team were aware of specific issues that needed to be addressed (e.g. concerns about equipoise or the referral process).
Participants were assessed at baseline by the recruiting researcher and at 12 months after randomisation by a separate researcher who was masked to allocation status. Each participant was given £20.00 for their time at the baseline and 1-year follow-up interview. Participants were not reimbursed for any contacts they had with the employment consultants.
At baseline, participants were assessed using the Structured Clinical Assessment in Neuropsychiatry (SCAN) Reference Wing, Babor, Brugha and Burke10 to determine their diagnosis. Participants with a SCAN diagnosis of schizophrenia, schizoaffective disorder, delusional disorder or other psychotic disorders were categorised as having a psychotic disorder. Participants with a SCAN diagnosis of bipolar disorder, mania or depression were categorised as having a mood disorder.
At the baseline and 1-year follow-up interviews, data were collected on demographic information, service use and employment status over the previous 12 months. Measures of psychosocial functioning were:
(a) Manchester Short Assessment (MANSA) version 2, Reference Priebe, Huxley, Knight and Evans11 a brief modified version of the Lancashire Quality of Life Profile;
(b) Camberwell Assessment of Need (CAN) short version, Reference Slade, Thornicroft, Loftus, Phelan and Wykes12 a tool for assessing the unmet needs of people with severe mental illness;
(c) Rosenberg Self-Esteem Scale; Reference Rosenberg13
(d) Brief Psychiatric Rating Scale (BPRS); Reference Overall, Gorham and Guy14
(e) Global Assessment of Functioning (GAF); Reference Endicott, Spitzer, Fleiss and Cohen8
(f) Client Service Receipt Inventory (CSRI), Reference Beecham, Knapp and Thornicroft15 a measure of health and social resource use during the previous 12 months.
At 1-year follow-up the primary outcome measure was competitive employment in the previous 12 months. Open competitive employment was defined as a job paying at least the minimum wage, located in a mainstream socially integrated setting not set aside for persons with disabilities, held independently (i.e. was not agency owned) and the participant was in continuous employment for at least 30 days (with part-time employment rated pro-rata). Information on this and the other employment outcomes was collected by employment consultants whenever they saw their clients or, for clients who were not seeing employment consultants regularly (i.e. those in the control arm and those not engaged with the IPS programme), by employment consultants at 4-monthly intervals through the study (either face-to-face or by telephone at the person's convenience). Where this was not possible (e.g. if the client was on holiday or unobtainable), data were obtained from clinical staff if possible. Data were then compared with data collected by the researcher and any discrepancies were investigated by asking clients for clarification, or, if they were not available, checking with other sources of information. Job satisfaction was assessed using the Indiana Job Satisfaction Scale. Reference Resnick and Bond16
We calculated that a sample size of 75 in each group (150 in total) would be sufficient to detect a difference in employment from 10% in the TAU group compared with 30% in the experimental group, assuming a significance level P = 0.05 (double sided) at 80% power. Allowing for a 30% attrition rate, our target recruitment was 108 (216 in total) in each group. The base rate of 10% and the likely refusal and attrition rates were based on a study in Baltimore Reference Lehman, Goldberg, Dixon, McNary, Postrado and Hackman17 and as discussed above are typical of many populations of people in the UK with schizophrenia.
Due to the nature of the study, it was not possible for participants or those administering the intervention to be masked to the participants' allocation status. However, the researcher conducting the follow-up assessment was not told the participants' allocation status and asked the participant at the beginning of follow-up interviews not to disclose the allocation if possible. A test of the success of masking was carried out by comparing the researcher's guessed allocation with the actual allocation.
The IPS fidelity scale Reference Bond, Becker, Drake and Vogler18 was completed by a lead occupational therapist independent from the study who interviewed service users in the intervention arm, the IPS workers and care coordinators in community mental health teams, and assessed the structure of the services. This scale rates good IPS implementation if a score of 66–75 is obtained, fair IPS implementation for scores 56–65, and a failure to implement IPS if a score of 55 or below is obtained (maximum rating is 75). During the course of the study, employment workers were required to record prospectively any events that occurred in relation to the client. These events included direct client contact and any non-client contacts (of more than 10 min) that were to do with the client, e.g. contact with the client's community psychiatric nurse. Employment workers were required to record the amount of time spent per contact, where and when the contact took place and the nature of the contact.
Statistical and health economic analysis
All data were analysed using SPSS for Windows (version 15.0). The primary and secondary hypotheses were tested on the whole group. For comparing groups, t-tests and χ2-tests were used to compare means and proportions respectively, unless the data were highly skewed, in which case non-parametric tests were used. Logistic and linear regression models were also fitted including potential confounding variables: (grouped) age, gender, ethnic group, educational level, symptomatology and diagnosis. All data were analysed in groups as randomised, whether or not receiving an intervention (i.e. intention-to-treat). Data were compared for those followed up with those not followed up, overall and by each treatment arm.
The amount of time spent by employment workers with clients was recorded and this was multiplied by their hourly cost. This was based on annual costs per employment worker of £40 000, a working week of 35 h and 42 weeks per year, and a ratio of direct to indirect client time of 0.67 (this ratio was not directly available from the employment services and therefore was based on the ratio of occupational therapist direct to indirect client time Reference Curtis19 and used to calculate the cost of the IPS intervention). The resulting cost was £45 per hour. The costs of other services were calculated by combining service data with appropriate unit cost information. Reference Curtis19 Total costs at follow-up were compared between the two groups using a regression model with the group indicator as the single independent variable. Confidence intervals around the cost difference were estimated using bootstrapping with 10 000 resamples due to the likely skewed cost distribution.
Participants were recruited between November 2004 and September 2006. Of the 375 people referred to the study, 220 entered into the study (Fig. 1). One participant withdrew between the baseline assessments and randomisation leaving 219 participants. There were no substantial differences between the two randomised arms on any baseline variables (Table 1). Twenty-two participants were lost to follow-up at 1 year. There were no significant baseline differences at P<0.05 in sociodemographic or clinical variables between those who were and those who were not lost to follow-up (Table 2), although there was some weak evidence that the BPRS and GAF were different (at P<0.1), with those followed up being slightly less symptomatic and higher functioning.
|Males, n (%)||72 (66)||75 (69)|
|Borough A, n (%)||47 (43)||38 (35)|
|Borough B, n (%)||63 (57)||71 (65)|
|Worked in the past 5 years, n (%)||61 (56)||56 (51)|
|Living alone, n (%)||58 (53)||60 (55)|
|Ethnicity, n (%)|
|White||45 (41)||37 (34)|
|Black||45 (41)||49 (45)|
|Other||19 (17)||22 (20)|
|Diagnosis, n (%)|
|Psychotic disorder||72 (69)||76 (76)|
|Mood disorder||32 (31)||24 (24)|
|Age, years: mean (s.d.)||38.3 (9.3)||38.4 (9.5)|
|Camberwell Assessment of Need unmet needs, median (range)||2 (11)||0-3||1 (8)||0-3|
|Rosenberg Self-Esteem Questionnaire, mean (s.d.)||2.29 (0.55)||2.24 (0.56)|
|Overall Manchester Short Assessment score, mean (s.d.)||3.86 (0.71)||3.98 (0.75)|
|Brief Psychiatric Rating Scale, median (range)||33 (41)||28-41||33 (32)||28-42|
|Global Assessment of Functioning score, mean (s.d.)||47.66 (6.26)||48.31 (7.00)|
|Follow-up||Lost to follow-up||Statistical test||P|
|Group, n (%)||χ2(1) = 0.001||0.98|
|Intervention||98 (50)||11 (50)|
|Control||99 (50)||11 (50)|
|Males, n (%)||134 (68)||13 (59)||χ2(1) = 0.715||0.40|
|Borough, n (%)||χ2(1) = 0.045||0.83|
|A||76 (39)||9 (41)|
|B||121 (61)||19 (59)|
|Worked in the past 5 years, n (%)||104 (53)||13 (62)||χ2(1) = 0.561||0.45|
|Living alone, n (%)||105 (53)||13 (59)||χ2(1) = 0.267||0.61|
|Ethnicity, n (%)||0.23b|
|White||75 (38)||7 (33)|
|Black||87 (44)||7 (33)|
|Other||34 (17)||7 (33)|
|Diagnosis, n (%)||χ2(1) = 0.271||0.78|
|Psychotic disorder||134 (72)||14 (78)|
|Mood disorder||52 (28)||4 (22)|
|Age, years: mean (s.d.)||38.60 (9.32)||35.91 (9.96)||t(217) = 1.274||0.20|
|Camberwell Assessment of Need unmet needs, median (range)||1.52 (0-11)||0-3||1 (0-8)||0-3||Z = -0.484c||0.63|
|Rosenberg Self-Esteem Questionnaire, mean (s.d.)||2.26 (0.56)||2.31 (0.52)||t(214) = 0.425||0.67|
|Overall Manchester Short Assessment score, mean (s.d.)||3.93 (0.72)||3.79 (0.80)||t(217) = 0.855||0.39|
|Brief Psychiatric Rating Scale, median (range)||38 (24-65)||40-43||33 (24-63)||28-42||Z = -1.765c||0.09|
|Global Assessment of Functioning score, mean (s.d.)||48.24 (6.49)||45.73 (7.58)||t(216) = -1.693||0.09|
Individual placement and support fidelity was found to be high; the IPS programme in the two boroughs received a good IPS rating of 67 (Borough A) and 69 (Borough B); the IPS programme scored less well on the organisation dimension (specifically, integration of rehabilitation with mental health treatment) and, for Borough A, the services dimension (specifically, for the rapid search for a competitive job) compared with other dimensions. Of the 109 patients in the intervention arm, 73 (67%) engaged with staff (i.e. had at least one direct contact with an employment consultant); of these individuals, the mean number of contacts with or on behalf of clients was 14 (s.d. = 10).
The researchers who conducted the 1-year follow-up interviews were masked to allocation status, but guessed 119 correctly out of the 197 (60%) clients assessed compared with a hypothesised 50% (with random guesses); this is significant at P = 0.005 (single sample, double-sided test of a proportion).
One protocol deviation was noted in that one participant randomised to TAU received the intervention. Out of 219 individuals recruited, 20 (9%) reported having worked between baseline and the 1-year follow-up, in jobs that met the competitive employment criteria. Of those followed up, 7 out of 99 (7%) were from the control group and 13 out of 98 (13%) were from the intervention group (risk ratio 1.35, 95% CI 0.95–1.93, χ2(1) = 2.07, P = 0.15). Controlling for variables associated with loss to follow-up (BPRS and GAF) made no difference to the conclusions (P = 0.127). Sensitivity analyses were conducted assuming the 22 participants who were lost to follow-up had either all worked or all not worked, but this made little difference (P = 0.288 and P = 0.153 respectively). If the primary outcome had been alternatively defined (as in some other studies Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7 – working at least 1 day) there would still have been no significant difference in obtaining competitive employment: 22/197 (11%) of participants worked in a job for at least 1 day, 8/99 (8%) from the control group and 14/98 (14%) from the intervention group (χ2(1) = 1.911, P = 0.167). When sensitivity analyses were conducted assuming the 22 participants who were lost to follow-up had either all worked or all not worked, this made little difference (P = 0.296 and P = 0.170 respectively).
The median number of hours worked per week was not significantly different in the control group (32.5 h, interquartile range (IQR) 4–45, range 3–63 for 7 jobs obtained) and the intervention group (8 h, IQR 4–16, range 3–50 for 15 jobs obtained) (Mann–Whitney Z = –1.45, P = 0.15). Similarly, the length of job tenure (median 13 weeks, IQR 8–15, range 4–26 for control group; 18 weeks, IQR 7–30, range 4–52 for intervention group; Mann–Whitney Z = –1.20, P = 0.23) and the median salary (£7.73, IQR 6.15–14.18, range 6–16 for control group; £6.00, IQR 5.35–7.20, range 4.85–30 for intervention group; Mann–Whitney Z = 1.48, P = 0.14) were not significantly different in the two groups. Out of the 20 participants who had worked in competitive employment in the 1-year follow-up, 2 had worked in two different competitive employment roles. The hypothesis that the intervention would increase net contributions to the national economy is closely linked to the hypotheses regarding net earnings and hours worked. Higher earnings would potentially lead to increased tax revenues. However, because there was no significant difference in earnings there is also no significant net gain to the economy.
Within the intervention arm, there was a significant difference in employment outcome by borough: within Borough A, 1 participant (3%) obtained competitive employment, whereas within Borough B, 12 participants (17%) obtained competitive employment (Fisher's exact test P = 0.03). At 1-year follow-up there were no adverse events for any clients.
Forty-five per cent (10 of 22) of the jobs gained at 1-year follow-up were elementary occupations (e.g. bar work, kitchen porter); 23% (5 of 22) of jobs were sales and customer service occupations; 14% (3 of 22) were administrative and secretarial occupations, 9% (2 of 22) were professional occupations. Other jobs obtained were skilled trade's occupation (n = 1), and process, plant and machine operative (n = 1).
Table 3 shows there were no significant differences in secondary outcomes when comparing outcomes by randomisation group. We also found no significant differences in outcomes for participants who had obtained work and those who had not.
|Camberwell Assessment of Need unmet needs (n = 195), median (range)||1 (0.9)||0-3||1 (0-9)||0-3.5||Z = -0.409||0.68|
|Rosenberg Self Esteem questionnaire (n = 196), mean (s.d.)||2.26 (0.53)||2.25 (0.55)||t(194) = -0.124||0.90|
|Overall Manchester Short Assessment (MANSA) (n = 195), mean (s.d.)||4.03 (0.82)||3.95 (0.87)||t(194) = -0.705||0.48|
|Work||3.05 (1.50)||2.85 (1.39)||t(193) = -0.988||0.32|
|Finance||3.29 (1.64)||3.39 (1.58)||t(194) = 0.443||0.66|
|Leisure||3.44 (1.56)||3.47 (1.56)||t(194) = 0.137||0.89|
|Social life||4.39 (1.39)||4.30 (1.40)||t(189) = -0.424||0.67|
|Indiana Job Satisfaction Scale Reference Resnick and Bond16 (n = 16),a mean (s.d.)||2.08 (0.53)||1.78 (0.45)||t(14) = -1.10||0.29|
|Brief Psychiatric Rating Scale (n = 195), median (range)||36 (24-80)||29-44||37 (24-58)||30-44||Z = -0.484||0.63|
|Global Assessment of Functioning (n = 196), mean (s.d.)||53.34 (14.75)||52.46 (13.16)||t(194) = -0.440||0.66|
Regression analyses were applied to control for ethnicity, age, gender, diagnosis, symptom level and level of functioning in order to test if there were any factors that were masking a difference between the two treatment arms in any of the outcomes. However, none of the possible masking or confounding factors were found to be significant, and there were no significant interactions between these variables and treatment group.
Service use and costs
During the 1-year follow-up, two-thirds of the intervention group received input from employment workers at mean cost of £296 (Table 4). There were not substantial differences in the number of people using other services. However, control group participants who were admitted spent substantially more days in hospital than in-patients in the intervention group. This resulted in a difference in in-patient costs of £1522. Total costs were £2176 significantly higher in the control group (bootstrapped 95% CI £445–4168).
|Using service||Contactsa||Costb||Using service||Contacts Reference Priebe, Huxley, Knight and Evans11||Cost Reference Thornicroft, Tansella, Becker, Knapp, Leese and Schene2|
|Received individual placement and support||1||1||12.0||-||5||47||73||67||9.3||7.8||296||416|
|District nurse||1||1||3.0||-||< 1||1||5||5||4.2||1.8||2||11|
|Community mental health nurse||59||60||4.6||2.9||65||99||55||56||4.1||2.7||49||77|
In this largest RCT of supported employment in the UK to date, there was no evidence of significant benefit of the IPS model in helping people with severe mental illness obtain competitive employment. The study context – a socioeconomically deprived area with a somewhat rigid welfare benefits system, and an intervention provided by an external agency – may have made successful implementation of the IPS model more challenging. Although in this sample the IPS model achieved higher rates of competitive employment compared with standard services, this was not significant, and it is striking that in both arms the rates of competitive employment were very low (considerably lower than the rates reported for control groups in other studies). This suggests that even with a larger sample size that may have allowed us to detect a significant difference between the intervention arm and TAU, a very large number of people would need to receive the intervention in order to achieve relatively low rates of competitive employment.
Although we did not find evidence that IPS was effective in this study, we did not find the intervention arm to be significantly more expensive than the control arm; in fact, the control group spent more days in hospital and therefore was significantly more expensive (although this is as a result of a small number of outliers and has therefore skewed the results). This may be a chance finding or may reflect the supportive nature of the intervention, which may prevent in-patient admissions.
Successful effective implementation of IPS may be context-dependent, and it may be difficult to achieve even moderate rates of competitive employment in some settings. Our results contrast markedly with previous results reported in RCTs of the IPS model and there are a number of possible explanations for our findings. First, the IPS model may be less helpful in non-USA settings – a recent review of RCTs of supported employment found that two of the three studies with the lowest employment rates were non-US studies, Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7,Reference Latimer, Lecomte, Becker, Drake, Duclos and Piat20,Reference Drake and Bond21 although some non-US studies have found IPS to be effective (e.g. Latimer et al). Reference Latimer, Lecomte, Becker, Drake, Duclos and Piat20 These findings may be because of the different nature of the labour market; the lack of employer incentives in employing individuals with severe mental illness outside the USA; payment for healthcare in the USA, which may increase the pressure to find work in US settings; and the differences in welfare benefit systems found in different countries – Burns et al's EQOLISE study Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7 found that the degree of economic disincentives for working across several countries predicted the success of IPS programmes. They also reported considerable recruitment difficulties, despite carrying out the study in a relatively more affluent area of London (Richmond) that is likely to have more entry-level jobs; this suggests that some of these disincentives existed at the London site in their study. However, whereas local unemployment rates do predict vocational outcomes for people with severe mental illness, Reference Cook, Mulkernb, Greya, Millera, Blylerc and Razzanoa22 there is evidence from the USA that even in areas with high unemployment, people who receive IPS have superior outcomes to those in the control condition, Reference Cook, Mulkernb, Greya, Millera, Blylerc and Razzanoa22 so local unemployment rates are unlikely to be the main reason for our results.
Second, the same review Reference Drake and Bond21 found lower employment rates in a trial that did not require clients to have a goal of competitive employment. Reference Resnick and Bond16 Whereas our eligibility criteria included individuals only if they wanted to obtain competitive employment we did not test participants' motivation or restrict access to the trial to people with high levels of motivation. However, the lack of engagement with employment consultants that occurred in a quarter of the participants suggests that participants may have been less motivated compared with those in other trials. Recent studies have suggested that the most robust predictors of work outcomes include recent employment history, motivation and self-efficacy. Reference Rinaldi, Perkins, Glynn, Montibeller, Clenaghan and Rutherford23 Ethnographic research suggests that people who say they would like to work can be categorised as ‘active job-seekers and passive job-seekers’, and the latter make little progress in obtaining a job. Reference Alverson, Carpenter and Drake24 We may therefore have demonstrated greater effectiveness of the intervention if we had selected only participants who were ‘active job-seekers’; 25% of participants obtained employment in the UK Sesami study, where participants were self-selected and had already engaged with supported employment programmes. Reference Schneider, Slade, Secker, Rinaldi, Boyce and Johnson25 Indeed, some of the American trials have held two participant consent meetings, to screen out those who are less highly motivated to participate, whereas we held one consent meeting only. These findings suggest that IPS programmes should routinely include tests of motivation before entry into the programme.
Third, the IPS programme was not structurally or managerially integrated with community mental health services and these may be important components that determine the effectiveness of IPS. Previous research has found that the degree to which integration of IPS with mental health services occurs predicts success. Reference Drake and Bond21 It may therefore be relevant that the borough with the higher fidelity rating (i.e. where IPS was more successfully implemented, although still without structural or managerial integration) had better employment rates than the other borough. We have not been able to find many studies that report data on the number of contacts between IPS staff and clients to compare with our other process data, but the two studies (of 11 RCTs included in a recent review) Reference Bond, Drake and Becker4 with such data report higher numbers of contacts, which also suggests IPS implementation may not have been optimal. Other aspects of IPS implementation, which are not captured by the IPS fidelity scale, such as the nature of the relationship between the IPS worker and the client (which was an independent predictor of outcome in the EQOLISE study) Reference Catty, Lissouba, White, Becker, Drake and Fioritti26 may also explain the poorer outcomes found in this study. A recently developed modified IPS fidelity scale (IPS+) also includes rating essential processes such as consumer choice, Reference Paulson, Post and Herinckx27 and the IPS fidelity rating carried out in this study may have been lower if the IPS+ had been used.
Fourth, the relatively high proportion of participants from ethnic groups other than White (62%) could also limit the success of IPS in the UK, as people from other ethnic groups are more likely to be unemployed in the UK. 28 We also used an inclusion criteria of a maximum level of functioning of 60 (as measured by the GAF scale) in order to examine the effectiveness of IPS in people with severe mental illness. However, this means that our study population may have been more impaired than those recruited in other studies. In addition, qualitative research reported elsewhere Reference Howard, de Salis, Tomlin, Thornicroft and Donovan29 found that the clinicians involved in this RCT had idiosyncratic interpretations of the eligibility criteria used for the trial (e.g. some staff would not refer individuals with positive symptoms to the study team even though this was not an exclusion criterion). This may have led to motivated potential participants not being referred and entering the trial, as staff wished to avoid the ‘stress’ of job-seeking in case this led to relapse. Finally, attitudes towards employing people with severe mental illness may differ in different contexts; there is evidence that experienced and anticipated discrimination do vary across countries, Reference Thornicroft, Brohan, Leese and Sartorius30 but there are no current data on international differences in discrimination for employment.
Strengths and limitations
Strengths of this study include the largest sample size for an RCT of IPS in the UK, the comparatively diverse group of participants and high levels of retention at follow-up. The limitations include the pragmatic nature of this trial in a ‘real world’ setting in a socially deprived inner-city catchment area where most people with severe mental illness do not obtain employment, and as is typical in the UK, the IPS programme was provided by an external supported employment agency. Such well-established but financially insecure non-governmental agencies may have less IPS-focused staff than new staff employed to set up IPS programmes (as in previous RCTs) where the enthusiasm and commitment of staff is likely to be particularly high. Despite the high IPS fidelity rating, the separation of the supported employment agency from mental health services may also have led to differences in the intervention received by participants in this study compared with that delivered in more fully integrated models of care.
Other studies have not usually reported on rates of engagement with IPS so it is difficult to compare data but the failure to engage all participants may reflect an individual's lack of motivation, severity of illness or staff attitudes. Mental health professionals involved had low expectations of their clients' ability to obtain competitive employment, as reported elsewhere. Reference Howard, de Salis, Tomlin, Thornicroft and Donovan29,Reference Boardman, Grove, Perkins and Shepherd31,Reference Thornicroft32 The UK site of the only European RCT of IPS also found recruitment difficult and slow, Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7 suggesting these factors are not unique to our study. There is evidence from other UK studies that mental health professionals do not focus on vocational needs Reference Bertram and Howard33 and may actually advise people not to work. Reference Rinaldi and Hill34
There was evidence for researchers guessing allocation arm but as the primary outcome was derived from administrative methods in addition to participant and healthcare professional recall this is unlikely to affect the main findings. The methods to calculate patient costs relied on recall of service utilisation. There may be some concerns about recall accuracy but this is an accepted method (especially in the absence of databases designed to measure service use for insurance systems) and previous studies have shown the method to be acceptable. Reference Calsyn, Allen, Morse, Smith and Templehoff35,Reference Goldberg, Seybolt and Lehman36
This study therefore suggests that implementing IPS in the context studied here can be challenging. Even in the studies carried out in North America, IPS is successful in obtaining employment for only around 50% of people with serious mental illness in the total population of people with disablility. Thus it is not clear that IPS alone can overcome an individual's reluctance to join the workforce, deficits in social skills and neurocognitive impairments that militate against enduring employment, Reference Liberman37 and research into additional interventions such as motivational interviewing, cognitive interventions and benefits counselling Reference Drake and Bond21 may help identify further ways of helping people with severe mental illness into work. We conclude from our study that IPS will not be the only intervention necessary to improve social inclusion for people with severe mental illness. Alternative or additional interventions such as motivational interviewing and/or cognitive remediation may be needed to help obtain work; for those who do not obtain work, other measures will be necessary to promote social inclusion.
This study was supported by the Wellcome Trust (GR071272MA); the supported employment programme was funded partly by the King's Fund and the South London and Maudsley Charitable Trust.
We thank the members of our trial steering committee (Professor B. Everitt, Professor S. Johnson, Dr B. Grove and Dr D. Rose) for their advice through the course of the study, and Hilary Williams for carrying out the IPS fidelity ratings. We would also like to thank all the staff and participants who were involved in the trial.