Skip to main content Accessibility help
Hostname: page-component-5c569c448b-r8t2r Total loading time: 0.521 Render date: 2022-07-03T06:33:11.524Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true } hasContentIssue true

Changing nurses’ views of the therapeutic environment: randomised controlled trial

Published online by Cambridge University Press:  29 January 2019

Emese Csipke*
Doctor, Senior Research Fellow, Division of Psychiatry, University College London, UK
Til Wykes
Professor Dame, Institute of Psychiatry, Psychology and Neuroscience, King's College London, South London and Maudsley NHS Foundation Trust, UK
Stephen Nash
Statistician, London School of Hygiene and Tropical Medicine, UK
Paul Williams
Statistician, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, UK
Leo Koeser
Research Assistant, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, UK
Paul McCrone
Professor of Health Economics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK
Diana Rose
Professor of User-Led Research, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK
Tom Craig
Professor of Social Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK
Correspondence: Emese Csipke, University College London, Division of Psychiatry, 6th Floor, Maple House, 149 Tottenham Court Road, London W1T 7NF, UK. Email:
Rights & Permissions[Opens in a new window]



Although patients value evidence-based therapeutic activities, little is known about nurses' perceptions.


To investigate whether implementing an activities training programme would positively alter staff perceptions of the ward or be detrimental through the increased workload (trial registration: ISRCTN 06545047).


We conducted a stepped wedge cluster randomised trial involving 16 wards with psychology-led nurse training as the intervention. The main outcome was a staff self-report measure of perceptions of the ward (VOTE) and secondary outcomes measuring potential deterioration were the Index of Work Satisfaction (IWS) and the Maslach Burnout Inventory (MBI). Data were analysed using mixed-effects regression models, with repeated assessments from staff over time.


There were 1075 valid outcome measurements from 539 nursing staff. VOTE scores did not change over time (standardised effect size 0.04, 95% CI –0.09 to 0.18, P = 0.54), neither did IWS or MBI scores (IWS, standardised effect size 0.02, 95% CI –0.11 to 0.16, P = 0.74; MBI standardised effect size –0.09, 95% CI –0.24 to 0.06, P = 0.24). There was a mean increase of 1.5 activities per ward (95% CI –0.4 to 3.4, P = 0.12) and on average 6.3 more patients attended groups (95% CI –4.1 to 16.6, P = 0.23) following training. Staff feedback on training was positive.


Our training programme did not change nurses' perceptions of the ward, job satisfaction or burnout. During the study period many service changes occurred, most having a negative impact through increased pressure on staffing, patient mix and management so it is perhaps unsurprising that we found no benefits or reduction in staff skill.

Declaration of interest


Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright © The Royal College of Psychiatrists 2019


As bed numbers continue to fall, mental health wards are reserved for the most acutely ill with consequent increases in the proportion of in-patients who are compulsorily detained. These factors are likely to increase levels of behavioural disturbance.Reference Priebe1, Reference Bowers2 This can lead to a very fraught ward atmosphere, with nurses spending the majority of their time dealing with crises rather than engaging in therapeutic activities or interacting with patients. Long before the Francis report,Reference Bowers2, Reference Francis3 the UK Department of Health4 acknowledged that in spite of advocating therapeutic environments, this fire-fighting activity, along with an abundance of administrative work, lack of support and inadequate supervision make in-patient wards very challenging for staff. Patients have been reported to spend as little as 4% of their time interacting with nursing staff.Reference Bowers, Brennan, Winship and Theoridou5 Taken together this often brings about ward environments that appear to be more custodial than therapeutic.

Nurses and therapeutic care

Nurses report the primary reason for not spending time on therapeutic activities or direct patient contact is the need to focus on resolving crises for a small number of patients, in addition to increased administrative duties.Reference Bowers, Brennan, Winship and Theoridou5Reference Seed, Torkelson and A7 Yet patients value time spent interacting with nursing staff and taking part in therapeutics activities, regardless of how acutely ill they might beReference Csipke, Flach, McCrone, Rose, Tilley and Wykes8, Reference Evans, Rose, Flach, Csipke, Glossop and McCrone9 and such provision improves the patient experience especially for those compulsorily admitted to in-patient wards.Reference Csipke, Flach, McCrone, Rose, Tilley and Wykes8, Reference Wykes, Csipke, Williams, Koeser, Nash and Rose10 Nurses strive to provide good-quality patient careReference Laker, Rose, Flach, Csipke, McCrone and Craig11 and undoubtedly any lack of achievement has an impact on staff morale and burnout,Reference Johnson, Osborn, Araya, Wearn, Paul and Stafford12 which has been linked to concerns about recruitment, retention and training of mental health practitioners around the world (see for example Paris & HogeReference Paris and Hoge13).

Improved patient-reported quality of care and decreases in staff sicknesses have been reported when staff spent more time in individualised activitiesReference Dodds and Bowles14 and there is evidence that increases in group activities is related to significantly greater involvement of patients in their own care, greater support between staff and patients, and more opportunities to practise skills needed following discharge.Reference Hansen and Slevin15 More recentlyReference Ward and Cowman6 it has been advocated that nursing staff should provide evidence-based psychotherapy in in-patient settings and it has been report that patients appreciated activity-based and psychoeducational groups run by a range of health professionals.Reference Cowls and Hale16, 17

Advocacy groups across the world (for example the Mental Health Council of Australia, the National Alliance on Mental Illness) support the implementation of evidence-based therapeutic activities in in-patient settings. However, we are aware that asking nurses to spend more time doing these activities, without a decrease in their other duties, as well as a shrinking number of staff, may lead to negative effects because of a perceived increase in workload.

In the UK many training/accreditation programmes address the issue of the quality of in-patient care such as Starwards and Aiquip (see for example: Bowers et al Reference Bowers, Brennan, Winship and Theoridou5 and Royal College of Psychiatrists18). ClarkeReference Clarke19 has outlined that these programmes could be improved by providing adequate training and supervision for ward-based staff, a suggestion echoed by a US committee dedicated to the welfare of US-based mental health professionals.20 All this evidence suggests that activities – especially those that are evidenced based – are beneficial to patients but there has been little information on their impact on the morale and job satisfaction of nursing staff. Most studies are short term and therefore do not investigate whether there are longer-term implications for the sustainability of any changes in nursing staff time allocation.

Our project implemented a programme of evidence-based activities to improve in-patient services and rigorously evaluated their impact. The aims of the study are: (a) to investigate the changes to staff perceptions of the environmental milieu of providing staff training for therapeutic activities and (b) to explore any development of negative effects (particularly on staff morale).



The study was a cluster randomised trial of 16 wards (clusters) in a stepped wedge design (whereby the timing of the intervention to a ward is randomised) (trial registration: ISRCTN 06545047, As all clusters eventually receive the intervention, this design is often used in situations where for ethical reasons the researchers do not wish to deny the intervention to any particular cluster.Reference Mdege, Man, Taylor Nee Brown and Torgerson21 Another key reason for selecting this design method was that it allows multiple clusters to be included even when it is not possible to intervene in all clusters simultaneously. The use of this design is relatively uncommon and analysis methods are recent.Reference Hussey and Hughes22 For pragmatic reasons the randomisation was carried out in three waves, with the first eight wards being randomised, four in the second wave with the final four following. Further details are described in our companion paper relating to patient experience.Reference Wykes, Csipke, Williams, Koeser, Nash and Rose10

Nursing staff working on these wards during any assessment period were recruited and although they were aware of whether their ward had been randomised or not, assessments took place before any training was completed or activities were run. All staff assessments were self-report. We collected data at nine time points, approximately 6 months apart. Staff completed assessments each time they were available and consented to do so and so repeat measurements per person were permitted.

The setting

This study was carried out in two mental health trusts that cover five distinct geographic areas representing varied deprivation scores from the high deprivation inner city through to suburban affluent areas (for more detail see Csipke et al Reference Csipke, Flach, McCrone, Rose, Tilley and Wykes8).

  1. (a) Borough 1 serves an inner-city population with a high deprivation index. Five 18-bedded wards participated.

  2. (b) Borough 2 serves a suburban affluent area. Three wards participated, two wards had 22 beds and the remaining one had only 8 beds.

  3. (c) Borough 3 has a high deprivation score and four 18-bedded wards provide acute in-patient care.

  4. (d) Borough 4 had two 18-bedded mixed-gender wards that serve an area with a high deprivation score.

  5. (e) Borough 5 had two 18-bedded wards that serve an area that is more suburban and affluent.


Nurses of any grade were eligible to take part. Temporary staff were required to have completed seven shifts in the previous month in order to be eligible so that we could be sure that they were sufficiently familiar with the ward environment to complete the measures. Ethical approval for the study was granted by Bexley and Greenwich Research Ethics Committee (Ref: 07/H0809/49) and all eligible participants were approached and gave written informed consent.

Main outcome

For all measures obtained by questionnaires it was a requirement that more than 80% of the questions were completed. Questionnaires with less than 80% item completion were excluded as missing.

We used the Views of the therapeutic Environment (VOTE),Reference Laker, Rose, Flach, Csipke, McCrone and Craig11 as the main outcome. This is a 20-item measure, each on a six-point Likert scale with good reliability and validity that captures staff perceptions of the daily pressures of working on acute in-patient mental health wards. The outcomes investigated were the total scores and for secondary analyses three subscales (workload intensity, team dynamics and interaction anxiety). Low scores represent positive views. VOTE was user developed by nurse researchers with in-patient ward nursing staff specifically for this study to reflect the concerns and views of the nurses. As far as we know there is no other nurse-developed measure of nurse views.

Secondary outcomes

Our secondary outcomes were as follows.

  1. (a) The Maslach Burnout Inventory-Human Services Survey (MBI).Reference Maslach, Jackson and Leiter23 This captures work-related ‘burnout’ over 22 items, each on a six-point Likert scale. It has good psychometric properties and is widely used. The MBI consists of three subscales: emotional exhaustion, depersonalisation and personal accomplishment. High scores on the emotional exhaustion and depersonalisation subscales indicate poor functioning but higher scores on the personal accomplishment subscale indicate better functioning

  2. (b) The Index of Work Satisfaction (IWS).Reference Stamps and Piedmonte24 This consists of 44 items measured on a seven-point Likert scale with a total score reflecting job satisfaction, with higher scores representing less job satisfaction.

  3. (c) Demographic information on staff. Participants' age, gender, ethnicity, employment band and length of employment were collected.

  4. (d) Costs. In order to estimate the cost of the intervention we recorded the number of participants in the training sessions and their duration along with the staff input required to deliver the training.

Post-training feedback

One group session and four interviews were carried out at the end of the trial. Topic guides were developed and nurses were invited to discuss their experience of completing the training programme and running the subsequent groups. In particular, they were asked to reflect on what worked well and what did not. The responses and discussions were recorded and transcribed to provide an additional source of data to compliment results from the quantitative analyses, and to continue to seek nurses' views in a non-quantitative manner.

Staff training intervention

In keeping with the ethos of involving ward nurses at every stage of the study, the Trust clinical leads, nursing management and direct care staff on the wards met several times to consult on the evidence-based activities that were eventually chosen. National Institute for Health and Care Excellence (NICE) guidelines were taken into account, and the group chose to include other staff (occupational therapists and pharmacists) in the training as they also have a presence on the ward. This consultancy group decided to have four activities as compulsory as they applied to all wards.

  1. (a) A single session of cognitive–behavioural therapy-based communications and understanding/avoiding aggression training for nurses (cofacilitated by a patient educator).

  2. (b) Social cognition and interaction trainingReference Penn, Roberts, Combs and Sterne25 aimed at helping people understand social situations better in order to avoid misunderstandings, a common occurrence on wards.

  3. (c) Computerised cognitive remediation therapy (in order to involve occupational therapists), designed to address cognitive deficits such as problems with memory, organisation and concentration.Reference Reeder, Pile, Crawford, Cella, Rose and Wykes26

  4. (d) Where pharmacists were available, they were recruited to run a medication education group.

Two optional nurse-provided therapies were chosen by wards themselves based on their patient's needs (wards had unique mixes of patients) and the mix of skills available from the following: problem-solving skills, emotional coping skills group, hearing voices group, relaxation/sleep hygiene and coping with stigma group.

Training for each of the activities lasted 2–3 h. Clinical psychologists delivered the training, and cofacilitated the groups themselves alongside the nurses until the nurses were able and confident to deliver it independently. Nurses were required to run the groups independently by the end of 6 months. The groups themselves ran for 45 min once a week. See and Csipke et al Reference Csipke, Flach, McCrone, Rose, Tilley and Wykes8 for more detailed information.


The training intervention time was randomised by one of the trial statisticians using the random permutations of numbers using the ralloc procedure in Stata. Wards 1 to 8 were randomised first (two wards at a time), then wards 9 to 12, and finally wards 13 to 16. After the baseline period two wards were selected at random to receive staff training, with a further two wards selected every 6 months until all wards had received the training. The final four wards joined the study at a later period as they became available to participate in the study, therefore it was not possible to collect more extensive preintervention data. The design is shown in Table 1.

Table 1 Randomisation schedulea

a. 0, represents data collection prior to receipt of the staff training (i.e. control wards); 1, represents data collection after receipt of the staff training (i.e. intervention wards). Time is measured in periods of 6 months. Further information available from the authors on request.

Statistical analysis

The effect of staff training was estimated through effects on staff members including more general ward effects as well as individual-level data. All analysis was carried out using Stata version 11 or later.

The primary outcome (VOTE score) was analysed using a mixed-effects regression model with a random effect to account for the variance because of repeated measurements within staff. Ward and time were accounted for as fixed effects, by using an indicator variable to account for the ward and a time variable that counted collection periods as shown in Table 1. A similar method was used in the service arm of the studyReference Wykes, Csipke, Williams, Koeser, Nash and Rose10 but without the repeated measures random effect.

The analysis of the VOTE subscales (workload intensity, team dynamics and interaction anxiety) and secondary outcomes followed the same pattern as for the main analysis. The following variables derived from previous studiesReference Ward and Cowman6, Reference Evans, Rose, Flach, Csipke, Glossop and McCrone9 were considered as potential confounders: gender, age, ethnicity, employment band, first language and length of employment. Confounding factors were assessed and included in the model if they were associated with both the intervention and the outcome with a significance of P<0.10. We performed exploratory analyses by including an interaction between the subgroup and the staff training arm and tested the nested models using likelihood-ratio tests on all outcomes to explore potential differential effects between men and women, and between White and Black and minority ethnic staff.


Staff members could participate on more than one occasion. In total, 560 staff members consented to take part, with 444 taking part in the pretraining intervention and 280 post-training, of whom 64% were women. We had 1075 valid VOTE scores (677 for the pretraining intervention and 398 post training) from 539 staff members. Only 3.4% of questionnaires were excluded as a result of completing less than 80% of the items.

Demographic characteristics between the pre- and post-training phases of the trial were balanced. Table 2 describes characteristics of participants as a single group at first point of entry to the study. In the UK, Bands 2–6 are direct care staff working with patients on the wards, with Band 2 nurses being student nurses and Band 6 nurses being senior direct care nurses. Bands 7 and 8 are managerial nurses.

Table 2 Baseline characteristics of staff participants

a. For staff with repeated measures, the earliest response was used.


Demographic characteristics of baselines characteristics broken down by wards are presented in supplementary Table 1 (available at The number of completed assessments per time point per ward are included in supplementary Table 2. We did not include an interaction with time but site effects are listed in supplementary Table 3.


What effects did activities have on staff views of the wards?

In total, 677 VOTE scores were from 428 individuals on pretraining wards (mean score 68.2, s.d. = 12.0) and 398 scores were from 271 individuals on post-training wards (mean score 70.3, s.d. = 12.7), where a lower score is indicative of a better perception of the ward. Results from the regression model showed no evidence of a change in VOTE scores (standardised effect size (ES) = 0.04, 95% CI –0.09 to 0.18, P = 0.54) (Table 3).

Table 3 Adjusted results for Views of the therapeutic Environment (VOTE) and VOTE subscales (standardised scales)

s.e. standard error; ICC, intraclass coefficient.

a Adjusted for length of employment.

b Adjusted for age.

Our exploratory analysis examined effects for staff subgroups and suggested a difference between men and women but not for ethnicity. The model including an interaction between trial arm and gender estimated a negative effect for men (ES = 0.19, 95% CI 0.01 to 0.36, P = 0.02) and a non-significant benefit for women (ES = –0.05, 95% CI –0.20 to 0.11, P = 0.55).

A clinically meaningful increase in a standardised effect is difficult to ascertain in this new measure, VOTE. As a guide, when using Cohen's d (a different standardised effect measurement), the Food and Drug Administration of the USA offers the guideline that an effect is large if d > 0.8 and small if d < 0.5.

Our interview and focus group data from nurses suggested that they enjoyed running the activities and appreciated the opportunity to feel that they were providing something therapeutic rather than paperwork and crisis management. But they also reported that it was hard to be consistent, especially when there were high numbers of temporary staff on the ward. Activities were repetitive and they and patients sometimes became bored of them especially when they were in hospital for longer periods.

Secondary outcome analyses

Secondary analyses on satisfaction and burnout show no significant effects as for the main outcomes (Table 4). For the factors in the MBI we compared the factors with the norms for mental health staff in the MBI Handbook.Reference Maslach, Jackson and Leiter23 We found that emotional exhaustion was higher both before and after training (pretraining 21.5 (s.d. = 11.1) post-training, 22.3 (s.d. = 12.1 compared with the MBI handbook, 16.89 (s.d. = 8.90)). Depersonalisation was about the same but personal accomplishment was higher (i.e. better) (pretraining 35.5 (s.d. = 6.1) post-training, 35.2 (s.d. = 6.0) compared with the MBI Handbook 30.87 (s.d. = 6.37). But no factors were affected by the training.

Table 4 Adjusted results for Index of Work Satisfaction (IWS) and Maslach Burnout Inventory (MBI)

ICC, intraclass coefficient.

a Adjusted for age.

Did we increase the number of ward activities?

From data obtained from ward records, the mean number of activities per week before the staff training was 5.9 whereas after training it was 8. The regression analysis yielded a mean increase of 1.5 activities following the training (95% CI –0.4 to 3.4, P = 0.12) (Table 5). Before and after the training the average number of patients in activities was 28 and 39, respectively. A regression model estimated an average increase of 6.3 patients attending groups following the training (95% CI –4.1 to 16.6, P = 0.23).

Table 5 Ward reported activities (adjusting for ward effects)

What was the cost of the intervention?

Over the 16 wards in this study, we estimated that the total cost of staff training amounted to approximately £156 000. Taking into account the average occupancy rates and the average post-training follow-up of 55 weeks this yielded a cost of £10 per patient-week.


Our study sought to investigate changes in perceptions of the ward environment among those working on acute in-patient mental health wards by training staff in the skills needed to provide evidence-based therapeutic activities. We also sought to explore the development of any negative impact on staff morale. Our study was unique in that the main outcome measure, as well as the development of the training programme, involved the nurses working on the very wards included in the study in order to investigate what it is that matters to them rather than researcher-imposed priorities. Over the course of the study, nurses' perceptions of the wards did not improve. Nevertheless, we did make an impact on the day-to-day life of the ward in a number of ways, which needs to be balanced against the positive effects noted in patients.Reference Csipke, Flach, McCrone, Rose, Tilley and Wykes8

Impact of training on nurses

There was no strong evidence for change in nurses' perceptions of the ward milieu 6 months following the delivery of the training. Our study was run in an environment that was already in flux, which makes attempting to change systems a challenge. We did find that workload intensity and overall VOTE scores significantly worsened (in men) following training. As we carried out several analyses this result is not conclusive but is interesting. We consider that the main result of no effect, and that our training was clearly valued, could be a reflection of the changes in the local National Health Service (NHS) services during the study although we have no empirical support for this contention. Three of the five catchment areas in our study reduced the number of wards, and budgets were also reduced. In one ward, staff had to reapply for their own jobs, and some wards did not have ward managers but were run by junior staff or overseen on a part-time basis only. In light of this turbulent background it is surprising that being asked to deliver more activities had no effect on their view of the ward.

In spite of the training requiring nurses to engage and interact with patients more frequently, there were no effects on interaction anxiety, as measured by the VOTE subscale. In the post-training feedback we found that nurses reported their confidence was growing following training. For instance:

‘Because we were a bit unsure of ourselves…But I think once we got into it we didn't have a problem at all, we got more and more confident and it's not a problem anymore to, and I think it's helped us to run other groups because we're more confident in that so now we can run other groups as well without any problem.’

We did not find any differences in burnout, as measured by the MBI. Emotional exhaustion was higher in our sample both pre- and post-training than the means reported in the manual, which may reflect the turbulent nature of the wards. However, it is interesting that there was no effect on personal accomplishment (although this was near ceiling pre-training; meaning that the scale range allowed little scope to detect improvements post-training). There were also no effects on the MBI depersonalisation scale nor on the interaction anxiety measure of VOTE, so increases in activities did not have either a beneficial or a detrimental effect. Even though we invested on wards and observed positive effects on patients who were admitted under legal sanction, neither voluntarily admitted patients or nursing staff demonstrated any change of view. For female members of staff being part of the training group seemed to mitigate time effects (the changes wards underwent) but not for men. In support of the lack of changes in views of the ward, our measure of job satisfaction did not change either as a result of the training intervention.

Impact on wards

Our intervention consisted of training nurses to deliver evidence-based activities with more talking therapies for example hearing voices groups as recommended by NICE.27 We had no hypotheses relating to the increase in activities, but nevertheless thought it worth looking at. We had no control over the number of sessions that were actually run or how many patients participated; however, we were successful in increasing the average number of activities albeit this was not statistically significant. All wards had an activity schedule at baseline and it is not clear how many of these activities were substituted by the therapeutic ones that training was provided for in this study. However, the activities that the training was provided for were evidence-based talking therapies.27

The cost of the training was small, and we have reported that the increase in spending per patient was negligible. Patients also reported that they had more contacts and activities following the skills training, which patients said they valued. In our companion paper we considered whether the training might have any knock-on effects on the cost of service use as reported by patients but no statistically significant effect was found.Reference Evans, Rose, Flach, Csipke, Glossop and McCrone9 Administrative duties and crisis management often means that therapeutic activities are relegated to the bottom of the ward's to-do list.Reference Ward and Cowman6, Reference Seed, Torkelson and A7

Costs and cost-effectiveness

Training required investment and when apportioned over the wards and patients this amounted to £10 per patient-week. The cost of meaningful service contacts and activities increased (non-significantly) by £33 per patient-week following training (including the training cost). This does not reflect an actual increase in expenditure but rather a relative increase in resources from a fixed amount. For example, if the actual cost per patient-week of an in-patient stay is £2100 (7 days at £300 per day) then £33 per week extra represents a shift of 2% towards direct patient care.

Clinical implications

One of the most common complaints about in-patient services is the extreme boredom and lack of therapeutic activities occurring on the wards (see for example Care Quality Commission28 and Walsh & BoyleReference Walsh and Boyle29). There was a slight increase in the number of activities and we had positive feedback from those patients participating in the study,Reference Csipke, Flach, McCrone, Rose, Tilley and Wykes8 demonstrating that with effort and dedication wards can be changed. Although there was variation in terms of implementation success it should be possible to adopt our design to increase the skills and confidence of nursing staff, which has often been found to be perceived as poor.Reference Francis3 This alone would be a positive outcome, not well captured in our current study but revealed in feedback from staff.

Strengths and limitations

The generalisability of findings is fundamental in research. The original Trust taking part in the project is part of a longstanding and nationally recognised academic NHS partnership and has recently become part of an academic science centre. The users of these services are research active, as are the staff working in them, and senior management are very supportive of research activities. However, we also tested the effects in a trust with fewer academic ties, although we did not perform any statistical tests on trusts (or boroughs) because of multiplicity and absence of any original hypothesis. We believe our results are generalisable to comparable trusts in large cities nationwide regardless of their research activity. We also gave the nursing staff a choice about which optional groups they could run based on their own perceived needs. Future research could investigate the differences between qualified and unqualified staff, especially as it is often unqualified staff who spend the most face-to-face time with patients.

The stepped wedge design did lead to some challenges not least because of changes (for example managerial and staffing levels) to wards during the study so potentially increasing variation, a problem in all pragmatic health service studies. But this design also had benefits over a simple cross-sectional comparison. Neither the wards nor the researchers knew which wards would be randomised next. Pre-intervention measures of the variables that were to be our key outcomes were measured by self-report prior to each randomisation so neither staff nor patients in the control wards knew whether they would receive the active treatment. We followed CONSORT guidelines and the CONSORT checklist for the study can be found in supplementary Table 4.

Some practicalities of implementing the study could not be avoided. Three different psychologists were employed to carry out the training of ward staff, which may have led to training variation although the study team thought that quality improved as more experience was gained over time.


Implementing staff training in an unstable environment had many challenges. Nurses are working in a pressured system and their perceptions were mixed in spite of the positive views of our project. The number of activities taking place shifted in the right direction and were a clear benefit of the training. Our investigation of the effects on patients showed that we did benefit those individuals who have been the most critical – individuals who are compelled to receive treatment under a legal sanction.Reference Csipke, Flach, McCrone, Rose, Tilley and Wykes8 This was achieved without having a large detrimental effect on staff even if we did not achieve our goal of improving their perceptions of the working environment.


This article presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme (NIHR Applied programme grant PGfAR RP-PG-0606-1050). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Supplementary material

Supplementary material is available online at



These authors are joint first authors.


1Priebe, S. Institutionalization revisited - with and without walls. Acta Psychiat Scand 2004; 110: 81–2.CrossRefGoogle Scholar
2Bowers, L. Reasons for admission and their implications for the nature of acute inpatient psychiatric nursing. J Psychiatritr Ment Health Nurs 2005; 12: 231–6.CrossRefGoogle ScholarPubMed
3Francis, R. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. TSO, 2013.Google Scholar
4Department of Health. Mental Health Policy Implementation Guide: Adult Acute Inpatient Care Provision. Department of Health Publications, 2002.Google Scholar
5Bowers, L, Brennan, G, Winship, G, Theoridou, C. Talking with Acutely Psychotic People. City University, 2009.Google Scholar
6Ward, M, Cowman, S. Job satisfaction in psychiatric nursing. J Psychiatr Ment Health Nurs 2007; 14: 454–61.CrossRefGoogle ScholarPubMed
7Seed, MS, Torkelson, DJ, A, R. The role of the inpatient psychiatric nurse and its effect on job satisfaction. J Am Psychiatr Nurses Assoc 2009; 15: 120–5.CrossRefGoogle Scholar
8Csipke, E, Flach, C, McCrone, P, Rose, D, Tilley, J, Wykes, T, et al. Inpatient care 50 years after the process of deinstitutionalisation. Soc Psychiatry Psychiatr Epidemiol 2014; 49: 665–71.CrossRefGoogle ScholarPubMed
9Evans, J, Rose, D, Flach, C, Csipke, E, Glossop, H, McCrone, P, et al. VOICE: developing a new measure of service users’ perceptions of inpatient care, using a participatory methodology. J Ment Health 2012; 21: 5771.CrossRefGoogle ScholarPubMed
10Wykes, T, Csipke, E, Williams, P, Koeser, L, Nash, S, Rose, D, et al. Improving patient experiences of mental health inpatient care: a randomised controlled trial. Psychol Med 2018; 48: 488–97.CrossRefGoogle ScholarPubMed
11Laker, C, Rose, D, Flach, C, Csipke, E, McCrone, P, Craig, T, et al. Views of the Therapeutic Environment (VOTE): stakeholder involvement in measuring staff perceptions of acute in-patient care. Int J Nurs Stud 2012; 49: 1403–10.CrossRefGoogle ScholarPubMed
12Johnson, S, Osborn, DP, Araya, R, Wearn, E, Paul, M, Stafford, M, et al. Morale in the English mental health workforce: questionnaire survey. Br J Psychiatry 2012; 201: 239–46.CrossRefGoogle ScholarPubMed
13Paris, M Jr, Hoge, MA. Burnout in the mental health workforce: a review. J Behav Health Serv Res. 2010; 37: 519–28.CrossRefGoogle ScholarPubMed
14Dodds, P, Bowles, N. Dismantling formal observation and refocusing nursing activity in acute inpatient psychiatry: a case study. J Psychiatr Ment Health Nurs 2001; 8: 183–8.CrossRefGoogle ScholarPubMed
15Hansen, JT, Slevin, C. The implementation of therapeutic community principles in acute care psychiatric hospital settings: an empirical analysis and recommendations to clinicians. J Clin Psychol 1996; 52: 673–8.3.0.CO;2-L>CrossRefGoogle ScholarPubMed
16Cowls, J, Hale, S. It's the activity that counts: what clients value in psycho-educational groups. Can J Occup Ther 2005; 72: 176–82.CrossRefGoogle ScholarPubMed
17Royal College of Psychiatrists. Do the Right Things: How to Judge a Good Ward. Occasional Paper 79. Royal College of Psychiatrists, 2011 ( Scholar
18Royal College of Psychiatrists. Aims. Royal College of Psychiatrists, 2014 ( Scholar
19Clarke, S. Acute Inpatient Mental Health Care: Education, Training and Continuing Professional Development for All. NIMHHE/SCMH, 2004.Google Scholar
20Annapolis Coalition on the Behavioural Health Workforce. An Action Plan for Behavioral Health Workforce Development. Annapolis Coalition on the Behavioural Health Workforce, 2007 ( Scholar
21Mdege, ND, Man, MS, Taylor Nee Brown, CA, Torgerson, DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol 2011; 64: 936–48.CrossRefGoogle ScholarPubMed
22Hussey, MA, Hughes, JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007; 28: 182–91.CrossRefGoogle ScholarPubMed
23Maslach, C, Jackson, S, Leiter, M. Maslach Burnout Inventory Manual (3rd edn). Consulting Psychologists Press, 1996.Google Scholar
24Stamps, PL, Piedmonte, EB. Nurses and Work Satisfaction: An Index for Measurement. Health Administration Press, 1986.Google Scholar
25Penn, DL, Roberts, DL, Combs, D, Sterne, A. The development of the social cognition and interaction training program for schizophrenia spectrum disorders. Psychiat Serv 2007; 58: 449–51.CrossRefGoogle ScholarPubMed
26Reeder, C, Pile, V, Crawford, P, Cella, M, Rose, D, Wykes, T, et al. The feasibility and acceptability to service users of CIRCuiTS, a computerized cognitive remediation therapy programme for schizophrenia. Behav Cogn Psychoth 2016; 44: 288305.CrossRefGoogle Scholar
27National Institute for Health and Care Excellence. Schizophrenia: Core Interventions in the Treatment and Management of Schizophrenia in Primary and Secondary Care. NICE, 2010.Google Scholar
28Care Quality Commission. Monitoring the Mental Health Act in 2012/13. Care Quality Commission, 2014.Google Scholar
29Walsh, J, Boyle, J. Improving acute psychiatric hospital services according to inpatient experiences. A user-led piece of research as a means to empowerment. Issues Ment Health Nurs 2009; 30: 31–8.CrossRefGoogle ScholarPubMed
Figure 0

Table 1 Randomisation schedulea

Figure 1

Table 2 Baseline characteristics of staff participants

Figure 2

Table 3 Adjusted results for Views of the therapeutic Environment (VOTE) and VOTE subscales (standardised scales)

Figure 3

Table 4 Adjusted results for Index of Work Satisfaction (IWS) and Maslach Burnout Inventory (MBI)

Figure 4

Table 5 Ward reported activities (adjusting for ward effects)

Supplementary material: File

Csipke et al. supplementary material

Tables S1-S4

Download Csipke et al. supplementary material(File)
File 26 KB
Submit a response


No eLetters have been published for this article.
You have Access Open access

Save article to Kindle

To save this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the or variations. ‘’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Changing nurses’ views of the therapeutic environment: randomised controlled trial
Available formats

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Changing nurses’ views of the therapeutic environment: randomised controlled trial
Available formats

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Changing nurses’ views of the therapeutic environment: randomised controlled trial
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *