The community mental health team (CMHT) remains the core service for delivering community care for the severely mentally ill people in England and Wales. 1 Despite noble efforts, measuring health outcomes in routine mental health services has proved difficult. Reference Gilbody, House and Sheldon2 In an era of commissioning, and to meet the requirements of the recent Darzi report, 3 measuring the quality of services is becoming a necessity. If the quality agenda can dovetail with the outcome agenda, renewed energy in seeking ways of delivering these important measures can be mustered. There is a danger that measures that seem meaningless to patients and clinicians will be imposed on teams ‘from the top’. We believe it is important for patients and clinicians to decide at the coalface which quality measures matter and then to get on with measuring them. Quality measures need to be owned by and embedded in the team. They should be sustainable, of a minimum burden and clinically useful.
This paper describes standards we have measured in an east Cornwall CMHT. They are never going to be definitive or static but a start of an ongoing dialogue. Using these measures, we aimed to write a ‘quality report’ for the local user forum, the trust and the commissioners.
Standards
We have tried to be systematic and pragmatic in deciding which standards to measure. As a systematic process, we asked the question what are CMHTs meant to do? A suggested list is given in the Department of Health policy implementation guide for CMHTs, which is a useful start. 1 New factors emphasised since these guidelines were published include staff performance and delivering evidence-based practice. Therefore, we sought to include measures of access and assessment, process and review, staff performance, delivery of evidence-based practice, liaison with other services and outcome of interventions. At a pragmatic level, we have sought to set standards that are measurable by the team, utilise national quality audits already available and use quality measures already started by team members. Standards measured need to be uncomplicated but useful to staff, so that motivation to complete them is maintained. Local data from national audits can be utilised so that work is not unnecessarily repeated, and also involvement in national audits is encouraged as staff can see a direct usefulness in taking part.
Measuring patient satisfaction, access, process and review
The standards we have chosen to measure are listed in Table 1. Patient satisfaction may be problematic as an outcome measure, Reference Elwyn, Buetow, Hibbard and Wensing4 as discussed later, but it is a primary standard because it asks patients directly what they think of the service they are getting. The national patient survey can compare local services with national averages and map changes over time. 5
Table 1. Quality standards for east Cornwall community mental health team (CMHT)
| Standard | Assessment | Means | Progress |
|---|---|---|---|
| Patient satisfaction and experience | Outcome | National patient survey 5 , local CMHT patient satisfaction survey | Completed 2008 |
| Length of time between referral and assessment appointment offered | Access and assessment | Audit of referrals and time before assessment offered | Completed 2008 |
| Adequate documentation of care plans, risk assessments, reviews, discharge process | Process | Case-note audit | Done in 2007, to be repeated for 2009 |
| Assessment of staff performance, ideally by 360-degree appraisal including patient feedback | Staff performance | The results of staff multisource feedback | Medical staff 360-degree appraisals completed, other staff performance indicators needed |
| Audit of the delivery of NICE guidelines for schizophrenia | Delivery of evidence-based practice | Audit of NICE guidelines | Started, aim for December 2009 |
| Audit of medication prescribing against NICE guidelines | Delivery of evidence-based prescribing | Audit of prescribing | For 2008, use local data on depot prescribing from national POMH audit 6 |
| Quality of psychological therapy | Process and outcome | CORE audit in psychological therapies | Ongoing |
| Audit of group work | Process and outcome | Undecided | To be done |
| Physical healthcare monitoring of patients with severe mental illness between the team and GP | Process and liaison with primary care | Audit of physical healthcare monitoring | Completed in December 2006, to repeat in 2009 |
| Percentage of patients with a psychosis in paid employment | Outcome | Audit of employment status of patients with psychosis | Audit completed in 2007, needs to be repeated in 2010 |
| Carers' support | Process and outcome | To devise local feedback form | To be completed by carer support worker |
Patients care about access and how long they have to wait before an assessment, and this can be measured fairly easily by staff with aims for maximum waiting times. For process and review, a case-note audit to monitor the adherence to care programme approach standards has been undertaken. As adequate documentation is necessary for safe clinical practice as well as medico–legal protection, documentation audits will need to evolve with greater use of electronic records.
Assessing staff performance
Assessment of staff performance in their clinical work has received a surprisingly low emphasis in measuring the quality of services. Multisource feedback, or 360-degree assessment is now mandatory for medics but not other staff. The 360 assessment for consultant psychiatrists (ACP 360) developed by the Royal College of Psychiatrists (www.rcpsych.ac.uk/clinicalservicestandards/centreforqualityimprovement/acp360.aspx), is an important measure of the quality of clinical work and needs to include feedback from patients. With the consent of each doctor, the current report includes feedback on medical staff, utilising the College ACP system for consultants and the multisource feedback workplace-based assessment for trainee psychiatrists, both already completed for staff appraisals. A future aim would be to develop multisource feedback for all clinical staff.
Evidence-based practice
Delivering evidence-based care is an aim for the service, but to measure all aspects would be overwhelming and what can seem straightforward, such as auditing National Institute for Health and Clinical Excellence (NICE) prescribing guidelines, can be complex and difficult. However, we can utilise data from the national Prescribing Observatory for Mental Health (POMH) survey on depot medication prescribing, 6 which we have already contributed to and which provides a clear framework. Audits of the implementation of NICE schizophrenia guidance have been published Reference MacPherson, Hovey, Ranganath, Uppal and Thompson7 and can be replicated locally.
Outcome data
The quality of psychological therapies and group work present a challenge. We shall start with an outcome audit of psychotherapy treatment (clinical outcomes in routine evaluation, CORE) Reference Evans, Mellor-Clark, Margison, Barkham, Audin and Connell8 as this had been introduced by psychologists in the team already and it has published validity and reliability. We are still exploring ways of auditing group work.
Employment of people with psychosis is important to patients and can be improved with active intervention. Reference Burns, Catty, Becker, Drake, Fioritti and Knapp9 It can be argued that having a job is a generic measure of overall functioning and therefore an outcome appropriate to define and measure. It is also a measure of social inclusion. The practical problem is that individuals who are stable and have a job are now often discharged back to primary care. It may be more valid to establish a joint register of individuals with severe mental illnesses between primary and secondary care, and audit the employment status of this group.
An audit of the physical care of individuals with schizophrenia in primary care, undertaken locally in 2006, has the advantage of measuring an important aspect of patients’ welfare while also monitoring the liaison between primary and secondary care. The POMH is pioneering national audits in this area and future audits may link in with this. Carer satisfaction will be audited by the carer support worker using a local tool.
Developing the quality report
Developing a quality report for our CMHT has been complex and cannot satisfy all desires. The activities of the team are as complex as some of the needs of our patients. Where do we start? We decided to start from this proposal as it is based on a systematic description of what we do and on work we had already done or started. We submitted a quality report in December 2008, although not all of the measures had been completed. We hope to complete outstanding audits by December 2009. Realistically, we feel that a similar report can be delivered every 3 years.
This framework can also give a focus for clinical audits for junior doctors. The team has a succession of F2 psychiatrists keen to complete audits for their portfolio but having only 4 months in the job. The F2 and ST4–6 doctors in the team have already contributed to the quality report and new trainees can quickly plug into the system on arrival.
Potential weaknesses of the proposal
Patient satisfaction is a measure that seems patient centred but is difficult to get right, as reported in a Health Technology Assessment Reference Crow, Gage, Hampson, Hart, Kimber and Storey10 and demonstrated by research in mental health settings. Reference Greenwood, Key, Burns, Bristow and Sedgewick11 It depends on patient expectations, not just the quality of services. However, it is a source of direct accountability to our patients and as such we felt it was appropriate to include in the report. Hopefully, the national patient survey 5 will be of sufficient quality and provide sufficient comparative data to be useful, although it provides trust-wide data rather than information for individual CMHTs. Our local satisfaction survey should be interpreted with the above reservations in mind.
A major omission from this plan is an audit of the care of people with a severe personality disorder. This group of patients, although small in number, have significant needs, can consume considerable resources and yet clear treatment aims are not always agreed between patients, clinicians and commissioners. We are still exploring how we might measure the quality of services to these patients.
After due consideration, we have not included a measure of quality of life as we wished to avoid introducing a new scale for clinicians who may be sceptical about filling in another piece of paperwork, but believed employment rates and CORE measures both reflect aspects of general quality of life.
Purpose of a quality report
What is the point of a quality report and how will we know whether this exercise has borne fruit? From a slightly cynical perspective, it is likely that this task will be demanded anyway from commissioners, so we feel it is better for local teams to take the initiative and measure what is important. This should focus the team on how improved care might be implemented. Real success will be achieved if a series of quality reports indicate a measurable improvement in the care the CMHT provides. This may take years to demonstrate, but ‘where there is no vision, the people perish’.
This paper describes a proposal to measure quality standards in a CMHT. Hopefully, it will generate debate, discussion and suggestions for improvement. Having a fixed template for all CMHTs is probably undesirable. ‘One size fits all’ solutions are rarely satisfactory, as we have experienced in Cornwall where the National Service Framework, 12 clearly designed for an urban environment, has been difficult to implement, with national targets putting unrealistic pressures on the service. These measures have come out of work that has evolved in the team over several years, and as such are more likely to be owned by staff. We have delivered a quality report using most of these measures, and hope it will be useful for patients, clinical staff, managers and commissioners.
Declaration of interest
None.
Open access
eLetters
No eLetters have been published for this article.