Approximately 3500 children under the age of 16 years have heart surgery each year in the United KingdomReference Cunningham, Franklin, Bridgewater and Deanfield1 and since 2000, all cardiac centres have contributed procedure data to the National Congenital Heart Disease Audit. Centre-specific mortality outcomes for individual procedures have been published online since 2007 by the National Congenital Heart Disease Audit.2 The focus of quality assurance and quality improvement initiatives has broadened to incorporate longer-term survivalReference Rogers, Pagel and Sullivan3 and non-fatal adverse outcomes (for instance, Jacobs et alReference Jacobs, O’Brien and Jacobs4).
The routine national mandated data collection as part of the United Kingdom audit provides a framework for future national monitoring and reporting of morbidity. Local routine monitoring of risk-adjusted mortality has been shown to be feasible and acceptableReference Pagel, Utley and Crowe5, and United Kingdom centres use software developed through a previous research study to do this. As survival continues to improve, developing routine monitoring tools for morbidities is necessary to support continued improvements in care.
In this paper, we report on the development of a prototype tool to support the routine monitoring by clinical teams of early post-operative morbidities following paediatric cardiac surgery as part of a 4-year prospective multi-centre research study in the United Kingdom. In the broader study, nine morbidities (Fig 1) were selected as important by a panel of family representatives, surgeons, intensivists, nurses, and paediatriciansReference Pagel, Brown and McLeod6, and then defined by a separate panel of clinicians and data managers.Reference Brown, Pagel and Brimmell7
Figure 1. Final set of morbidity icons. See Brown et alReference Brown, Pagel and Brimmell7 for morbidity definitions and their measurement protocols.
The incidence of each complication was measured among 3090 cases between 1 October, 2015 and 30 June, 2017 at 5 United Kingdom centres. In parallel to this data collection, we sought to develop a means for centres to, routinely and in a timely manner, monitor local morbidity rates using the incidence of each morbidity from the study data as a benchmark.
In the following, we first describe the process followed to develop both our tool and our graphical representations (Materials and methods section). Then, we present the output of our study (Results section), including the final set of icons representing morbidities and the summary displays of morbidities data organised into an automatically generated Microsoft PowerPoint presentation. We conclude (Discussion section) with some comments about the outcome of our study.
This paper represents the first United Kingdom work to develop a routine monitoring tool for morbidities following paediatric heart surgery for use in all specialist hospitals. While the tool is still in the prototype stage, it represents an important first step to adding morbidity information to the ongoing monthly mortality reviews within hospitals.
Materials and methods
Selection and definition of morbidities
The details of the selection and definition of the morbidities have been published elsewhere but we provide a brief overview here to provide context for the reader.
From 2014 to 2018, we undertook an National Institute for Health Research-funded project to select, define, and measure the incidence and impact of important early morbidities following paediatric surgery.8 This included prospective monitoring of consecutive cardiac surgery at 5 of the United Kingdom’s 10 paediatric cardiac specialist centres over 21 months. Below, we give a brief summary of how communication was chosen and defined for this study. A full write-up of the selection process of all measured morbidities is available in Pagel et alReference Pagel, Brown and McLeod6 and a full account of the definitions of each morbidity is available in Brown et alReference Brown, Pagel and Brimmell7 (see also Supplementary File S1).
A key aim of the project was to incorporate a broad set of perspectives, including those from family representatives and professionals from different sectors on what early morbidities were important to monitor in routine practice. We convened a study selection panel which met twice in 2014 to select up to 10 morbidities for prospective monitoring during the study. In the first panel, a longlist of 66 potential morbidities was discussed by the panel. These were drawn from a combination of a literature review, an online forum with parents, and three focus groups with parents held across the United Kingdom. For the first meeting, panelists were requested not to censor their suggestions on grounds of the perceived difficulty of definition or measurement. Through a combination of secret voting and discussion, a shortlist of 24 morbidities was selected for consideration by an independent definitions panel. At the second selection panel, nine morbidities were chosen for prospective monitoring by the panel. We note that inclusion on the panel of family representatives and clinicians from outside the tertiary surgical centres brought other issues such as problems feeding to greater prominence than if the panel had consisted solely of tertiary clinicians or if the study investigators had chosen the morbidities themselves.
We convened a separate definitions panel which met twice in 2014, following the first and second meetings of the selection panel, respectively. The panel included three paediatric cardiac surgeons (one was the chair), three paediatric cardiologists (one specialising in adult coronary heart disease), three paediatric intensive care specialists, and two children’s heart disease nurses. As part of their work, the definitions panel: (i) established the diagnostic criteria that constitute the definition of each of the chosen morbidities; (ii) defined the measurement protocol for each of the morbidities.
Initial design of icons and data summaries
As an initial step, we designed a set of icons intended to represent the morbidities in data summaries.
For morbidities affecting specific sites in the body (brain, kidney, bowel, pleural space, and surgical wound), we adapted widely used icons of that body site. For events (unplanned reoperation, major adverse event) and interventions (extracorporeal life support), we aimed to convey the essential characteristics of the morbidity. For feeding problems, we initially used safety iconography of a red circle with a bar across to indicate “nil by mouth”.
We then constructed basic data displays incorporating the icons to present hypothetical data on the counts and proportion of cases having each complication (in isolation, in combination with at least one of the other selected morbidities, and in total).
At this point, we visited several of the surgical centres involved in the study to discuss with data managers and available clinicians whether and how they envisaged routine monitoring of the measured morbidities being incorporated into their quality assurance processes. During each meeting, we discussed: local initiatives and practice concerning the monitoring and feedback of early morbidities or morbidities; ease of recognition of the icons developed; the team’s responses to our proposed data summaries and ideas for other data summaries that would be useful.
Feedback from these discussions and from presentations to the studies steering group informed the redesign of icons where necessary and informed the functional specification of the prototype software tool developed.
Prototype software tool development
We decided to build a prototype tool within Microsoft Office, specifically an Excel spreadsheet application that could be used to generate a PowerPoint presentation file containing graphical data summaries. Our choice was based on previous experience of developing an outcome monitoring tool for United Kingdom centres performing paediatric cardiac surgery and on the software tools currently used by sites to collate, analyse, and present data: all hospitals can easily have access to the tool, without the need for training or complex installation of new software.
Given the anticipated interest that routine monitoring of complication data may prompt from payers, regulatory bodies, families, and the media, and because of parallel initiatives to explain complication rates to families and patients, we sought to ensure that methods for monitoring morbidities, while primarily designed for an expert clinical audience, were accessible to non-experts.
Results
Icons
The final set of icons developed for use in graphical summaries of morbidity data are shown in Figure 1. Feedback from clinicians and data managers at the participating sites indicated that the icons developed were, generally, readily associated with the morbidities they were designed to represent.
Changes made to the initial designs to incorporate feedback were:
replacing the “nil by mouth” icon based on the international prohibition sign of a barred red circle with a drawing of an infant with a nasogastric tube (feedback was that the prohibition sign could be interpreted as the clinical team saying the child was not allowed to feed, rather than the child having difficulty feeding).
redrawing the blood bag component of the icon for “major adverse event” to avoid it looking like a syringe.
re-colouring the patient depicted in the icon representing extracorporeal life support to reflect feedback that the clinical experience (and indeed intent) was of children on extracorporeal life support being notably pink.
Incorporating feedback about summary displays
The summary displays of morbidities data were welcomed by data managers and clinicians, albeit with feedback and suggestions for improvement and development. Clinicians stressed the importance of expressing the incidence of morbidities as a percentage of operations performed as well as in terms of absolute counts. It was requested that, once the data from the ongoing study had been analysed, we add some form of benchmarking to place local morbidity data in the context of data from multiple sites, acknowledging that such benchmarking would not, initially, take account of case-mix differences between sites.
Concerning the incidence of different combinations of morbidities, there was interest in exploring the sequence of morbidities in individual patients. Although we recognised the clinical motivation for this request and created some “mock-ups” of how such displays would look (Fig 2), further discussion between the project team and clinical teams highlighted the need for a more in-depth study to understand sequencing in morbidity. In particular, the order in which morbidities are recorded in the data is not necessarily the same as the order of their clinical presentation. We thus did not include morbidity sequencing within this prototype tool. Once this further research is done to understand morbidity sequencing, how it can be extracted routinely from data and how such information can feed into monitoring tools for quality improvement, we will revisit its inclusion in this tool.
Figure 2. Example of representation of all possible sequences of morbidities (not based on actual data).
There was a degree of interest in building subgroup analyses into the prototype displays, with a particular focus on the potential for monitoring morbidity rates among specific complex diagnoses (for instance, those associated with a functionally univentricular heart) and patient groups (for instance, neonates). While we understood the value of routine monitoring of morbidity data to allow for subgroup analysis in time, we took the view that different subgroups might be of interest at different times at different centres, depending on local quality improvement initiatives, for example, and so should not be hardwired into the prototype tool. Subgroup analyses could, instead, be performed by changing the input data in the Excel tool to comprise only patients in a specific group.
Our initial ideas for presenting changing rates of morbidity over time were viewed as being too complicated, which led us to incorporate time series displays instead, employing the same formalisms as used in routine monitoring of mortality in United Kingdom centres. The following sections summarise the structure and content of the prototype tool designed incorporating this set of feedback and suggestions.
Development of the time series approach, building on widely used and accepted mortality variable life-adjusted display charts
Variable life-adjusted display charts are simple graphs providing an intuitive representation of the occurrence of a given clinical outcome over time, measured against a baseline risk. They were originally proposed as a way to indicate whether a surgeon’s outcomes were better or worse than might be expected based on the case-mix of their practice (difference between predicted and actual cumulative mortality)Reference Lovegrove, Valencia, Treasure, Sherlaw-Johnson and Gallivan9 and have been adopted by the United Kingdom coronary heart disease community for monitoring 30-day survival in children after heart surgeryReference Rogers, Pagel and Sullivan3, Reference Pagel, Utley and Crowe5.
We adapted variable life-adjusted display charts to measure the occurrence of a given morbidity over time compared to a constant national benchmark (i.e. a population baseline risk b between 0 and 1). In the chart, every procedure is plotted from left to right (in chronological order) on a horizontal axis (Fig 3):
if the procedure is not associated with the morbidity, the line moves up by an amount equal to the risk of occurrence of that morbidity (i.e. b);
if the procedure is associated with the morbidity, the line moves down by an amount equal to the chance of that morbidity not occurring (i.e. 1 − b).
Figure 3. Example of VLAD-style chart (not based on actual data). The incidence of feeding problems seems to be, on average, in line with the compared benchmark (baseline incidence) but with an interesting cyclic trend where periods characterised by an incidence lower than the baseline alternate with periods characterised by an incidence higher than the baseline.
Use of risk-adjusted rates instead of simple benchmark rates could be introduced in future refinements of the tool.
Excel tool
We developed a tool for the automatic creation of a document containing a structured set of the graphical summaries. “Navigation buttons” allow users to move through the presentation, starting from a “Home” slide and accessing different data summaries.
The tool was developed by embedding Visual Basic for Applications programming code into a Microsoft Excel spreadsheet which contained source data on procedures. The input data are simply a list of procedures (one procedure per row in the spreadsheet) along with procedure date, current life status, and diagnosed morbidities (yes/no for each of the morbidities considered in this study). Benchmark risks for each morbidity were taken from the overall results of the study as an input to generate time series graphs, but can also be specified by the user if wished (e.g. to correspond to local recent incidence). A “Run” button enables the automatic creation of a Microsoft PowerPoint presentation as a separate file. The generated PowerPoint presentation incorporates action buttons to facilitate navigation (see Supplementary File S2 for an example, not based on actual data). Full documentation of the tool is provided in Supplementary File S3 (“Morbidity monitoring tool user guide.pdf”).
From the home page, users can access a set of slides (Fig 4a and b) summarising the number of morbidities reported in the source data. Icons are reported in a circular layout and labelled with numbers or percentages (the user can easily switch between the two types of visualisation at the press of a button) representing morbidity occurrences. Navigation buttons also allow to switch between: (i) summaries considering procedures with exactly one morbidity; (ii) summaries considering procedures with one or more morbidities; (iii) summaries considering procedures associated with at least two morbidities. Information on the number of deaths and on procedures without any of the recorded morbidities is also reported.
Figure 4. Examples of slides included in the output presentation (not based on actual data).
A button “Time series” gives access to a slide reporting variable life-adjusted display charts for all morbidities (Fig 4c), where users can have a quick overview of the temporal trend of each morbidity across a time window covering the source data.
Finally, from the “Procedures with multiple morbidities” slide (Fig 4b), users can click on each morbidity icon to access a slide summarising how many times that morbidity co-occurred with each of the other morbidities across all procedures (Fig 4d).
The output shown in Figure 4 was produced from data collected between 1 October, 2015 and 30 June, 2017 at five United Kingdom centres. A total of 3090 procedures were included in the dataset, among which: 2415 procedures were associated with none of the selected morbidities; 675 procedures were associated with either one or more than one morbidities (418 and 257, respectively); in 105 procedures, the child died. The Table 1 summarises these data by morbidity.
Table 1. Number of procedures associated with each morbidity
3090 procedures in total, of which 2415 without any of the selected morbidities. Data prospectively collected between 1 October, 2015 and 30 June, 2017 at five United Kingdom centres
* Excluding extra corporeal life support. See Brown et alReference Brown, Pagel and Brimmell7 for morbidity definitions and their measurement protocols
Discussion
We have described the design process and outcomes of an exercise to develop a tool for use in routinely monitoring morbidity data. The tool allows users to generate a set of PowerPoint slides from a simple Excel spreadsheet. The PowerPoint slides summarise the morbidity incidence in the context of national data for routine feedback of recent outcomes to clinicians in multi-disciplinary team meetings or quality improvement collaboratives.
The value of engaging early and openly with potential end users while developing the tool was reinforced several times, with feedback and suggestions from clinicians and data managers playing a key role in the specification for the current prototype. Also, this engagement provided us with valuable insight to the limitations of the tool at present and additional functionality that clinicians would wish to see in the future.
The prototype tool has been presented to a group of clinicians and data managers associated with the study. There was some discussion of how the use of variable life-adjusted display charts in which an ascending line indicates a lower than benchmarked incidence of morbidities (chosen to be consistent with current presentation of (predicted–actual) mortality) could be reconciled with standard presentations of, say, infection data.
That aside, the prototype tool was received positively and several centres expressed an interest in using the tool once final decisions had been made at a national level about the set of morbidities recommended for routine monitoring and any modifications to the definitions used in the course of the study. For instance, a Clinical Nurse Specialist at one of the United Kingdom centres involved gave the following feedback:
It definitely would be a useful tool for reviewing morbidity. The fact that the aim is to make this work with the National Institute for Cardiovascular Outcomes Research extracts and to present the data in an easy to use slide deck will be invaluable as units move forward. More frequently we need to use data visualisation tools like this to explain the complexity of the data… The slides are easy to use and understand and the fact that in essence we can drill down to see more specific data is great.
The morbidities discussed in this paper were collected as part of a prospective research study. The United Kingdom National Congenital Heart Disease Audit have recently started collecting data on some but not all of these morbidities and they are not yet included in the public annual report. Data to national audit are submitted quarterly by specialist hospitals, where case ascertainment is by the local surgical team as it is for all the other data in the audit. The data are externally validated (sample for each centre) as they are for any other data item. As routine monitoring of morbidities is fully adopted by and embedded within United Kingdom centres with the support of the National Congenital Heart Disease Audit, the next steps for this work would involve any necessary adaption (to the set of morbidities collected) and implementation of the prototype tool at surgical centres. Future development of additional functionality to incorporate risk adjustment for local case-mix and to support robust subgroup analysis would then be possible, subject to sharing of accrued data for this purpose.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951119002956
Acknowledgements
Study contributors: Sheryl Snowball, Luke Maidment, Sarah Bohannon, Liz Smith, Rodney Franklin, Alison Jones, Kate Penny-Thomas, Joanne Webb, Sinead Cummins, John Stickley, Natasha Khan, Teresa Dickson, Ray Samson, Isobel Mcleod, Paul Wellman, Thomas Witter, Rhian Lakhani, Karen Sheehan, Kathleen Selway, Carrie Cherrington, Andrew Parry, Rob Tulloh, Bill Gaynor, Lisa Allera, Kate Bull, Trevor Ritchens, Branko Mimic, Jon Smith, Lyvonne Tume, Vibeke Hjortdal, Michael Vath, Tom Treasure, Anne Keatley Clarke, Bea Tuten, and all the patients who participated in the study.
Financial Support
This project was funded by the National Institute for Health Research Health Services and Delivery Research programme (Project No: 12/5005/06). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the National Institute for Health Research Health Services and Delivery Research programme or the United Kingdom Department of Health and Social Care. Katherine L. Brown and Victor Tsang were supported by the National Institute for Health Research Biomedical Research Centre at Great Ormond Street Hospital for Children National Health Service Foundation Trust and University College London. Martin Utley was supported in part by the National Institute of Health Research Collaboration for Leadership in Applied Health Research and Care North Thames at Bart’s Health National Health Service Trust.
Conflicts of Interest
None.
Ethical Standards
The study has ethical approval from London City Road Research Ethics Committee (14-LO-1442).
Open access
