Introduction
Treatment of ventilator-associated infection (VAI) is among the most common reasons antibiotics are prescribed in the pediatric intensive care unit (PICU). It is estimated that approximately one-third of antibiotics used to treat ventilator-associated pneumonia (VAP) and ventilator-associated tracheitis (VAT) are prescribed inappropriately. Reference Blinova, Lau and Bitnun1–Reference Tribble, Lee and Flett4 The reasons for antibiotic overuse in this scenario are complex, and include the lack of a diagnostic gold standard, inability of tracheal aspirate cultures to distinguish infection from colonization, and frequent overlap in infectious and non-infectious symptoms in medically complex, mechanically ventilated patients. Reference Kalil, Metersky and Klompas5–Reference Prinzi, Parker, Thurm, Birkholz and Sick-Samuels9
Diagnostic test stewardship is an antibiotic stewardship strategy in which the process of ordering, performing, or reporting a diagnostic test is modified to best inform the decision to prescribe antibiotics. Reference Morgan, Malani and Diekema10,Reference Fabre, Davis and Diekema11 The effectiveness of this approach is best studied for interventions focused on reducing urine culture overuse, with several reports demonstrating reductions in both test overutilization and antibiotic use following diagnostic test stewardship efforts. Reference Trautner, Grigoryan and Petersen12–Reference Vaughn, Gupta and Petty14 Similar to the lower urinary tract, where colonization with potentially pathogenic bacteria can be detected in urine cultures obtained in patients with asymptomatic bacteriuria, tracheostomy and endotracheal tubes of mechanically ventilated patients are typically colonized with potentially pathogenic bacteria as early as 24 hours after placement. Reference Durairaj, Mohamad and Launspach15–Reference Willson, Kirby and Kicker17 A “positive” culture, therefore, does not necessarily indicate infection, but may be misinterpreted as such, thereby contributing to overdiagnosis and downstream antibiotic overuse. Reference Willson, Kirby and Kicker6,Reference Prinzi, Parker, Thurm, Birkholz and Sick-Samuels9,Reference Albin, Saravolatz, Petrie, Henig and Kaye18 Few studies have evaluated the impact of diagnostic test stewardship focused on tracheal aspirate cultures, though limited available data supports the effectiveness of these interventions in the PICU setting. Reference Sick-Samuels, Linz and Bergmann19,Reference Ormsby, Conrad and Blumenthal20 We therefore sought to build upon these data by evaluating the effect of implementing a tracheal aspirate culture guideline on both culture utilization and antibiotic use in our tertiary care PICU.
Methods
Study setting
This study was performed in the tertiary care PICU and progressive care unit (PCU) at Children’s Hospital of Philadelphia (CHOP). The PICU includes 74 beds and the PCU, a postacute care unit for patients with chronic respiratory failure, includes 24 beds. Prior to this project, no standardized guideline regarding obtaining tracheal aspirate cultures was available. Throughout the study period, broad-spectrum antibiotic administration required preauthorization from an established antimicrobial stewardship program for durations exceeding 48 hours, including for the treatment of VAI.
Study design and population
We performed a quality improvement intervention assessing the impact of the guideline to reduce tracheal aspirate culture orders in mechanically ventilated children (Figure 1). All cultures collected in the CHOP PICU or PCU between September 2018 to August 2020 (preintervention) and September 2020 to August 2021 (postintervention) were included in the primary analysis. We additionally performed a sustainability assessment including cultures collected between September 2021 and August 2022.
Figure 1. Consensus-based local guideline for collection of tracheal aspirate cultures.
Guideline development
We established a multidisciplinary team including pediatric critical care, pulmonology, and infectious diseases attending physicians, critical care nurses and respiratory therapists, and a clinical microbiologist to generate a consensus-based guideline establishing when a tracheal aspirate culture should and should not be obtained (Figure 1). Our local guideline was informed by national guidelines, published literature, expert opinion, and a retrospective review evaluating common clinical scenarios in which tracheal aspirate cultures were collected in the PICU and PCU (Supplemental Table 1).
Interventions
We performed a series of Plan-Do-Study-Act (PDSA) cycles to promote guideline implementation and adherence. The first PDSA cycle occurred in August–September 2020 and included education of PICU, infectious diseases, and pulmonary clinicians in virtual division meetings, as well as linking the tracheal aspirate culture guideline within the tracheal aspirate culture order in the electronic health record. The guideline was shared by email and posted in all PICU/PCU clinician workrooms, and screensavers emphasizing key elements of the guideline were displayed throughout the month of August and September. Education about the new guideline was shared in preshift nursing and respiratory therapy huddles.
The second PDSA cycle focused on audit with feedback and targeted education, occurring from October 2020 to January 2021. During this time, data on performance and key areas for improvement were shared with PICU clinicians approximately monthly during a standing quality improvement meeting. In addition, an email was sent weekly by the study principal investigator (K.C.) to the specific PICU attendings, fellows, and nurse practitioners on clinical service reminding them of the guideline recommendations.
Finally, the third PDSA cycle was performed in April 2021 and involved resuming weekly emails to on-service PICU clinicians during the month of April reminding them of the tracheal aspirate culture guideline. We performed no further PDSA cycles and continued to measure the sustainability of the intervention until August 2022.
Study measures and outcome
The primary outcome measure was the monthly rate of tracheal aspirate cultures per 100 ventilator days. Cultures collected in the PICU or PCU or in the ED on the day of admission were included. Cultures collected from bronchoalveolar lavage, protected brush, pleural fluid, or lung tissue were excluded. All ventilator days contributed by patients with either endotracheal tubes or tracheostomy tubes were counted in the denominator. All cultures collected between September 2019 and August 2021 were reviewed by a physician (K.C. or G.S.) and classified as “appropriate” or “inappropriate” based on the consensus guideline recommendations (Figure 1), with a recommendation to “send” or “consider sending” classified as “appropriate.” Fifty-one cases were reviewed by both physicians and interrater reliability was excellent (kappa = 0.84).
Antibiotic use in mechanically ventilated patients was measured using broad-spectrum days of therapy (DOT) per 100 ventilator days. Antibiotic DOT is the sum of all antibiotics administered on a given calendar day in a patient mechanically ventilated on that day. Broad-spectrum antibiotics included third and fourth generation cephalosporins, carbapenems, fluoroquinolones, vancomycin, linezolid, and piperacillin-tazobactam. Broad-spectrum antibiotics per 100 ventilator days was selected given that these antibiotics are commonly used for treatment of VAI, while narrow spectrum antibiotics may be used for more varied indications not impacted by this intervention, including prophylaxis. The frequency of treatment of VAI following a tracheal aspirate culture and per 100 ventilator days was ascertained through manual chart review.
Key balancing measures included PICU/PCU length of stay, 14-day PICU/PCU and hospital readmission, in-hospital mortality, and VAP diagnosis. VAP diagnosis was defined both using International Classification of Diseases, Tenth Revision (ICD-10) as well as National Healthcare Safety Network (NHSN) surveillance definitions. 21
Analysis
For our primary analysis, we generated statistical process control (SPC) charts measuring the rate of tracheal aspirate cultures per 100 ventilator days. The centerline reflects the mean number of tracheal aspirate cultures per 100 ventilator days. Upper and lower control limits reflect three standard deviations above or below the mean. Starting centerline and control limits were calculated using a baseline period of 18 months prior to the COVID-19 pandemic (September 2018 to March 2020). We applied Nelson rules to establish special cause variation in tracheal aspirate culture utilization and antibiotic use measures, with eight or more consecutive points above or below the centerline prompting a centerline shift and recalculation of the mean and control limits. Reference Nelson22 We elected to allow a centerline shift at the point of the COVID-19 pandemic because pandemic-related mitigation measures and case-mix changes within our PICU were a clear mechanism for a change in tracheal aspirate culture rates and antibiotic use. As a sensitivity analysis to differentiate the impact of COVID-19 pandemic from the intervention, we analyzed the sustainability of the intervention for an additional year (September 2021–August 2022), when many pandemic mitigation strategies had been lifted. Antibiotic DOT per 100 ventilator days were assessed using similar methods. We also performed a secondary analysis using Poisson regression to compare the rate of respiratory cultures per 100 ventilator days in the pre- versus postintervention periods using incidence rate ratios (IRR).
Baseline clinical and demographic characteristics and balancing measures were compared using chi square testing for categorical variables and the Wilcoxan rank-sum test for continuous variables. All analyses were performed using Stata 14 (StataCorp, College Station, TX). Study findings are reported in alignment with the Standards for Quality Improvement Reporting Excellence (SQUIRE) 2.0. 23
Ethical considerations
This project was deemed exempt research by the CHOP Institutional Review Board.
Results
Demographic and clinical characteristics
A total of 1997 patients had one or more tracheal aspirate cultures collected in the two-year preintervention period and 978 patients had one or more tracheal aspirate cultures collected in the one-year postintervention period. Postintervention patients were slightly older (5.5 versus 4.6 years, P < 0.01), less likely to have a congenital or genetic condition (32 versus 39%, P < 0.01), more likely to receive a vasopressor within 24 hours of tracheal aspirate culture collection (16 vs 11%, P < 0.01), and less likely to receive inhaled nitric oxide within 24 hours of tracheal aspirate culture collection (7 versus 9%, P = 0.03) (Table 1).
Table 1. Baseline clinical and demographic characteristics
Note. IQR, inter-quartile range; CCC, complex chronic condition; PRISM, pediatric risk of mortality.
a Available for 477 patients in the preintervention group, 199 in the postintervention group.
Tracheal aspirate culture utilization
During the 18-month baseline period, the rate of tracheal aspirate cultures was 4.6 cultures per 100 ventilator days. A centerline shift downward to 3.1 cultures per 100 ventilator days occurred starting in April 2020, coincident with the peak of the COVID-19 pandemic in our region. Following guideline implementation in September 2020 and our first two PDSA cycles, the centerline again shifted downward starting in December 2020 to 2.0 cultures per 100 ventilator days. This centerline was maintained through the planned intervention period (September 2020–August 2021) as well as for the additional one-year sustainability assessment (September 2021–August 2022) (Figure 2).
Figure 2. Impact and sustainability of respiratory culture diagnostic test stewardship intervention, September 2018–Augsut 2022. Legend: PDSA #1 (August–September 2020): clinician education, link to guideline within respiratory culture order, posting guideline in PICU/PCU team workrooms, and screen savers; PDSA #2 (October 2020–January 2021): audit with feedback, weekly emails to on-service PICU/PCU clinicians; PDSA #3 (April 2021): weekly emails to on-service PICU clinicians. Abbreviations: COVID-19, coronavirus disease 2019; PDSA, Plan-Do-Study-Act; UCL, upper control limit; LCL, lower control limit.
The mean number of tracheal aspirate cultures collected monthly decreased from 68 in the preintervention period (September 2018–August 2020) to 32 in the postintervention period (September 2020–August 2021). The monthly rate of tracheal aspirate cultures per 100 ventilator days decreased from 4.3 cultures per 100 ventilator days to 2.3 cultures per 100 ventilator days (IRR 0.52, 95% confidence interval 0.47–0.59, P < 0.01). When limited to cultures from the PICU only, the monthly rate decreased from a baseline of 5.2 cultures per 100 ventilator days to 2.8 cultures per 100 ventilator days (IRR 0.54, 95% confidence interval 0.47–0.62, P<0.01). When limited to cultures obtained in the PCU only, the monthly rate decreased from a baseline of 1.7 cultures per 100 ventilator days to 0.8 cultures per 100 ventilator days (IRR 0.48, 95% confidence interval 0.36–0.62, P < 0.01) (Table 2).
Table 2. Process and balancing measures
Note. IRR, incidence rate ratio; CI, confidence interval; PICU, pediatric intensive care unit; PCU, progressive care unit (postacute care unit); DOT, days of therapy; VAI, ventilator-associated infection; NHSN, National Healthcare Safety Network; VAP, ventilator-associated pneumonia; ICD-10, international classification of diseases, version 10.
a Measured 9/2019–8/2021.
Tracheal aspirate culture appropriateness
A total of 1153 tracheal aspirate cultures were reviewed between September 2019 and August 2021. During the preintervention baseline, 67% of cultures were classified as inappropriate, versus 60% in the postintervention period (P = 0.04) (Table 2). Considering only the final 6 months of the postintervention period, 50% of cultures were classified as inappropriate (Figure 3).
Figure 3. Appropriateness of Tracheal Aspirate Culture Utilization, September 2019–August 2021. Legend: PDSA #1 (August–September 2020): clinician education, link to guideline within respiratory culture order, posting guideline in PICU/PCU team workrooms, and screen savers; PDSA #2 (October 2020–January 2021): audit with feedback, weekly emails to on-service PICU/PCU clinicians; PDSA #3 (April 2021): weekly emails to on-service PICU clinicians. The total bar indicates the rate of tracheal aspirate cultures per 100 ventilator days, including both appropriate and inappropriate cultures. The black portion of the bar indicates the rate of appropriate cultures, while the gray portion of the bar indicates inappropriate tracheal aspirate cultures. The bars are labeled with the percentage of all cultures that are inappropriate (percentage on the gray bars) and appropriate (percentage on the black bars). Classification of appropriate versus inappropriate cultures was based on physician adjudication using the respiratory culture guideline as the gold standard. Abbreviations: COVID-19, coronavirus disease 2019; PDSA, Plan-Do-Study-Act.
Antibiotic use measures
During the 18-month baseline prior to the COVID-19 pandemic, there was a mean of 32.0 broad-spectrum antibiotic DOT per 100 ventilator days. We observed a centerline shift downward to 28.7 broad-spectrum DOT per 100 ventilator days in May 2020, with no further changes to broad-spectrum antibiotic DOT per 100 ventilator days following our intervention (Figure 4). Antibiotic DOT per 100 ventilator days decreased in the pre- versus postintervention periods from 31.9 to 29.3 broad-spectrum DOT per 100 ventilator days (IRR 0.92, 95% confidence interval 0.89–0.95, P < 0.01). Antibiotic treatment of clinician-diagnosed VAI decreased following the intervention from a baseline of 1.0 treatment courses per 100 ventilator days to 0.7 treatment courses per 100 ventilator days (IRR 0.69, 95% confidence interval 0.55–0.86, P < 0.01). The proportion of patients treated for VAI following collection of a tracheal aspirate culture increased from a baseline of 0.24 in the preintervention period to 0.31 following the intervention (P = 0.03) (Table 2).
Figure 4. Broad-spectrum antibiotic DOT per 100 ventilator days, September 2018–August 2021. Legend: PDSA #1 (August–September 2020): clinician education, link to guideline within respiratory culture order, posting guideline in PICU/PCU team workrooms, and screen savers; PDSA #2 (October 2020–January 2021): audit with feedback, weekly emails to on-service PICU/PCU clinicians; PDSA #3 (April 2021): weekly emails to on-service PICU clinicians. Abbreviations: COVID-19, coronavirus disease 2019; PDSA, Plan-Do-Study-Act; UCL, upper control limit; LCL, lower control limit.
Balancing measures
No differences were detected in PICU mortality (7% versus 5%, P = 0.16) or PICU length of stay (median 4 days versus 4 days, P = 0.97). Both hospital length of stay (9 days versus 7 days, P < 0.01) and frequency of PICU readmission (2% versus 1%, P = 0.04) decreased in the postintervention period. There was no increase in the frequency of VAP per NHSN criteria, VAP using ICD-10 diagnosis codes, or the frequency of septic shock following the intervention (Table 2).
Discussion
We conducted a diagnostic test stewardship intervention, which included implementation of a guideline, clinician education, and audit with feedback, to reduce overuse of tracheal aspirate cultures in mechanically ventilated patients in a single, tertiary care PICU. Following the intervention, we demonstrated a sustained reduction in culture utilization up to two years after the initial intervention, with no increase in mortality, length of stay, PICU readmissions, or VAP diagnosis. Antibiotic use, as measured by the number of antibiotic courses per 100 ventilator days, also decreased during the postintervention period. This work adds to the published data supporting the role of tracheal aspirate culture diagnostic test stewardship in the ICU setting and introduces several novel findings.
Consistent with our findings, Sick-Samuels et al demonstrated a 41% decrease in tracheal aspirate culture orders from a baseline of 10.9 cultures per 100 ventilator days to 6.5 cultures per 100 ventilator days after implementing a tracheal aspirate culture algorithm, without a change in mortality, readmissions, or length of stay. Reference Sick-Samuels, Linz and Bergmann19 Ormsby and colleagues similarly decreased tracheal aspirate culture utilization from a baseline of 10.7 cultures per 100 ventilator days to 7.1 cultures per 100 ventilator days through a series of interventions, including standardizing the process of collecting a tracheal aspirate culture, implementing lab rejection criteria for cultures collected within 72 hours, and through implementation of a consensus guideline for tracheal aspirate culture collection. Reference Ormsby, Conrad and Blumenthal20 Of note, both studies had a substantially higher rate of tracheal aspirate culture utilization than our cohort, suggesting that even in settings with lower baseline tracheal aspirate culture utilization, including in the postacute care setting, tracheal aspirate culture diagnostic test stewardship appears safe and effective.
Further, and a relative strength of our study, we additionally systematically evaluated the appropriateness of tracheal aspirate cultures collected using our consensus guideline as a gold standard for adjudicating appropriateness. Reassuringly, we observed a decrease in “inappropriate” tracheal aspirate culture utilization, while the rate of “appropriate” testing increased following our intervention. Coupled with no observed changes in patient-centered outcomes including length of stay, mortality, and VAP diagnosis, this finding supports both the safety of such diagnostic test stewardship interventions as well as the ability of these interventions to specifically target unnecessary testing, including in the complex PICU environment. Additionally, we believe providing feedback on specific scenarios where cultures were inappropriately obtained facilitated uptake of our intervention. In a mixed-methods study conducted concurrently with this diagnostic test stewardship effort, we identified that practice variation as to what clinical scenarios triggered collecting a respiratory culture as well as collecting tracheal aspirate cultures as a “default” practice were drivers of culture overuse. By focusing our tracheal aspirate culture guideline and our monthly feedback on specific clinical scenarios where tracheal aspirate cultures are particularly low yield (for example, in the setting of isolated fevers or isolated secretion changes), we were able to at least partially address these barriers. Reference Chiotos, Marshall and Kellom24
We also demonstrated a reduction in antibiotic treatment of VAI following the intervention, lending additional support that diagnostic test stewardship is a promising antibiotic stewardship strategy in the PICU setting. Reference Barlam, Cosgrove and Abbo25 This finding aligns with work done by Woods-Hill and colleagues, who performed a multicenter study evaluating the impact of blood culture stewardship across 14 PICUs, which demonstrated a significant reduction in broad-spectrum antibiotic use following the intervention. Reference Woods-Hill, Colantuoni and Koontz26 Outside the PICU setting, the potential for diagnostic test stewardship interventions to reduce antibiotic overuse is also illustrated by urine culture diagnostic test stewardship efforts in adults. Reference Trautner, Grigoryan and Petersen12–Reference Vaughn, Gupta and Petty14 Finally, greater utilization of tracheal aspirate cultures has been associated with greater antibiotic utilization, highlighting tracheal aspirate culture diagnostic test stewardship as a high-yield target for future PICU-based antibiotic stewardship efforts. Reference Prinzi, Parker, Thurm, Birkholz and Sick-Samuels9,Reference Albin, Saravolatz, Petrie, Henig and Kaye18
Finally, commonalities in the various consensus-based guidelines across reports highlight key targets for reducing tracheal aspirate culture overuse. First, cultures collected in the setting of isolated fever without any other symptoms localizing to the respiratory tract account for between 20% and 44% of tracheal aspirate culture orders, highlighting this as a potentially impactful target to reduce overall culture utilization rates. Reference Chiotos, Marshall and Kellom24,Reference Sick-Samuels, Fackler, Berenholtz and Milstone27 Second, and likely related, tracheal aspirate cultures are obtained reflexively along with blood and urine cultures in the evaluation of fever or other instability in up to approximately one-third of patients. Reference Albin, Saravolatz, Petrie, Henig and Kaye18,Reference Sick-Samuels, Fackler, Berenholtz and Milstone27 In a study conducted in an adult ICU, having a tracheal aspirate cultures obtained as part of a “pan culture” event increased the odds of receiving two or more days of broad-spectrum antibiotics by approximately 2.5-fold. Reference Albin, Saravolatz, Petrie, Henig and Kaye18 Finally, repeat cultures collected within 72 hours are seldom informative but are nevertheless common, so implementation of lab-based rejection criteria or other automated decision support to avoid oversampling within a short interval may prove to be a low-burden, but impactful interventions. Reference Ormsby, Conrad and Blumenthal20,Reference Sick-Samuels, Fackler, Berenholtz and Milstone27,Reference Feldman, Shah and Ahn28
Our study has several important limitations. First, we implemented our intervention during the COVID-19 pandemic, which itself impacted culture rates and antibiotic use prior to our intervention, likely due to impacts on respiratory virus epidemiology and PICU case mix. Reference Kanthimathinathan, Buckley and Davis29 We sought to overcome this limitation by analyzing an additional year of postintervention data and demonstrated no change in our centerline. Although culture utilization rates were trending upward at the end of this time, rates nevertheless remained substantially lower than the pre-COVID-19 baseline period. We also evaluated the appropriateness of tracheal aspirate cultures and showed a significant decrease in the proportion of cultures classified as inappropriate, an outcome measure less sensitive to pandemic effects and more specific to the intervention itself. Second, our study was performed in a single center and may not generalize to other settings. However, the consistency of our findings across at least two other studies strengthens generalizability, at least to other tertiary care PICUs. Finally, our assessment of tracheal aspirate culture appropriateness is limited to only scenarios where cultures were obtained, rather than scenarios where cultures were not obtained. Such an analysis was not feasible given that we were unable to systematically ascertain scenarios where a clinician decided not to obtain a culture.
Overall, our findings add to a growing body of literature supporting tracheal aspirate culture diagnostic test stewardship as an effective strategy to both reduce test overutilization and decrease antibiotic overuse stemming from overdiagnosis of VAI. Future studies should focus on optimal strategies for implementing tracheal aspirate culture stewardship interventions, which will likely include clinician-facing interventions such as guidelines coupled with lab-based interventions including rejection or reflex criteria.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2024.105.
Acknowledgements
The authors thank the Respiratory Culture Quality Improvement team for their support of this research study. This research was supported in part by a CDC Cooperative Agreement FOA#CK16-004-Epicenters for the Prevention of Healthcare Associated Infections.
Financial support
This work was supported by the Agency for Healthcare Research and Quality [K12-HS026393 to K.C.] and the National Institutes of Health [T32GM112596-06 to G.K., K23HL151381 to C.W.H.].
Competing interests
G.P.K. reports using a medical device loaned to CHOP for research purposes unrelated to this manuscript. All other authors report no conflicts.
