Hostname: page-component-848d4c4894-8bljj Total loading time: 0 Render date: 2024-06-30T08:56:18.324Z Has data issue: false hasContentIssue false

Comparison of the performance of a clinical classification tree versus clinical gestalt in predicting sepsis with extended-spectrum beta-lactamase–producing gram-negative rods

Published online by Cambridge University Press:  07 March 2022

Hayden L. Smith
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa
Mohamed A. Elfeki
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Dana Lowry
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Rebecca Sabates
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Molly Ropte
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Katherine R. Sittig
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Stephanie Finch
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Kristina Braun
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
William Pruett
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Aaron Wasson
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Kenna Fischman
Affiliation:
Iowa Methodist Medical Center, UnityPoint Health-Des Moines, Des Moines, Iowa Internal Medicine Department, University of Iowa, Des Moines, Iowa
Jennifer Peterson
Affiliation:
Des Moines University College of Osteopathic Medicine, Des Moines, Iowa
Rossana Rosa*
Affiliation:
Internal Medicine Department, University of Iowa, Des Moines, Iowa Infectious Diseases Service, UnityPoint Health–Des Moines, Des Moines, Iowa
*
Author for correspondence: Rossana Rosa MD, MSc, Jackson Health System, 1611 NW 12th Ave, Miami, FL33136. E-mail: rossana.rosa@jhsmiami.org

Abstract

A clinical decision tree was developed using point-of-care characteristics to identify patients with culture-proven sepsis due to extended-spectrum β-lactamase–producing Enterobacterales (ESBL-PE). We compared its performance with the clinical gestalt of emergency department (ED) clinicians and hospital-based clinicians. The developed tree outperformed ED-based clinicians but was comparable to inpatient-based clinicians.

Type
Concise Communication
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction in any medium, provided the original article is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America

Infections with extended-spectrum β-lactamase–producing Enterobacterales (ESBL-PE) are associated with a higher mortality risk compared to infections by susceptible organisms, Reference Shamsrizi, Gladstone and Carrara1 due partly to inadequate empirical antimicrobial therapy. Reference Rottier, Ammerlaan and Bonten2 Carbapenems are the preferred agents for the treatment of bloodstream infections due to ESBL-PE, Reference Harris, Tambyah and Lye3 but their widespread use can lead to acquisition of carbapenem-resistant organisms. Reference van Loon4

Clinical decision trees (hereafter referred to as decision trees) have been proposed as a tool to determine a patient’s probability of infection with an ESBL-PE to better optimize empirical antimicrobial therapy. Reference Goodman, Lessler and Cosgrove5,Reference Sousa, Pérez-Rodríguez and Suarez6 However, the applicability of existing decision trees might be limited by baseline antimicrobial resistance rates and specific demographic characteristics of the cohorts used to build them. We built a decision tree to identify patients presenting with sepsis due to ESBL-PE in a setting with low prevalence of resistance and compared it to clinicians’ ability to correctly choose empiric antimicrobial therapy.

Methods

We conducted a retrospective review of medical records from 3 hospitals within a health system in Des Moines, Iowa, from July 2015 to December 2019. We first identified adult patients that met the Sepsis-2 criteria for sepsis, severe sepsis, or septic shock at the time of presentation to the emergency department (ED) and had a blood culture obtained as part of their evaluation. Reference Dellinger, Levy and Rhodes7 These definitions comprised the criteria applied at our hospitals during the study period. The final cohort included only individuals with culture-proven infection defined as growth of Enterobacterales in at least 1 blood culture. We included the first episode of bacteremia per patient. Patients with growth from only nonsterile sites (eg, urine and sputum) were excluded due to difficulties in differentiating colonization versus infection.

The study outcome was bloodstream infection with an ESBL-PE (Supplementary Document 1). Based on previous studies, the point-of-care variables of interest were age, documented history of ESBL-PE colonization or infection in prior 2 years, documented antibiotic use in prior 6 months, care facility residence, and central line present on admission. Reference Goodman, Lessler and Cosgrove5 Empirical administration of a carbapenem was used as a surrogate indicator of a clinician’s level of suspicion (ie, clinical gestalt) for infection with an ESBL-PE. Because empiric antibiotic selection choices between ED and hospital-based clinicians often varies, we decided to evaluate these 2 groups separately.

We estimated descriptive statistics using a classification tree (r package: rpart version 4.1-13 software, R Foundation for Statistical Computing, Vienna, Austria) was fit with ESBL-PE status as the dependent variable and point-of-care information was fit as the independent variable. ESBL-PE status was considered a potential rare event in the patient population. Steps were taken to control threats related to analyzing an imbalanced outcome (Supplementary Document 2).

The final pruned tree was fit to the full data set, and ESBL-PE screening metrics included the following: sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and accuracy. The area under the receiver operating characteristic curve (AUROCC) for the final tree was compared with both the AUROCC for patients prescribed a carbapenem by the ED or by admitting hospital-based clinician. Further details of the tree-building process are presented in Supplementary Document 2. The study protocol was reviewed and approved by the local institutional review board.

Results

The study sample included 621 patients, with 56 (9%) were positive for an ESBL-PE infection. Patient characteristics stratified by ESBL-PE status are presented in Table 1 and Supplementary Figure 1. Of the study patients, 28 (4%) had a history of ESBL-PE infection or colonization in the previous 2 years, 97 (16%) came from a care facility, and 28 (5%) had a central line present on admission.

Table 1. Emergency Department Point-of-Care Information Available in Patients Presenting With Sepsis, Severe Sepsis, or Septic Shock (N = 621)

Note. ESBL-PE, extended-spectrum β-lactamase–producing Enterobacterales; SD, standard deviation.

a Units unless otherwise specified.

b Not used in tree as point-of-care information.

c Not point-of-care information.

The selected decision tree for ESBL-PE status included 2 splits. The first split was between patients with a documented history of ESBL-PE and patients without an ESBL-PE history. These groups were then split based on whether they lived in a care facility (Fig. 1 and Supplementary Fig. 2). Patients with a history of ESBL-PE had a 71% (95% CI, 53%–90%) higher rate of ESBL-PE than patients without a history of ESBL-PE and non–care-facility residence. Patients without a history of ESBL-PE, but with care-facility residence, had a 14% (95% CI, 4%–23%) higher rate of ESBL-PE than patients without an ESBL-PE history and a non–care-facility residence. The sensitivity and specificity of the tree were 0.64 (95% CI, 0.50–0.77) and 0.86 (95% CI, 0.83–0.89), respectively. The positive predictive value and negative predictive value were 0.32 (95% CI, 0.23–0.41) and 0.96 (95% CI, 0.94–0.98), respectively. The positive likelihood ratio was 4.66 (95% CI, 3.51–6.18) and the negative likelihood ratio was 0.41 (95% CI, 0.29–0.59).

Fig. 1. Classification tree fit based on emergency department point-of-care information available in patients presenting with sepsis, severe sepsis, or septic shock (N = 621). Overall, 56 patients were ultimately determined to have culture-proven sepsis with extended-spectrum β-lactamase–producing Enterobacterales (ESBL-PE) in blood, representing 9% of the sample.

The tree had an accuracy of 0.84 (95% CI, 0.81–0.87) and its calibration curve is presented in Supplementary Figure 4. The AUROCC for the tree was 0.77 (95% CI, 0.71–0.84), and the AUROCC for whether the patient was prescribed a carbapenem empirically by the ED clinician was 0.52 (95% CI, 0.48–0.55) (Supplementary Fig. 3). The difference between these 2 values was 0.26 (95% CI, 0.19–0.33; S value, 40.9). The AUROCC value for whether a patient was empirically prescribed a carbapenem by the admitting service was 0.70 (95% CI, 0.63–0.77). Hypothetically, using the tree in addition to the hospital-based clinicians’ recorded practice patterns would result in a combined AUROCC of 0.83. Sensitivity analyses revealed fitting a tree based on ESBL-PE balancing via equal prior probabilities would result in an identical tree as the one presented in this paper.

Discussion

In the presented study, the combination of a history for ESBL-PE colonization or infection and residence in a care facility had a NPV in the upper ninety percentiles in a sample with a low prevalence of ESBL. Although both variables have been independently associated with an increased risk of infection with ESBL-PE, Reference Goodman, Lessler and Cosgrove5,Reference Sousa, Pérez-Rodríguez and Suarez6,Reference Denis, Lafaurie and Donay8,Reference Flokas, Alevizakos, Shehadeh, Andreatos and Mylonakis9 we showed that application of a decision tree making use of these 2 point-of-care variables can help better identify patients and potentially guide therapeutic decisions. Furthermore, the decision-tree accuracy to identify individuals who would benefit from empiric carbapenem therapy was higher than that of ED clinicians’ gestalt alone. Also, the decision-tree accuracy was comparable to that of hospital-based clinicians’ gestalt but without full overlap. This latter result could be indicative of the impact of the local antibiotic stewardship program (ASP), which frequently engages with hospital-based clinicians using prospective review and feedback. It has been recognized that ASPs should work more closely and collaboratively with ED clinicians to improve antimicrobial use. Reference Pulia, Redwood and May10

This study had several limitations. The sample was taken from a medium-sized city with a relatively homogeneous population and low prevalence of ESBL-PE, which constrained the number of variables that could be examined in the tree-building process and limits the generalizability of our results. We created a classification tree and acquired its estimates based on a single data sample, which may result in an overly optimistic generalizability. This concern was partially lessened by cross validation in the tree-building process. Furthermore, stronger evidence in favor of carbapenems as the preferred agent in the treatment of ESBL-PE was published toward the end of the study period, which could have changed prescribing practices. Reference Harris, Tambyah and Lye3

A decision tree derived from variables available at point-of-care can help assess the need for empiric carbapenem use among patients with culture-proven sepsis due to Enterobacterales. Antibiotic stewardship programs should consider engaging ED clinicians when deploying strategies for improving antimicrobial use.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/ash.2021.253

Acknowledgments

Financial support

No financial support was provided relevant to this article.

Conflicts of interest

All authors report no conflicts of interest relevant to this article.

Footnotes

Current affiliation: Department of Quality and Patient Safety, Jackson Health System, Miami, Florida [R.R.]

References

Shamsrizi, P, Gladstone, BP, Carrara, E, et al. Variation of effect estimates in the analysis of mortality and length of hospital stay in patients with infections caused by bacteria-producing extended-spectrum beta-lactamases: a systematic review and meta-analysis. BMJ Open 2020;10(1):e030266.CrossRefGoogle ScholarPubMed
Rottier, WC, Ammerlaan, HSM, Bonten, MJM. Effects of confounders and intermediates on the association of bacteraemia caused by extended-spectrum β-lactamase–producing Enterobacteriaceae and patient outcome: a meta-analysis. J Antimicrob Chemother 2012;67:13111320.CrossRefGoogle ScholarPubMed
Harris, PNA, Tambyah, PA, Lye, DC, et al. Effect of piperacillin-tazobactam vs meropenem on 30-day mortality for patients with E. coli or Klebsiella pneumoniae bloodstream infection and ceftriaxone resistance: a randomized clinical trial. JAMA 2018;320(10):984994.CrossRefGoogle ScholarPubMed
van Loon, K, Voor in‘t Holt AF, Vos MC. A systematic review and meta-analyses of the clinical epidemiology of carbapenem-resistant Enterobacteriaceae. Antimicrob Agents Chemother 2018;62(1):e0173017.CrossRefGoogle Scholar
Goodman, KE, Lessler, J, Cosgrove, SE, et al. A clinical decision tree to predict whether a bacteremic patient is infected with an extended-spectrum β-lactamase–producing organism. Clin Infect Dis 2016;63:896903.CrossRefGoogle ScholarPubMed
Sousa, A, Pérez-Rodríguez, MT, Suarez, M, et al. Validation of a clinical decision tree to predict if a patient has a bacteraemia due to a β-lactamase–producing organism. Infect Dis 2019;51:3237.CrossRefGoogle Scholar
Dellinger, RP, Levy, MM, Rhodes, A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock. Crit Care Med 2013;41:580637.CrossRefGoogle ScholarPubMed
Denis, B, Lafaurie, M, Donay, JL, et al. Prevalence, risk factors, and impact on clinical outcome of extended-spectrum beta-lactamase–producing Escherichia coli bacteraemia: a five-year study. Int J Infect Dis 2015;39:16.CrossRefGoogle ScholarPubMed
Flokas, ME, Alevizakos, M, Shehadeh, F, Andreatos, N, Mylonakis, E. Extended-spectrum β-lactamase–producing Enterobacteriaceae colonisation in long-term care facilities: a systematic review and meta-analysis. Int J Antimicrob Agents 2017;50:649656.CrossRefGoogle ScholarPubMed
Pulia, M, Redwood, R, May, L. Antimicrobial stewardship in the emergency department. Emerg Med Clin N Am 2018;36:853872.CrossRefGoogle ScholarPubMed
Figure 0

Table 1. Emergency Department Point-of-Care Information Available in Patients Presenting With Sepsis, Severe Sepsis, or Septic Shock (N = 621)

Figure 1

Fig. 1. Classification tree fit based on emergency department point-of-care information available in patients presenting with sepsis, severe sepsis, or septic shock (N = 621). Overall, 56 patients were ultimately determined to have culture-proven sepsis with extended-spectrum β-lactamase–producing Enterobacterales (ESBL-PE) in blood, representing 9% of the sample.

Supplementary material: File

Smith et al. supplementary material

Smith et al. supplementary material

Download Smith et al. supplementary material(File)
File 178.2 KB