Hostname: page-component-77c89778f8-swr86 Total loading time: 0 Render date: 2024-07-18T00:00:16.007Z Has data issue: false hasContentIssue false

A comparison of National Healthcare Safety Network surveillance definitions to physician review for central-line–associated bloodstream infections

Published online by Cambridge University Press:  23 June 2023

Justin J. Kim*
Affiliation:
Section of Infectious Disease and International Health, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire Collaborative Healthcare-associated Infection Prevention Program, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire Geisel School of Medicine at Dartmouth College, Hanover, New Hampshire
HeeEun Kang
Affiliation:
Section of Infectious Disease and International Health, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire Geisel School of Medicine at Dartmouth College, Hanover, New Hampshire The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire
Asif N. Khan
Affiliation:
Section of Infectious Disease and International Health, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire Geisel School of Medicine at Dartmouth College, Hanover, New Hampshire
Rebecca Wang
Affiliation:
Section of Infectious Disease and International Health, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire Geisel School of Medicine at Dartmouth College, Hanover, New Hampshire
Stephanie C. Casale
Affiliation:
Collaborative Healthcare-associated Infection Prevention Program, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire
Caitlin M. Adams Barker
Affiliation:
Collaborative Healthcare-associated Infection Prevention Program, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire
Michael S. Calderwood
Affiliation:
Section of Infectious Disease and International Health, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire Geisel School of Medicine at Dartmouth College, Hanover, New Hampshire Quality Assurance and Safety, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire
Kathleen O. Stewart
Affiliation:
Collaborative Healthcare-associated Infection Prevention Program, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire
*
Corresponding author: Justin J. Kim; Email: justin.j.kim@hitchcock.org
Rights & Permissions [Opens in a new window]

Abstract

For 147 hospital-onset bloodstream infections, we assessed the sensitivity, specificity, positive predictive value, and negative predictive value of the National Healthcare Safety Network surveillance definitions of central-line–associated bloodstream infections against the gold standard of physician review, examining the drivers of discrepancies and related implications for reporting and infection prevention.

Type
Concise Communication
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America

A central-line–associated bloodstream infection (CLABSI) is a healthcare-associated infection resulting in increased morbidity, mortality, and healthcare costs. CLABSIs are strictly defined by surveillance rules of the Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN). 1 Although the NHSN surveillance definitions have been internally validated in multiple studies, Reference Bagchi, Watkins, Pollock, Edwards and Allen-Bridson2 few studies have assessed their external validity. Reference Advani, Murray, Murdzek, Aniskiewicz and Bizzarro3 We assessed the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the NHSN definitions against a gold standard of physician review of hospital-onset bloodstream infections (BSIs) with eligible central lines at our institution. We have described the drivers of any discrepancies between the surveillance and clinical designations as well as the implications of these discrepancies on the reporting of quality metrics and opportunities for infection prevention.

Material and methods

In total, 147 hospital-onset BSIs (ie, bacteremia occurring on hospital day 3 or later) with eligible central lines (ie, central line present for >2 calendar days after first being accessed, until the calendar day following line removal or patient discharge, whichever sooner) occurred at our 400-bed, rural, tertiary-care, academic medical center between July 2019 and June 2022. As part of routine hospital operations, all BSIs were investigated by at least 2 infection preventionists using NHSN surveillance definitions and were given a designation of CLABSI or bacteremia secondary to a primary infection. 1 Additional input was solicited from other infection preventionists on the team, the medical director of infection prevention, and via emails to the NHSN for difficult cases.

For comparison, physicians independently reviewed notes, laboratory studies, and imaging from the medical record to designate BSIs as CLABSI or secondary to a primary infection. Infectious disease fellows conducted a first review (H.K. and A.N.K.), and an infectious disease attending physician conducted a second review (J.J.K.) of all BSIs. Another infectious disease attending physician conducted a third review (R.W.) in cases of disagreement. We assessed interrater reliability between the first 2 reviews using the Cohen κ.

We calculated the sensitivity, specificity, PPV, and NPV of the NHSN definitions for CLABSIs compared to the gold standard of physician review. We examined discrepancies between the NHSN definitions and physician review. This study was designated “not human research” by the Dartmouth-Hitchcock Institutional Review Board.

Results

A confusion matrix summarizing our findings is provided in Figure 1. By physician review, 70 (48%) of 147 BSIs were CLABSIs, whereas 28 (19%) were secondary to an abdominal infection, 17 (12%) were pneumonia, 10 (7%) were urinary tract infection (UTI), and 10 (7%) were infective endocarditis. The remaining 12 BSIs were secondary to other sources (ie, 4 were secondary to skin and soft-tissue or surgical site infections and 2 were secondary to bone and joint infections), had an unclear source (n = 5), or contaminant (n = 1). By NHSN surveillance definitions, 63 (43%) of 147 BSIs were CLABSIs, whereas 22 (15%) were secondary to an abdominal infection, 30 (20%) were secondary to pneumonia, 10 (7%) were secondary to UTI, and 11 (7%) were secondary to infective endocarditis. The remaining 11 BSIs were secondary to other sources: 4 bone and joint infections, 4 skin and soft-tissue or surgical-site infections, 2 central nervous system infections, and 1 reproductive system infection. Compared against physician review, the NHSN definitions of CLABSI had sensitivity of 71%, specificity of 79%, PPV of 97%, and NPV of 76%. The NHSN definitions misclassified 20 (14%) of 147 BSIs as being secondary to a primary infection instead of a CLABSI (ie, false negatives), whereas the NHSN definitions misclassified 13 (9%) of 147 BSIs as being a CLABSIs instead of secondary to a primary infection (ie, false positives). The reasons for these discrepancies are summarized in Table 1.

Figure 1. Confusion matrix comparing NHSN CLABSI surveillance definitions to physician review. The bolded center box draws attention to a 2×2 table where true positives are highlighted with medium-gray shading. True negatives are white with the underlined diagonal demonstrating agreement on secondary infection source. False positives are dark gray and false negatives are light gray. Blank cells have a count of zero. Note. CLABSI, central-line–associated bloodstream infection; NHSN, National Healthcare Safety Network; Abd, abdominal infection; PNA, pneumonia; UTI, urinary tract infection; IE, infective endocarditis; other, skin and soft tissue, surgical site, bone and joint, central nervous system, reproductive system, unclear source, and contaminants.

Table 1. Discrepancies Between NHSN CLABSI Surveillance Definitions and Physician Review

Note. NHSN, National Healthcare Safety Network; CLABSI, central-line–associated bloodstream infection; BSI, bloodstream infection; UTI, urinary tract infection.

Interrater reliability was high between the first and second physician reviews, with agreement on 138 (94%) designations (κ = 0.91). The first and second physician reviews disagreed most frequently over whether a BSI was a CLABSI (n = 6) versus secondary to a primary infection (ie, 2 abdominal infections, 2 pneumonia cases, and 2 UTIs).

Discussion

Our findings suggest that the NHSN surveillance definitions for CLABSI have moderately high sensitivity, specificity, PPV, and NPV compared to the gold standard of physician review, in part because the degree of underreporting (ie, 14% false negatives) was offset by a comparable degree of overreporting (ie, 9% false positives). Underreported cases are not as closely scrutinized in the daily work of infection prevention and represent potentially missed opportunities for the prevention of CLABSIs. Thus, a new quality metric is currently being considered to include all hospital acquired bacteremia, regardless of the presence of a central line. Reference Dantes, Abbo and Anderson4 A major driver of underreporting was the misclassification of BSIs as being secondary to pneumonia, particularly in cases in which pulmonary imaging was the only evidence supporting a diagnosis of pneumonia, accounting for 6 (30%) of 20 of false negatives. However, there are other reasons for chest imaging to be abnormal in hospitalized patients (eg, acute heart failure), and a matching organism in a respiratory specimen could support the attribution of a BSI to pneumonia in the correct clinical context. Reference Zhang, Yang and Makam5 Additionally, 10 (50%) of 20 false negatives had Staphylococcus aureus bacteremia, suggesting that clinicians typically attribute S. aureus (ie, a skin commensal that frequently colonizes catheters) to CLABSI, whereas the NHSN definition may misclassify these BSIs as being secondary to other infections. Reference Hindy, Quintero-Martinez and Lee6

Even though overreported cases receive similar attention as appropriately reported cases, overreporting is a dissatisfier for frontline healthcare providers whose buy-in is essential for the implementation of key infection prevention practices. The major drivers of overreporting were the misclassification of abdominal infections, pneumonia, and UTIs as NHSN CLABSIs. In clinical practice, BSIs are commonly attributed to translocation in the setting of distension from obstruction (eg, of the colon or common bile duct) or mucosal injury from inflammation (eg, from a gastrointestinal bleed or colitis) in immunocompetent hosts. However, mucosal barrier injury can only be invoked for immunocompromised patients with neutropenia or graft-versus-host disease in the current NHSN definitions. Reference Deitch7 Although stool and tissue cultures are not often obtained in clinical practice, select imaging studies could be used as objective evidence for BSIs from mucosal barrier injury organisms, as proposed in another study. Reference Advani, Murray, Murdzek, Aniskiewicz and Bizzarro3 For pneumonia, BSIs are commonly attributed to a respiratory source using lower thresholds of growth of a matching organism from respiratory cultures (eg, <103 colony-forming units per milliliter, few, or 1+), particularly in critically ill patients who may be receiving empiric antibiotics before bronchoscopy. Reference Zhang, Yang and Makam5 For UTIs, urinary cultures are frequently used to support a diagnosis of pyelonephritis without imaging. Reference Gupta, Grigoryan and Trautner8,Reference Lacy, Sidhu and Miller9 Moreover, pyelonephritis is not infrequently accompanied by bacteremia, and blood and urine cultures are rarely discordant. Reference Velasco, Martínez and Moreno-Martínez10 Even in the absence of typical symptoms, a positive urine culture might warrant treatment in the presence of neutropenia, chronic obstruction from malignancy, or undifferentiated critical illness.

To our knowledge, this study is one of the few external validation studies of the NHSN surveillance definitions for CLABSI. Reference Advani, Murray, Murdzek, Aniskiewicz and Bizzarro3 The major strength of this study is that both the NHSN and physician review were conducted by multiple individuals, which strengthens the internal validity of the study. However, we were limited by relatively small numbers from a single center and the potentially subjective interpretation of cases by physician review, despite being the gold standard. Larger, multicenter studies could confirm our findings and inform future iterations of the NHSN surveillance definitions. Additionally, we would strongly encourage similar validation procedures of future quality metrics such as hospital-onset bacteremia to support their clinical relevance.

Acknowledgments

Financial support

No financial support was provided relevant to this article.

Competing interests

All authors report no conflicts of interest relevant to this article.

References

National Healthcare Safety Network. Patient safety component (PSC). Centers for Disease Control and Prevention website. https://www.cdc.gov/nhsn/psc/index.html. Published December 29, 2022. Accessed March 1, 2023.Google Scholar
Bagchi, S, Watkins, J, Pollock, DA, Edwards, JR, Allen-Bridson, K. State health department validations of central-line–associated bloodstream infection events reported via the National Healthcare Safety Network. Am J Infect Control 2018;46:12901295.10.1016/j.ajic.2018.04.233CrossRefGoogle ScholarPubMed
Advani, SD, Murray, TS, Murdzek, CM, Aniskiewicz, MJ, Bizzarro, MJ. Shifting focus toward healthcare-associated bloodstream infections: the need for neonatal intensive care unit-specific NHSN definitions. Infect Control Hosp Epidemiol 2020;41:181186.Google ScholarPubMed
Dantes, RB, Abbo, LM, Anderson, D, et al. Hospital epidemiologists’ and infection preventionists’ opinions regarding hospital-onset bacteremia and fungemia as a potential healthcare-associated infection metric. Infect Control Hosp Epidemiol 2019;40:536540.10.1017/ice.2019.40CrossRefGoogle ScholarPubMed
Zhang, D, Yang, D, Makam, AN. Utility of blood cultures in pneumonia. Am J Med 2019;132:12331238.10.1016/j.amjmed.2019.03.025CrossRefGoogle ScholarPubMed
Hindy, JR, Quintero-Martinez, JA, Lee, AT, et al. Incidence trends and epidemiology of Staphylococcus aureus bacteremia: a systematic review of population-based studies. Cureus 2022;14:e25460.Google ScholarPubMed
Deitch, EA. Gut-origin sepsis: evolution of a concept. Surgeon 2012;10:350356.10.1016/j.surge.2012.03.003CrossRefGoogle ScholarPubMed
Gupta, K, Grigoryan, L, Trautner, B. Urinary tract infection. Ann Intern Med 2017;167:ITC49ITC64.10.7326/AITC201710030CrossRefGoogle ScholarPubMed
Lacy, ME, Sidhu, N, Miller, J. When does acute pyelonephritis require imaging? Cleve Clin J Med 2019;86:515517.10.3949/ccjm.86a.18096CrossRefGoogle ScholarPubMed
Velasco, M, Martínez, JA, Moreno-Martínez, A, et al. Blood cultures for women with uncomplicated acute pyelonephritis: are they necessary? Clin Infect Dis 2003;37:11271130.10.1086/378291CrossRefGoogle ScholarPubMed
Figure 0

Figure 1. Confusion matrix comparing NHSN CLABSI surveillance definitions to physician review. The bolded center box draws attention to a 2×2 table where true positives are highlighted with medium-gray shading. True negatives are white with the underlined diagonal demonstrating agreement on secondary infection source. False positives are dark gray and false negatives are light gray. Blank cells have a count of zero. Note. CLABSI, central-line–associated bloodstream infection; NHSN, National Healthcare Safety Network; Abd, abdominal infection; PNA, pneumonia; UTI, urinary tract infection; IE, infective endocarditis; other, skin and soft tissue, surgical site, bone and joint, central nervous system, reproductive system, unclear source, and contaminants.

Figure 1

Table 1. Discrepancies Between NHSN CLABSI Surveillance Definitions and Physician Review