Skip to main content Accessibility help
×
Home

Multicenter Evaluation of Computer Automated versus Traditional Surveillance of Hospital-Acquired Bloodstream Infections

  • Michael Y. Lin (a1), Keith F. Woeltje (a2), Yosef M. Khan (a3) (a4), Bala Hota (a1) (a5), Joshua A. Doherty (a2), Tara B. Borlawsky (a3), Kurt B. Stevenson (a3), Scott K. Fridkin (a6), Robert A. Weinstein (a1) (a5) and William E. Trick (a1) (a5)...

Abstract

Objective.

Central line–associated bloodstream infection (BSI) rates are a key quality metric for comparing hospital quality and safety. Traditional BSI surveillance may be limited by interrater variability. We assessed whether a computer-automated method of central line–associated BSI detection can improve the validity of surveillance.

Design.

Retrospective cohort study.

Setting.

Eight medical and surgical intensive care units (ICUs) in 4 academic medical centers.

Methods.

Traditional surveillance (by hospital staff) and computer algorithm surveillance were each compared against a retrospective audit review using a random sample of blood culture episodes during the period 2004–2007 from which an organism was recovered. Episode-level agreement with audit review was measured with κ statistics, and differences were assessed using the test of equal κ coefficients. Linear regression was used to assess the relationship between surveillance performance (κ) and surveillance-reported BSI rates (BSIs per 1,000 central line–days).

Results.

We evaluated 664 blood culture episodes. Agreement with audit review was significantly lower for traditional surveillance (κ [95% confidence interval (CI)] = 0.44 [0.37–0.51]) than computer algorithm surveillance (κ [95% CI] [0.52–0.64]; P = .001). Agreement between traditional surveillance and audit review was heterogeneous across ICUs (P = .001); furthermore, traditional surveillance performed worse among ICUs reporting lower (better) BSI rates (P = .001). In contrast, computer algorithm performance was consistent across ICUs and across the range of computer-reported central line–associated BSI rates.

Conclusions.

Compared with traditional surveillance of bloodstream infections, computer automated surveillance improves accuracy and reliability, making interfacility performance comparisons more valid.

Infect Control Hosp Epidemiol 2014;35(12):1483–1490

Copyright

Corresponding author

600 South Paulina Street, Suite 143, Chicago, IL 60612 (michael_lin@rush.edu).

References

Hide All
1. Pronovost, P, Needham, D, Berenholtz, S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355(26):27252732.
2. Klevens, RM, Edwards, JR, Richards, CL Jr, et al. Estimating health care-associated infections and deaths in U.S. hospitals, 2002. Public Health Rep 2007;122(2):160166.
3. Kahn, KL, Weinberg, DA, Leuschner, KJ, Gall, EM, Siegel, S, Mendel, P. The national response for preventing healthcare-associated infections: data and monitoring. Med Care 2014;52(2 suppl 1):S25S32.
4. Centers for Medicare and Medicaid Services (CMS) Media Relations (press release). CMS gives consumers access to more details about infection rates at America’s hospitals. Baltimore, MD: CMS, 2012. http://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2012-Press-releases-items/2012-02-07.html. Accessed February 25, 2014.
5. Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and fiscal year 2014 rates; quality reporting requirements for specific providers; hospital conditions of participation; payment policies related to patient status. Fed Regist 2013;78:50495.
6. Centers for Disease Control and Prevention. National Healthcare Safety Network (NHSN) Manual, Patient Safety Component Protocol. Atlanta, GA: Division of Healthcare Quality Promotion, Centers for Disease Control and Prevention, 2008:13.
7. Mayer, J, Greene, T, Howell, J, et al. Agreement in classifying bloodstream infections among multiple reviewers conducting surveillance. Clin Infect Dis 2012;55(3):364370.
8. McBryde, ES, Brett, J, Russo, PL, Worth, LJ, Bull, AL, Richards, MJ. Validation of statewide surveillance system data on central line–associated bloodstream infection in intensive care units in Australia. Infect Control Hosp Epidemiol 2009;30(11):10451049.
9. Lin, MY, Hota, B, Khan, YM, et al. Quality of traditional surveillance for public reporting of nosocomial bloodstream infection rates. JAMA 2010;304(18):20352041.
10. Centers for Disease Control and Prevention. National Healthcare Safety Network (NHSN) validation guidance and toolkit 2012: validation for central line–associated bloodstream infection (CLABSI) in ICUs. http://www.cdc.gov/nhsn/toolkit/validation-clabsi/index.html. Accessed February 25, 2014.
11. Hota, B, Lin, M, Doherty, JA, et al. Formulation of a model for automating infection surveillance: algorithmic detection of central-line associated bloodstream infection. J Am Med Inform Assoc 2010;17(1):4248.
12. Rubin, MA, Mayer, J, Greene, T, et al. An agent-based model for evaluating surveillance methods for catheter-related bloodstream infection. AMIA Annu Symp Proc 2008:631635.
13. Trick, WE, Zagorski, BM, Tokars, JI, et al. Computer algorithms to detect bloodstream infections. Emerg Infect Dis 2004;10(9):16121620.
14. Worth, LJ, Brett, J, Bull, AL, McBryde, ES, Russo, PL, Richards, MJ. Impact of revising the National Nosocomial Infection Surveillance System definition for catheter-related bloodstream infection in ICU: reproducibility of the National Healthcare Safety Network case definition in an Australian cohort of infection control professionals. Am J Infect Control 2009;37(8):643648.
15. Stone, PW, Dick, A, Pogorzelska, M, Horan, TC, Furuya, EY, Larson, E. Staffing and structure of infection prevention and control programs. Am J Infect Control 2009;37(5):351357.
16. Dixon-Woods, M, Leslie, M, Bion, J, Tarrant, C. What counts? an ethnographic study of infection data reported to a patient safety program. Milbank Q 2012;90(3):548591.
17. Fraser, TG, Gordon, SM. CLABSI rates in immunocompromised patients: a valuable patient centered outcome? Clin Infect Dis 2011;52(12):14461450.
18. Backman, LA, Melchreit, R, Rodriguez, R. Validation of the surveillance and reporting of central line–associated bloodstream infection data to a state health department. Am J Infect Control 2010;38(10):832838.
19. Sexton, DJ, Chen, LF, Anderson, DJ. Current definitions of central line–associated bloodstream infection: is the emperor wearing clothes? Infect Control Hosp Epidemiol 2010;31(12):12861289.
20. Fridkin, SK, Olmsted, RN. Meaningful measure of performance: a foundation built on valid, reproducible findings from surveillance of health care-associated infections. Am J Infect Control 2011;39(2):8790.
21. Kainer, M, Mitchell, J, Frost, B, Soe, M. Validation of central line associated bloodstream Infection data submitted to the National Healthcare Safety Network—a pilot study by the Tennessee Department of Health. In: Program and abstracts of Fifth Decennial International Conference on Healthcare-Associated Infections. Atlanta, GA: Centers for Disease Control and Prevention, 2010.
22. Woeltje, K, McMullen, K, Butler, AM, Goris, A, Doherty, J. Electronic surveillance for healthcare-associated central line–associated bloodstream infections outside the intensive care unit. Infect Control Hospital Epidemiol 2011;32(11):10861090.
23. Grove, WM, Zald, DH, Lebow, BS, Snitz, BE, Nelson, C. Clinical versus mechanical prediction: a meta-analysis. Psychol Assess 2000;12:1930.
24. Wright, MO, Fisher, A, John, M, Reynolds, K, Peterson, LR, Robicsek, A. The electronic medical record as a tool for infection surveillance: successful automation of device-days. Am J Infect Control 2009;37(5):364370.
25. National Healthcare Safety Network. Central line–associated bloodstream infection (CLABSI) event. 2013. http://www.cdc.gov/nhsn/PDFs/pscManual/4PSC_CLABScurrent.pdf. Accessed May 6, 2013.

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed