Hostname: page-component-8448b6f56d-mp689 Total loading time: 0 Render date: 2024-04-23T17:48:01.929Z Has data issue: false hasContentIssue false
  • English
  • Français

Lost in Translation? Reliability of Assessing Inpatient Antimicrobial Appropriateness With Use of Computerized Case Vignettes

Published online by Cambridge University Press:  02 January 2015

David N. Schwartz ,
Ulysses S. Wu ,
Rosie D. Lyles ,
Yingxu Xiang ,
Piotr Kieszkowski ,
Bala Hota  and
Robert A. Weinstein
David N. Schwartz*
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois Rush Medical College, Chicago, Illinois
Ulysses S. Wu
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois Rush Medical College, Chicago, Illinois
Rosie D. Lyles
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois
Yingxu Xiang
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois
Piotr Kieszkowski
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois
Bala Hota
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois Rush Medical College, Chicago, Illinois
Robert A. Weinstein
Affiliation:
Stroger Hospital of Cook County, Chicago, Illinois Rush Medical College, Chicago, Illinois
*
Division of Infectious Diseases, Stroger Hospital of Cook County, 1901 W. Harrison St., Chicago, IL 60612 (david.schwartz@hektoen.org)
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective.

To describe and measure reliability of a computer-assisted method of case vignette assembly and expert review to assess the appropriateness of antimicrobial therapy for hospitalized adults.

Design.

Feasibility and reliability analysis of computer-assisted tool used to compare the effects of antimicrobial stewardship interventions.

Setting.

Public teaching hospital.

Patients.

Randomly selected adult antimicrobial recipients admitted to inpatient medicine services.

Methods.

Clinical data abstracted from 504 paper medical records were merged with computerized laboratory and pharmacy data to assemble case vignettes that underwent expert review for appropriateness. We performed 3 validations, as follows: data for 35 vignettes abstracted independently by 2 research assistants were assessed for interrater agreement, expert review of 24 vignettes was compared with review of the corresponding paper medical records, and interrater reliability of antimicrobial appropriateness assessments by 2 experts was determined for 70 case vignettes.

Results.

Vignette assembly and expert review each required 10–12 minutes per case. Potentially important discrepancies occurred in 0%–32% of clinical findings abstracted independently by 2 research assistants. Expert review of 24 vignettes and the corresponding full paper medical records yielded fair agreement (kappa, 0.30). The 2 experts identified inappropriate initial antimicrobial therapy in 67% and 61% of case vignettes reviewed independently; interrater agreement was improved after sequential case discussion and stringent application of appropriateness criteria (kappa, 0.72).

Conclusions.

Our case vignette assembly and expert review method is efficient, but improvements in both technical and human performance are needed to be able to yield valid estimates of the prevalence of inappropriate antimicrobial use. Assessments of antimicrobial appropriateness require validation.

Type
Original Articles
Information
Infection Control & Hospital Epidemiology , Volume 30 , Issue 2 , February 2009 , pp. 163 - 171
Copyright
Copyright © The Society for Healthcare Epidemiology of America 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Schwartz, DN, Abiad, H, DeMarais, PL, et al.An educational intervention to improve antimicrobial use in a hospital-based long-term care facility. ; Am Geriatr Soc 2007;55:1236–1242.CrossRefGoogle Scholar
2.Gyssens, IC, Blok, WL, van den Broek, PJ, Hekster, YA, van der Meer, JWM. Implementation of an educational program and an antibiotic order form to optimize quality of antimicrobial drug use in a department of internal medicine. Eur J Clin Microbiol Infect Dis 1997;16:904–912.Google Scholar
3.Spinewine, A, Schmader, KE, Barber, N, et al.Appropriate prescribing in elderly people: how well can it be measured and optimised? Lancet 2007;370:173–184.Google Scholar
4.Henriksen, K, Kaplan, H. Hindsight bias, outcome knowledge and adaptive learning. Qual Saf Health Care 2003;12(Suppl 2):46–50.Google Scholar
5.Hanlon, JT, Schmader, KE, Samsa, GP, et al.A method for assessing drug therapy appropriateness. J Clin Epidemiol 1992;45:1045–1051.Google Scholar
6.Glowacki, RC, Itokazu, GS, Schwartz, DN, et al. Computer-assisted antimicrobial review to optimize therapy (CAROT): results of pharmacists' interventions. In: Proceedings of the 16th Annual Meeting of the Society for Healthcare Epidemiology of America; March 16-21, 2006; Chicago, IL. Abstract 27.Google Scholar
7.Hota, B, Schwartz, DN, Glowacki, RC, et al. Computer-assisted antimicrobial review to optimize therapy (CAROT): a randomized trial of antimicrobial stewardship programs. In: Proceedings of the 16th Annual Meeting of the Society for Healthcare Epidemiology of America; March 16-21, 2006; Chicago, IL. Abstract 317.Google Scholar
8.Schwartz, DN, Lyles, RD, Wu, US, et al. Computer-assisted antimicrobial recommendations for optimal therapy (CAROT): analysis of prescribing errors in a randomized trial of antimicrobial stewardship programs. In: Proceedings of the 17th Annual Meeting of the Society for Healthcare Epidemiology of America; April 14-17, 2007; Baltimore, MD. Abstract 298.Google Scholar
9.Wisniewski, MF, Kieszkowski, P, Zagorski, BM, et al.Development of a clinical data warehouse for hospital infection control. J Am Med Inform Assoc 2003;10:454–462.Google Scholar
10.Warren, JW, Abrutyn, E, Hebel, JR, Johnson, JR, Schaeffer, AJ, Stamm, WE. Guidelines for antimicrobial treatment of uncomplicated acute bacterial cystitis and acute pyelonephritis in women. Clin Infect Dis 1999;29:745–758.CrossRefGoogle ScholarPubMed
11.Mandell, LA, Bartlett, JG, Dowell, SF, File, TM, Musher, DM, Whitney, C. Update of practice guidelines for the management of community-acquired pneumonia in immunocompetent adults. Clin Infect Dis 2003;37:1405–1433.Google Scholar
12.Mills, GD, Oehley, MR, Arrol, B. Effectiveness of β lactam antibiotics compared with antibiotics active against atypical pathogens in non-severe community acquired pneumonia: meta-analysis. BMJ 2005;330:456.Google Scholar
13.Shefet, D, Robenshtok, E, Paul, M, Leibovici, L. Empirical atypical coverage for inpatients with community-acquired pneumonia: systematic review of randomized controlled trials. Arch Intern Med 2005;165:1992–2000.CrossRefGoogle ScholarPubMed
14.Gyssens, IC, van den Broek, PJ, Kullberg, B-J, Hekster, YA, van der Meer, JWM. Optimizing antimicrobial therapy: a method for antimicrobial drug use evaluation. J Antimicrob Chemother 1992;30:724–727.Google Scholar
15.Mol, PGM, Gans, ROB, Nannan Panday, PV, Degener, JE, Laseur, M, Haaijer-Ruskamp, FM. Reliability of assessment of adherence to an antimicrobial treatment guideline. J Hosp Infect 2005;60:321–328.Google Scholar
16.Taylor, CT, Stewart, LM, Byrd, DC, Church, CO. Reliability of an instrument for evaluating antimicrobial appropriateness in hospitalized patients. Am J Health Syst Pharm 2001;58:242–246.CrossRefGoogle ScholarPubMed
17.Linkin, DR, Fishman, NO, Landis, JR, et al.The effect of communication errors during calls to an antimicrobial stewardship program. Infect Control Hosp Epidemiol 2007;28:1374–1381.Google Scholar
18.Schouten, JA, Hulscher, MEJL, Wollersheim, H, et al.Quality of antibiotic use for lower respiratory tract infections at hospitals: (how) can we measure it? Clin Infect Dis 2005;41:450–460.Google Scholar
19.Chen, LM, Martin, CM, Morrison, TL, Sibbald, WJ. Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. Crit Care Med 1999;27:1999–2004.Google Scholar
20.Itokazu, GS, Glowacki, RC, Schwartz, DN, Wisniewski, MF, Rydman, RJ, Weinstein, RA. Antimicrobial consumption data from pharmacy and nursing records: how good are they? Infect Control Hosp Epidemiol 2005;26:395–400.CrossRefGoogle Scholar