Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-12-02T09:07:41.880Z Has data issue: false hasContentIssue false

Cost-effectiveness analysis of whole-genome sequencing during an outbreak of carbapenem-resistant Acinetobacter baumannii

Published online by Cambridge University Press:  13 December 2021

Thomas M. Elliott*
Affiliation:
QIMR Berghofer Medical Research Institute, Herston, Brisbane, Queensland, Australia
Patrick N. Harris
Affiliation:
The University of Queensland, Centre for Clinical Research, Herston, Brisbane, Queensland, Australia Central Microbiology, Pathology Queensland, Royal Brisbane and Women’s Hospital, Herston, Brisbane, Queensland, Australia
Leah W. Roberts
Affiliation:
School of Chemistry and Molecular Biosciences, University of Queensland, Brisbane, Queensland, Australia European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, United Kingdom
Michelle Doidge
Affiliation:
Infectious Diseases Unit, Royal Brisbane and Womens’ Hospital, Herston, Brisbane, Queensland, Australia
Trish Hurst
Affiliation:
Infectious Diseases Unit, Royal Brisbane and Womens’ Hospital, Herston, Brisbane, Queensland, Australia
Krispin Hajkowicz
Affiliation:
Central Microbiology, Pathology Queensland, Royal Brisbane and Women’s Hospital, Herston, Brisbane, Queensland, Australia Infectious Diseases Unit, Royal Brisbane and Womens’ Hospital, Herston, Brisbane, Queensland, Australia
Brian Forde
Affiliation:
The University of Queensland, Centre for Clinical Research, Herston, Brisbane, Queensland, Australia School of Chemistry and Molecular Biosciences, University of Queensland, Brisbane, Queensland, Australia
David L. Paterson
Affiliation:
The University of Queensland, Centre for Clinical Research, Herston, Brisbane, Queensland, Australia
Louisa G. Gordon
Affiliation:
QIMR Berghofer Medical Research Institute, Herston, Brisbane, Queensland, Australia The University of Queensland, School of Public Health, Brisbane, Queensland, Australia Queensland University of Technology, School of Nursing, Kelvin Grove, Brisbane, Queensland, Australia
*
Author for correspondence: Thomas Elliott, QIMR Berghofer Medical Research Institute, Population Health, 300 Herston Rd, Herston Q4006, Brisbane, Australia. E-mail: thomas.elliott@qimrberghofer.edu.au

Abstract

Background:

Whole-genome sequencing (WGS) shotgun metagenomics (metagenomics) attempts to sequence the entire genetic content straight from the sample. Diagnostic advantages lie in the ability to detect unsuspected, uncultivatable, or very slow-growing organisms.

Objective:

To evaluate the clinical and economic effects of using WGS and metagenomics for outbreak management in a large metropolitan hospital.

Design:

Cost-effectiveness study.

Setting:

Intensive care unit and burn unit of large metropolitan hospital.

Patients:

Simulated intensive care unit and burn unit patients.

Methods:

We built a complex simulation model to estimate pathogen transmission, associated hospital costs, and quality-adjusted life years (QALYs) during a 32-month outbreak of carbapenem-resistant Acinetobacter baumannii (CRAB). Model parameters were determined using microbiology surveillance data, genome sequencing results, hospital admission databases, and local clinical knowledge. The model was calibrated to the actual pathogen spread within the intensive care unit and burn unit (scenario 1) and compared with early use of WGS (scenario 2) and early use of WGS and metagenomics (scenario 3) to determine their respective cost-effectiveness. Sensitivity analyses were performed to address model uncertainty.

Results:

On average compared with scenario 1, scenario 2 resulted in 14 fewer patients with CRAB, 59 additional QALYs, and $75,099 cost savings. Scenario 3, compared with scenario 1, resulted in 18 fewer patients with CRAB, 74 additional QALYs, and $93,822 in hospital cost savings. The likelihoods that scenario 2 and scenario 3 were cost-effective were 57% and 60%, respectively.

Conclusions:

The use of WGS and metagenomics in infection control processes were predicted to produce favorable economic and clinical outcomes.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction in any medium, provided the original article is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America

Acinetobacter baumannii is an opportunistic pathogen commonly associated with bacteremia, pneumonia, and urinary tract infections in patients admitted to intensive care units (ICUs) and burn units. 1 The persistence of A. baumannii in hospital environments, and the ability of the organism to readily acquire or develop resistance to a wide range of antibiotics, has led to high rates of carbapenem-resistant A. baumannii (CRAB), a frequent cause of hospital outbreaks, Reference Venditti, Vulcano and D’Arezzo2 with rates reportedly 2- to 5-fold higher in ICUs. Reference Rossi, Royer and Ferreira3 Aqueous reservoirs, such as drains, sinks and toilets, have been identified as a pathway for spreading A. baumannii to patients. Reference Umezawa, Asai and Ohshima4 Approximately 5%–30% of surfaces can remain contaminated because existing detergent and disinfectant formulations cannot disrupt biofilms. Reference Vickery, Deva, Jacombs, Allan, Valente and Gosbell5 An enhanced focus on cleaning practices have been shown to be effective at reducing contamination and are cost-effective. Reference White, Barnett and Hall6 Identifying environmental contamination early in outbreaks is critical to limiting their spread.

Hospital infection control teams use microbiological screening to identify pathogens, their susceptibility in patients and for environmental screening. This process takes 1–3 days for bacteria that are cultured on routine media, with antimicrobial susceptibility testing adding 1–2 days. Reference Chiang and Dekker7 Whole-genome sequencing (WGS) has higher precision compared with conventional typing methods and is used to detect outbreaks, Reference McGann, Bunin and Snesrud8 monitor the evolution of drug resistance, Reference Matsumura9 and reconstruct transmission routes. Reference Fujikura, Hamamoto, Kanayama, Kaku, Yamagishi and Kawana10 WGS requires the sample to be cultured beforehand, whereas WGS shotgun metagenomics (metagenomics) attempts to sequence the entire genetic content straight from the sample. Metagenomics can characterize the subtypes, antimicrobial resistance, and pathogenic gene carriage of the microbial population. Reference Forbes, Knox, Peterson and Reimer11 Turnaround times for metagenomics range from 2 to 7 days from sample collection to results. Reference Simner, Miller and Carroll12 Because time to diagnosis is not always shortened, the diagnostic advantages lie in the ability to detect unsuspected, uncultivatable, or very slow-growing organisms, which produce negative results with standard assays. Reference Chiang and Dekker7 The overwhelming amount of host DNA present in primary clinical specimens collected from patients poses a major challenge in metagenomics, but this is not an issue when it is used on environmental samples. Reference Chiang and Dekker7

The importance of WGS in limiting multidrug-resistant organism (MDRO) nosocomial infections has been identified in outbreak Reference Elliott, Lee, Foeglein, Harris and Gordon13 and nonoutbreak settings. Reference Elliott, Hare and Hajkowicz14 Significant cost savings could be realized as the reduction in cases, and subsequent resources used for contact precautions, outweigh the additional cost of sequencing, Reference Elliott, Lee, Foeglein, Harris and Gordon13Reference Kumar, Sundermann and Martin15 but more evidence on the economic impact of WGS within infection control is needed. To our knowledge, the economic impact of metagenomics has not been studied, specifically in environmental contamination screening.

To address this gap, we performed a cost-effectiveness analysis of WGS and metagenomics of environmental samples to inform future resource use of WGS based on a hospital CRAB outbreak.

Methods

In the intensive care and burn units of a large metropolitan hospital in Brisbane, Australia, we used a hybrid agent-based and discrete-event simulation model to assess the management of a CRAB outbreak, and we incorporated background MDRO nosocomial infections and environmental screening. This study was approved by the QIMR Berghofer Medical Research Institute Human Research Ethics Committee (P2353) and the Queensland Government Public Health Act Human Research Ethics Committee (RD007427).

The outbreak

The hospital is a 978-bed, tertiary-care facility in Queensland, Australia, comprised of open-ward, 4-bed, 2-bed, and single-bed accommodation combinations. The outbreak predominantly affected a 34-bed ICU and an 18-bed burn unit over a 32-month period. Active transmission of 17 cases of sequence type (ST) 1050 (ST1050) CRAB were identified between May and August 2016; 6 cases were identified between December 2016 and August 2017; and 8 cases were identified between May and August 2018 (Fig. 1). Reference Roberts, Forde and Hurst16 Monthly WGS reporting starting in June 2016 identified environmental contamination as the likely source of the ongoing outbreak. Environmental metagenomics was introduced in November 2017 but had reduced sensitivity due to low DNA yields from sampling. Instead, areas of high bacterial load, such as drains and burn baths, were targeted in 2018, which revealed 4 areas positive for CRAB. Immediate reporting of WGS results were available when the CRAB outbreak resurfaced in May 2018. Additional details about the outbreak, implementation of metagenomics, environmental swabbing, and sequencing outcomes are discussed in the Supplementary Materials and by Roberts et al. Reference Roberts, Forde and Hurst16

Fig. 1. Outbreak timeline with investigation strategy changes. Each colored box represents a distinct outbreak of ST1050 CRAB. The red box consisted of 17 cases, the green box represents 8 cases and the orange box represents 11 cases. Each black arrow indicated initiation of environmental screening. To improve specimen collection the focus of environmental swabbing changed from high-touch areas to high bacterial load areas in July 2018. High-touch areas were defined as places commonly touched such as nurse keyboards, trolleys and door handles. High bacterial-load areas were defined as areas of high biomass such as floor drains, plumbing and inside burns bath drains. WGS was implemented as part of outbreak control in May 2018 and metagenomics in November of 2017. Note. WGS, whole-genome sequencing.

Comparison groups

We evaluated the cost-effectiveness of 3 screening scenarios as follows:

  • Scenario 1: The observed outbreak described above with initially no WGS, no metagenomics and environmental swabbing concentrating on high touch areas. WGS, metagenomics and environmental swabbing of high load areas were introduced later as described above (Fig. 1).

  • Scenario 2: Hypothetically testing immediate WGS use prior to the start of the outbreak. This would lead to identifying the need for environmental swabbing to focus on high bacterial load areas. As in Scenario 1, metagenomics was introduced in November 2017.

  • Scenario 3: Hypothetically testing WGS and metagenomics use prior to the start of the outbreak.

Model structure

Using AnyLogic dynamic simulation modeling software (AnyLogic, Chicago, IL), the evaluation combined methods of cost-effectiveness analysis, Reference Briggs, Weinstein, Fenwick, Karnon, Sculpher and Paltiel17 infectious disease modeling and system dynamics. Reference Pethes, Ferenci and Kovács18 A network approach was taken to connect the hospital environment, the dynamics of pathogen transmission, environmental pathogen contamination, patient movements, and decisions by the hospital infection control team. The intensive care and burns units were modeled; 29 of the 31 ST1050 CRAB cases were detected in these units. The model ran for 32 months (January 4, 2016, through January 1, 2019) in hourly units. The spread of CRAB was calibrated to outbreak data to obtain both patient and environmental transmission rates (see Supplementary Material, section ST1050 CRAB spread’). The spread of other MDROs through the intensive care and burns units were based on surveillance data between April 2016 and January 2019. The 3 main interacting components in the model were patient flow dynamics, pathogen transmission dynamics, and outbreak control team decisions. Full model details are provided in the Supplementary Materials (detailed description of model structure).

Pathogen transmission dynamics were modeled daily at the ward level through patient-to-patient transmission and contaminated room-to-patient transmission using a discrete-time event model. The daily probability a susceptible patient was colonised/infected was calculated using the frequency-dependent transmission formula ( $$1 - \exp \left\{ { - \beta CN} \right\}$$ ).19 We modeled new daily colonization and/or infections with the binomial distribution formula ( ${}_N^\;C{\;_X}*\;{P^X}{\left( {1 - P} \right)^{\left( {n - x} \right)}}$ ), where x is the number of transmissions (limited to 3), n is the number of susceptible patients, and P is the probability. Daily incident probabilities of other MDROs detected were included, and information on cluster spread was derived from the WGS results (Supplementary Table S2). Decisions by the infection control team, like outbreak designation and initiating environmental screening, were initially based on microbiological cultures before transitioning to WGS and metagenomics information (Fig. 2).

Fig. 2. Introduction of WGS and metagenomics into microbiology culture infection control process. Note. micro, microbiology; enviro, environmental; WGS, whole-genome sequencing; HTA, high-touch area.

Fig. 3. Scatterplot of incremental costs and QALYs (all patients) for scenario 3 versus scenario 1. Each dot represents an incremental cost and incremental QALY pairing, using the assigned distributions around each model parameter, selected randomly during 5,000 iterations. Dots falling below the diagonal line (the willingness-to-pay threshold of AU$50,000 per QALY) are considered cost-effective. The proportion of simulations considered cost-effective was 60.1%. Note: QALYs, quality-adjusted life years.

Model parameters

Patient hospital episode information was obtained from the hospital-based corporate information system (HBCIS) for all patients who spent time in either the intensive care or burns units between April 1, 2016, and January 1, 2019. HBCIS routinely collects all patient separations and patient days (or occupied bed days) that occur in public hospitals. Hospital daily admission rate, ward admission probability, ward transfer proportions, and ward length of stay were estimated empirically from the HBICS data set (Supplementary Tables S4S8). Ward stays were estimated as independent γ distributions for all observed ward pair combinations using the methods of moments approach. Reference Briggs, Sculpher and Claxton20

The sensitivity of detecting environmental CRAB contamination with microbiology cultures was estimated at 40% and metagenomics at 80% (Table 1). These estimates were calculated from 5 positive environmental samples from 50 sequenced samples. Due to the uncertainty of samples not being detected, these values were varied in sensitivity analysis (Table 1).

Table 1. Parameter Description, Values, and Sources Used in the Hybrid Simulation Model

Note. ICU, intensive care unit; SD, standard deviation; LOS, length of stay; Prob, probability; HTA, high-touch area; PPE, personal protective equipment; HBCIS, hospital-based corporate information system; ID, infectious disease; WGS, whole-genome sequencing; CRAB, carbapenem-resistant Acinetobacter baumannii.

a These values were used in one-way sensitivity analysis.

b These values were used for a β distribution in the probabilistic sensitivity analysis.

c These values were used for a γ distribution in the probabilistic sensitivity analysis.

d These values were calculated from 5 positive environmental samples (4 of 5 by metagenomics and 2 of 5 by culture) out of 50 total sequenced samples.

e Metagenomics cost entails $25 for a DNA extraction, $40 for the library preparation, $190 for sequencing and $100 for bioinformatics.

f Colistin administered at 275 mg for 14 d and tigecycline administered at 100 mg followed by 50 mg every 12 h for 14 d.

g Colistin administered at 275 mg for 14 d and meropenem administered at 1.0–2 g 3 times daily for 14 d.

h Patient health utility was assumed to change for 14 d after diagnosis, unless leaving hospital first.

The daily incidences of common MDROs were extracted from 3 years (April 2016 to January 2019) of MDRO surveillance data and converted to probabilities (Supplementary Table S2). ICD-10 codes from the HBICS data set identified the bloodstream, respiratory, and urinary tract infection rates for each of the MDROs (Table S3). The frequency of deaths in hospital from patients infected with any of the MDROs were obtained from published reports Reference Coombs, Daley and Collignon21 and ranged from 0.7% for Clostridioides difficile infection to 36.6% for vancomycin-resistant Enterococcus (Table S2).

Genetic relatedness was determined by examining the number of core-genome single-nucleotide polymorphisms (SNPs) that differ between any 2 isolates (pairwise core-genome SNP distance). Genetically related isolates were subdivided into clusters when the SNP distances between them were under a 5 SNPs per megabase threshold. Reference Octavia, Wang, Tanaka, Kaur, Sintchenko and Lan22 Two years (December 2017 to December 2019) of processed MDRO WGS results identified clusters of extended spectrum β-lactamase (ESBL)–producing Escherichia coli and ESBL-producing Klebsiella pneumoniae.

Health utilities are cardinal values that represent the strength of an individual’s preferences for specific health-related outcomes. They are scored on a scale between 0, worst health to 1, perfect health. Health utilities are used to calculate QALYs, a measure of patient benefit, where the length of time in a health state is adjusted to reflect the quality of life (health utility score). Health utilities were used in the model to estimate the health impact of sepsis (0.53), urinary tract infection (0.73), respiratory tract infection (0.58), and an uninfected health state (0.82) Reference White, Barnett and Hall6,Reference Lee, Bailey and Smith23 (Table 1). We applied a negative health utility when patients were isolated. Reference Mac, Fitzpatrick, Johnstone and Sander24

Healthcare costs, which were calculated in 2020 Australian dollars (1 AU$ = 0.68 USD 25 ), were assigned to WGS (AU$150, US$102), metagenomics (AU$355, US$241), microbiology culture tests (AU$82, US$56), cleaning (AU$70/AU$140, US$48/US$95), closed bed days (AU$216/AU$466, US$147/US$317), personal protective equipment (PPE) (AU$52, US$35), environmental screening (AU$3, US$2) and antibiotic treatments (AU$176–AU$4,585, US$120-US$3,118) (Table 1 and Supplementary Table S3). Reference Page, Barnett and Graves2629 The WGS and metagenomics costs comprised of sequencing and bioinformatics costs. Metagenomics was more expensive than WGS due to samples having ∼5 times more genetic content.

Analysis

The main outcomes were number of MDRO cases, hospital costs, and QALYs. Model outcomes were aggregated from events that emerged from the interacting processes of ‘patient flow dynamics,’ ‘pathogen transmission dynamics,’ and ‘outbreak control team decisions.’ These outcomes were averaged >5,000 stochastic model simulations and presented as means and interquartile range (IQRs). Future costs and QALYs were discounted at 5% per year to provide present values. Incremental cost-effectiveness ratios were calculated as the difference in costs between 2 groups divided by the difference in QALYs. Reference Briggs, Weinstein, Fenwick, Karnon, Sculpher and Paltiel17 Probabilistic sensitivity analyses were undertaken to assess the likelihood of the scenario being cost-effective, considered at a willingness-to-pay threshold of AU$50,000 (US$34,000) per QALY gain and AU$28,033 (US$19,062) per QALY gain. One-way sensitively analyses were performed on the costs of WGS, cost of metagenomics, mortality rate of CRAB, WGS turnaround time, and the sensitivity of environmental swab culture and metagenomics. Outbreak fadeouts are a hazard of stochastic simulations; therefore, a supplementary analysis was performed which only included simulations where a CRAB outbreak occurred. Access to the full working model is available via the AnyLogic cloud at https://cloud.anylogic.com/model/72981523-81b1-4f31-b152-06bc9a906b7f?mode=SETTINGS.

Results

In scenario 1, the infection control team detected on average 30 (IQR, 3–39) patients with CRAB, accrued total hospital costs of AU$1,608,571 (US$1,093,828) (IQR, AU$1,421,564, AU$1,677,308; US$966,664, US$1,140,569) and 6,578 QALYs (IQR, 6,476, 6,707) (Table 2). This compares with Scenario 2 outcomes of 14 fewer patients with CRAB, 59 additional QALYs and AU$75,099 (US$51,067) cost savings. When scenario 3 was compared with scenario 1, there were 18 fewer patients with CRAB, 74 additional QALYs and AU$93,822 (US$63,799) cost savings (Table 2). Both scenario 2 and scenario 3 were cost saving and improved patient QALYs compared with scenario 1. The increase in QALYs was primarily driven by increases in patient quality of life.

Table 2. Projected Health and Economic Outcomes Over the Outbreak by Scenario

Note. IQR, interquartile range; WGS, whole-genome sequencing; CRAB, carbapenem-resistant Acinetobacter baumannii; ST1050, subtype 1050; Q, quartile; enviro, environmental; MDROs, multdrug-resistant organisms; QALYs, quality-adjusted life years; PPE, personal protective equipment; diff, difference; PCR, polymerase chain reaction assay.

a Microbiological culture and PCR.

b Costs attributed to MDROs arising in addition to the ST1050 CRAB outbreak were included; however, no differences were observed across scenarios.

c Environmental swabs and microbiological cultures.

d Cleaning costs and closed bed-day costs.

Microbiology culture tests were a major driver of total costs with 66.3%, 72.0%, and 72.6% of total costs for scenario 1, scenario 2 and scenario 3, respectively (Supplementary Fig. S2). Metagenomics and WGS are relatively small fractions of total hospital costs (∼<2%).

Plots of incremental cost-effectiveness ratios identified 58% (scenario 2 vs 1), 60% (scenario 3 vs 1) and 53% (scenario 2 vs 3) of iterations were cost-effective at AU$50,000 per QALY (Fig. 2). This changed to 57% (scenario 2 vs 1), 60% (scenario 3 vs 1), and 52% (scenario 2 vs 3) of iterations at willingness-to-pay threshold of AU$28,033 per QALY. QALYs and total costs both decreased in ∼13% of iterations. Cost savings were identified in 50% (scenario 2 vs 1), 47% (scenario 3 vs 1), and 48% (scenario 2 vs 3) of iterations.

When plausible alternative values for critical parameters were used in the model, hospital cost savings and increases in QALYs were retained (Supplementary Table S9). When only simulations where a CRAB outbreak occurred were analyzed, the percentage of cost-effective iterations increased to 64% (Supplementary Table S10).

Discussion

Our results showed the joint use of WGS and metagenomics were associated with smaller outbreaks, lower hospital costs, and an increase in accrued QALYs. Cost savings accrued from avoided treatments and fewer contact precautions resulting from fewer CRAB cases. We highlight the relatively small cost of WGS surveillance across 8 MDRO species (WGS cost AU$22,642 (US$15,397) and metagenomics cost AU$13,343 (US$9,073) and how they are dwarfed by microbiology culture costs (AU$1,099,269, US$747,503), antibiotic treatment costs (AU$125,622, US$85,423), and cost of PPE (AU$248,919, US$169,265). Sensitivity analyses indicated that WGS and metagenomics use was cost-effective in this context with a high degree of confidence when model inputs varied from their base values.

Several studies report on the economic benefits of WGS; however, to the best of our knowledge, this is the first economic evaluation on the use of metagenomics for infection control. Our previous work on an ESBL E. coli outbreak in 5 extended-stay wards predicted significant cost savings if WGS was implemented early Reference Lee, Elliott and Harris30 and a budget impact analysis reported that statewide WGS surveillance targeting 6 MDROs could be cost saving. Reference Gordon, Elliott and Forde31 Furthermore, removing contact precautions for MDROs at low risk of spreading, confirmed from WGS data, may reduce hospital costs further. Reference Elliott, Hare and Hajkowicz14 International studies have identified cost savings from WGS through interventions targeting transmission routes Reference Kumar, Sundermann and Martin15 and reducing the spread of MRSA. Reference Dymond, Davies and Mealing32

Within our model, the stochastic processes created significant variance in the number of patients with CR-Ab across the scenarios that heavily drive the subsequent costs savings (−AU$781,840 to AU$264,347; −US$531,651 to US$179,756) and QALYs gained (−287 to 479). Approximately 20% of simulations resulted in no outbreaks occurring. Outbreak fadeouts are a hazard of stochastic outbreak simulations. Reference McBryde33 When the ∼20% of iterations without an outbreak were removed, the likelihood of cost-effectiveness increased to 61% and 64% for scenario 2 and scenario 3 respectively (Supplementary Table S10). Understanding how and why stochastic simulations deviate from deterministic simulations is being viewed with increasing importance. Reference Keeling and Ross34 For an outbreak to start, the pathogen must lie undetected long enough to spread to another patient or environmental object, which is influenced by stochastic parameters such as the patient’s length of stay, the regularity of screening, and ward transfer locations. The highly surveilled nature of the intensive care unit and burn unit added to the possibility that no outbreak occurs because patients would remain undetected for no longer than a few days. Several models are suitable for analyzing infection diseases. The trade-offs between compartmental models, agent-based simulation, and network-driven models are well documented. Reference van Kleef, Robotham, Jit, Deeny and Edmunds35 We chose an individual-based structure due to the data and computational software available to us, along with this level of detail being required to model the impact of WGS interventions and believe this is a valid choice.

The excessive cost and time to return a result has in the past been perceived as a limiting factor of implementing WGS into hospital practice. Reference McGann, Bunin and Snesrud8 The cost of pathogen WGS continues to decrease, with international studies quoting the price between US$70, Reference Kumar, Sundermann and Martin15 £100 Reference Dymond, Davies and Mealing32 and AU$350 (US$238). Reference Lee, Elliott and Harris30 The WGS cost of AU$150 (US$102) used in this study, the current operating price by our sequencing partners, consists of AU$124 (US$84) for the sequencing of the pathogen and AU$26 (US$18) for the bioinformatics. The bioinformatics costs dropped from AU$75 (US$51) Reference Lee, Elliott and Harris30 due to the recently established pipeline creating a more automated analysis. Because WGS was not a dominant cost in the model, changing the cost of sequencing in sensitivity analysis did not change the outcomes. A 7-day WGS turnaround time is too long, although our model found little impact in reducing it to 3 days. This could be due to the outbreak occurring in already highly surveilled wards of the hospital. Reducing WGS turnaround time would likely have greater impact on outbreak size outside the ICU. Reference Kumar, Sundermann and Martin15

This study had several limitations. This retrospective evaluation was based on a single hospital outbreak, which limits the generalizability of these findings. The results were dependent on decisions during model construction and several input parameters, which in line with the nature of infectious disease modeling, led to substantial uncertainty. Healthcare worker-to-patient transmission was not directly modeled, although spread to other rooms within the ward represents this type of transmission due to limited patient mobility. It was not possible to collect health utility scores directly from patients in this study, and scarce evidence was available in the literature. In the absence of other evidence, the sensitivity of metagenomics and microbiology cultures on detecting environmental CRAB contamination were estimated from 5 cases; however, sensitivity analyses on these values did not change the favorable incremental cost per QALY ratios. Parameters used in the model, like the environmental contamination values, did not have strong supporting evidence. Targeted metagenomics was used in this study because untargeted metagenomics as a surveillance tool has not been fully explored and would likely have lower sensitivity. Balanced against these limitations, is the use of a detailed simulation model, informed by accurate outbreak data, historical sequencing data, and 3 years of MDRO surveillance data. Our modeling incorporated the stochastic nature of outbreaks by limiting outbreak pathways through techniques like blocking, which removes unnecessary variation. 36 Limiting the maximum number of transmissions to 3 in the binomial distribution formula was an example of blocking. On balance, based on our analysis and using realistic model parameters, targeted metagenomics, as used in this hospital, yielded good value for the money spent.

In conclusion, introducing WGS and metagenomics into infection control was likely to have favorable economic and clinical outcomes. The low proportion of costs attributed to sequencing all MDROs within the ICU and burns unit (<2%) highlights the manageable ongoing costs and encourages further sequencing studies in other hospital settings. Implementing these sequencing technologies is likely to yield decreased hospital costs, decreased infections, and increased QALYs.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/ash.2021.233

Acknowledgments

We acknowledge all staff at the Royal Brisbane and Women’s Hospital and Pathology Queensland who were involved in patient care and the clinical response to the outbreak described in this study.

Financial support

This study was supported by Queensland Genomics, Queensland Health, the Queensland government.

Conflicts of interest

All authors report no conflicts of interest relevant to this article.

References

Acinetobacter in Healthcare Settings. Centers for Disease Control and Prevention website. https://www.cdc.gov/hai/organisms/acinetobacter.html. Published 2019. Accessed November 10, 2021.Google Scholar
Venditti, C, Vulcano, A, D’Arezzo, S, et al. Epidemiological investigation of an Acinetobacter baumannii outbreak using core genome multilocus sequence typing. J Glob Antimicrob Resist 2019;17:245249.CrossRefGoogle ScholarPubMed
Rossi, I, Royer, S, Ferreira, ML, et al. Incidence of infections caused by carbapenem-resistant Acinetobacter baumannii . Am J Infect Control 2019;47:14311435.CrossRefGoogle ScholarPubMed
Umezawa, K, Asai, S, Ohshima, T, et al. Outbreak of drug-resistant Acinetobacter baumannii ST219 caused by oral care using tap water from contaminated hand hygiene sinks as a reservoir. Am J Infect Control 2015;43:12491251.CrossRefGoogle ScholarPubMed
Vickery, K, Deva, A, Jacombs, A, Allan, J, Valente, P, Gosbell, IB. Presence of biofilm containing viable multiresistant organisms despite terminal cleaning on clinical surfaces in an intensive care unit. J Hosp Infect 2012;80:5255.CrossRefGoogle Scholar
White, NM, Barnett, AG, Hall, L, et al. Cost-effectiveness of an environmental cleaning bundle for reducing healthcare-associated infections. Clin Infect Dis 2020;70:24612468.CrossRefGoogle ScholarPubMed
Chiang, AD, Dekker, JP. From the pipeline to the bedside: advances and challenges in clinical metagenomics. J Infect Dis 2020;221:S331S340.CrossRefGoogle Scholar
McGann, P, Bunin, JL, Snesrud, E, et al. Real-time application of whole-genome sequencing for outbreak investigation—what is an achievable turnaround time? Diagn Microbiol Infect Dis 2016;85:277282.CrossRefGoogle ScholarPubMed
Matsumura, Y. Detection of antimicrobial resistance determinants using whole-genome sequencing. Japan J Chemother 2020;68:198203.Google Scholar
Fujikura, Y, Hamamoto, T, Kanayama, A, Kaku, K, Yamagishi, J, Kawana, A. Bayesian reconstruction of a vancomycin-resistant Enterococcus transmission route using epidemiologic data and genomic variants from whole-genome sequencing. J Hosp Infect 2019;103:395403.CrossRefGoogle ScholarPubMed
Forbes, JD, Knox, NC, Peterson, CL, Reimer, AR. Highlighting clinical metagenomics for enhanced diagnostic decision making: a step towards wider implementation. Comput Struct Biotech J 2018;16:108120.CrossRefGoogle ScholarPubMed
Simner, PJ, Miller, S, Carroll, KC. Understanding the promises and hurdles of metagenomic next-generation sequencing as a diagnostic tool for infectious diseases. Clin Infect Dis 2018;66:778788.CrossRefGoogle ScholarPubMed
Elliott, TM, Lee, XJ, Foeglein, A, Harris, PN, Gordon, LG. A hybrid simulation model approach to examine bacterial genome sequencing during a hospital outbreak. BMC Infect Dis 2020;20:72.CrossRefGoogle ScholarPubMed
Elliott, TM, Hare, N, Hajkowicz, K, et al. Evaluating the economic effects of genomic sequencing of pathogens to prioritise hospital patients competing for isolation beds. Australian Health Rev 2021;45:5965.CrossRefGoogle ScholarPubMed
Kumar, P, Sundermann, AJ, Martin, EM, et al. Method for economic evaluation of bacterial whole genome sequencing surveillance compared to standard of care in detecting hospital outbreaks. Clin Infect Dis 2021;73:e9e18.CrossRefGoogle ScholarPubMed
Roberts, LW, Forde, BM, Hurst, T, et al. Genomic surveillance, characterization and intervention of a polymicrobial multidrug-resistant outbreak in critical care. Microbial Genomics 2021. doi: 10.1099/mgen.0.000530.CrossRefGoogle Scholar
Briggs, AH, Weinstein, MC, Fenwick, EAL, Karnon, J, Sculpher, MJ, Paltiel, AD. Model parameter estimation and uncertainty analysis:a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group–6. Med Decision Making 2012;32:722732.CrossRefGoogle ScholarPubMed
Pethes, R, Ferenci, T, Kovács, L. Infectious hospital agents: an individual-based simulation framework. Paper presented at the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC); October 9–12, 2016.CrossRefGoogle Scholar
Gurieva, T, Dautzenberg, MJD, Gniadkowski, M, Derde, LPG, Bonten, MJM, Bootsma, MCJ. The transmissibility of antibiotic-resistant enterobacteriaceae in intensive care units. Clin Infect Dis 2018;66:489493.CrossRefGoogle ScholarPubMed
Briggs, A, Sculpher, M, Claxton, K. Decision Modelling for Health Economic Evaluation. London: Oxford University Press; 2006.Google Scholar
Coombs, GBJ, Daley, D, Collignon, P, et al. Australian Group on Antimicrobial Resistance Sepsis Outcomes Programs: 2018 Report. Sydney: Australian Commission on Safety and Quality in Health Care; 2019.Google Scholar
Octavia, S, Wang, Q, Tanaka, MM, Kaur, S, Sintchenko, V, Lan, R. Delineating community outbreaks of Salmonella enterica serovar typhimurium by use of whole-genome sequencing: insights into genomic variability within an outbreak. J Clin Microbiol 2015;53:10631071.CrossRefGoogle ScholarPubMed
Lee, BY, Bailey, RR, Smith, KJ, et al. Universal methicillin-resistant Staphylococcus aureus (MRSA) surveillance for adults at hospital admission: an economic model and analysis. Infect Control Hosp Epidemiol 2010;31:598606.CrossRefGoogle ScholarPubMed
Mac, S, Fitzpatrick, T, Johnstone, J, Sander, B. Vancomycin-resistant enterococci (VRE) screening and isolation in the general medicine ward: a cost-effectiveness analysis. Antimicrob Resist Infect Control 2019;8:168.CrossRefGoogle ScholarPubMed
Conversion rates. Organisation for Economic Co-operation and Development website. https://www.oecd-ilibrary.org/finance-and-investment/purchasing-power-parities-ppp/indicator/english_1290ee5a-en. Published 2020. Accessed November 8, 2021.Google Scholar
Page, K, Barnett, AG, Graves, N. What is a hospital bed day worth? A contingent valuation study of hospital chief executive officers. BMC Health Serv Res 2017;17:137.CrossRefGoogle Scholar
Viehman, JA, Nguyen, MH, Doi, Y. Treatment options for carbapenem-resistant and extensively drug-resistant Acinetobacter baumannii infections. Drugs 2014;74:13151333.CrossRefGoogle ScholarPubMed
Otter, JA, Burgess, P, Davies, F, et al. Counting the cost of an outbreak of carbapenemase-producing Enterobacteriaceae: an economic evaluation from a hospital perspective. Clin Microbiol Infect 2017;23:188196.CrossRefGoogle ScholarPubMed
Puritan sterile large foam swab, polypropylene handle w/thumb stop. Fisher Scientific website. https://www.fishersci.com/shop/products/puritan-macrofoam-tipped-applicators/22029573. Published 2020. Accessed May 11, 2020.Google Scholar
Lee, XJ, Elliott, TM, Harris, PNA, et al. Clinical and economic outcomes of genome sequencing availability on containing a hospital outbreak of resistant Escherichia coli in Australia. Value Health 2020;23:9941002.CrossRefGoogle ScholarPubMed
Gordon, LG, Elliott, TM, Forde, B, et al. Budget impact analysis of routinely using whole-genomic sequencing of six multidrug-resistant bacterial pathogens in Queensland, Australia. BMJ Open 2021;11:e041968.CrossRefGoogle ScholarPubMed
Dymond, A, Davies, H, Mealing, S, et al. Genomic surveillance of methicillin-resistant Staphylococcus aureus: a mathematical early modeling study of cost-effectiveness. Clin Infect Dis 2020;70:16131619.CrossRefGoogle ScholarPubMed
McBryde, ES. Mathematical and statistical modelling of infectious diseases in hospitals. PhD dissertation. Eprints Repository website. https://eprints.qut.edu.au/16330/. 2006. Accessed November 10, 2021.Google Scholar
Keeling, MJ, Ross, JV. On methods for studying stochastic disease dynamics. J R Soc Interface 2008;5:171181.CrossRefGoogle ScholarPubMed
van Kleef, E, Robotham, JV, Jit, M, Deeny, SR, Edmunds, WJ. Modelling the transmission of healthcare-associated infections: a systematic review. BMC Infect Dis 2013;13:294.CrossRefGoogle ScholarPubMed
University TPS. 4.1 - blocking scenarios. STAT 503: design of experiments, 2018 Penn State Eberly College of Science website. https://online.stat.psu.edu/stat503/lesson/4. Published 2018. Accessed January 3, 2019.Google Scholar
Ng, DHL, Marimuthu, K, Lee, JJ, et al. Environmental colonization and onward clonal transmission of carbapenem-resistant Acinetobacter baumannii (CRAB) in a medical intensive care unit: the case for environmental hygiene. Antimicrob Resist Infect Control 2018;7.Google Scholar
Lerner, AO, Abu-Hanna, J, Carmeli, Y, Schechner, V. Environmental contamination by carbapenem-resistant Acinetobacter baumannii: the effects of room type and cleaning methods. Infect Control Hosp Epidemiol 2019;41:166171.Google Scholar
Figure 0

Fig. 1. Outbreak timeline with investigation strategy changes. Each colored box represents a distinct outbreak of ST1050 CRAB. The red box consisted of 17 cases, the green box represents 8 cases and the orange box represents 11 cases. Each black arrow indicated initiation of environmental screening. To improve specimen collection the focus of environmental swabbing changed from high-touch areas to high bacterial load areas in July 2018. High-touch areas were defined as places commonly touched such as nurse keyboards, trolleys and door handles. High bacterial-load areas were defined as areas of high biomass such as floor drains, plumbing and inside burns bath drains. WGS was implemented as part of outbreak control in May 2018 and metagenomics in November of 2017. Note. WGS, whole-genome sequencing.

Figure 1

Fig. 2. Introduction of WGS and metagenomics into microbiology culture infection control process. Note. micro, microbiology; enviro, environmental; WGS, whole-genome sequencing; HTA, high-touch area.

Figure 2

Fig. 3. Scatterplot of incremental costs and QALYs (all patients) for scenario 3 versus scenario 1. Each dot represents an incremental cost and incremental QALY pairing, using the assigned distributions around each model parameter, selected randomly during 5,000 iterations. Dots falling below the diagonal line (the willingness-to-pay threshold of AU$50,000 per QALY) are considered cost-effective. The proportion of simulations considered cost-effective was 60.1%. Note: QALYs, quality-adjusted life years.

Figure 3

Table 1. Parameter Description, Values, and Sources Used in the Hybrid Simulation Model

Figure 4

Table 2. Projected Health and Economic Outcomes Over the Outbreak by Scenario

Supplementary material: File

Elliott et al. supplementary material

Elliott et al. supplementary material

Download Elliott et al. supplementary material(File)
File 185.2 KB