Introduction
Patients who receive care at long-term care facilities (LTCFs) are at increased risk for colonization and infection with multidrug-resistant organisms (MDROs), including emerging gram-negative and fungal pathogens. Reference Aliyu, Smaldone and Larson1–Reference Harris, Pineles and Johnson4 Across a recent, large sample of California nursing homes, 48% of residents were colonized with at least one MDRO, Reference McKinnell, Miller and Singh5 and 8% of patients residing in long-term acute care hospitals were colonized with carbapenem-resistant Enterobacterales specifically. Reference McKinnell, Singh and Miller6 LTCF patients also have high utilization rates of local hospital networks and frequent inpatient admissions, Reference Lee, Bartsch and Lin7–Reference Lin, Lyles-Banks and Lolans9 with studies consistently identifying LTCF exposure as one of the strongest risk factors for MDRO colonization and infection among hospitalized patients. Reference Prabaker, Lin and McNally10–Reference Goodman, Simner and Klein12 For example, in a study of all patients with Candida auris fungemia at a New York hospital between 2016 and 2018, 78% had been admitted from nursing homes. Reference Park, Bradley, Brooks, Burney and Wassner13
Due to LTCF patients’ high burden of MDRO colonization, accurate and timely identification of LTCF-exposed inpatients is important for infection control practice and research. Rapid identification allows for empirical isolation and/or targeted screening, reducing the potential for MDRO introduction and dissemination within acute care hospitals. Reference Harrison, Ludden and Brodrick14,15 However, data limitations make ascertainment difficult. Most hospitals do not systematically assess inpatients for LTCF exposure (eg, via intake questionnaire), and few states maintain databases for tracking patient movement across healthcare facilities. Reference Ray, Lin, Weinstein and Trick16 One place where LTCF exposure is often documented, however, is in the “History & Physical” (H&P) clinical note, Reference Prabaker, Hayden, Weinstein and Lin17 a narrative admission history obtained at or shortly after intake that describes the patient’s history, relevant clinical information, and present illness. Yet, because H&P notes are “unstructured” (ie, free-text) data, absent manual chart review, which is resource-intensive and cannot be incorporated into automated MDRO screening algorithms, or natural language processing (NLP) techniques for analyzing free-text data, their LTCF information is effectively unusable. To our knowledge, there are no publicly available NLP classifiers for automated extraction of LTCF information from clinical notes.
The electronic health record (EHR) admission source (point of origin) and discharge location (discharge status) fields, which US hospitals are required to document for all hospitalized patients, 18 offer an alternative strategy for identifying LTCF-exposed inpatients. As structured data fields, these variables are readily extractable in standardized, machine-readable formats, and their list of available locations includes LTCF facilities. However, apart from a 2012 study that evaluated the admission source field for identifying direct LTCF transfers, Reference Prabaker, Hayden, Weinstein and Lin17 the admission source and discharge location fields have not been validated for identifying LTCF exposure. Misclassifying LTCF-exposed patients could bias inferences in hospital epidemiology studies that rely on these variables and compromise the performance of MDRO screening algorithms Reference Goodman, Simner and Klein12 and day-to-day infection control practice that relies on accurately identifying LTCF patients. The objective of the current study was to evaluate the accuracy of the EHR “Admission Source” and “Discharge Location” fields against the clinical notes for identifying inpatients with LTCF exposure.
Methods
Study setting, population, and electronic health record data extraction
We conducted a retrospective study of adult admissions from 2016 to 2021 across 12 acute care hospitals in the University of Maryland Medical System (UMMS). UMMS hospitals are distributed across 13 Maryland counties and serve urban, suburban, and rural communities. For each admission, we manually reviewed the H&P note and electronically extracted the following data from the Epic EHR: (1) patient demographic and hospital data, (2) preadmission location, and (3) preadmission and discharge locations for all previous hospitalizations across UMMS hospitals in the preceding 90 days. The University of Maryland School of Medicine Institutional Review Board approved this study, with a waiver of informed consent.
Long-term care facility exposure definitions by ascertainment method
Using the structured EHR data, inpatients were classified as LTCF-exposed if their “Admission Source” was transfer from, or their “Discharge Location” for previous recent admissions was discharge to, an LTCF, skilled nursing facility (SNF), assisted living facility (ALF), rehabilitation facility, “chronic” facility, or “other subacute” facility (Supplemental Table 1). Using the unstructured H&P notes, patients were classified as LTCF-exposed using the sampling strategy and annotation guidelines outlined below.
Sampling strategy and manual annotation of clinical notes
We randomly sampled adult, unique-patient admissions for manual review of the H&P note (excluding the first quarter of 2016 so that all admissions would have full 90-day lookback periods for identifying previous hospitalizations). LTCF exposure was classified using standardized guidelines developed through expert consensus (K.G., P.T., A.H., E.K.) (Supplemental Table 1). Notes from 50 admissions, selected through random sampling, were reviewed by both K.G. and M.T., and chance-corrected inter-rater agreement was calculated using Cohen’s Kappa Statistic. Reference Cohen19 Based on the high agreement identified (Cohen’s Kappa = .85), the remaining notes were divided and reviewed individually (K.G. and M.T.). Reviewers met regularly to discuss and adjudicate uncertainties.
Statistical methods
Descriptive statistics were calculated using median (interquartile range [IQR]) for continuous variables or frequency count (percentage) for categorical variables. Previous research has found that the EHR admission source field and the physician-documented notes have ≥98% specificity for identifying admissions from other healthcare facilities, including LTCFs. Reference Prabaker, Hayden, Weinstein and Lin17 Therefore, assuming an indication of LTCF exposure in either the EHR structured data or the H&P note represented true exposure, we estimated each ascertainment method’s sensitivity with 95% Wilson CIs, stratified by exposure timing: “current” (occurring immediately prior to admission or a current LTCF resident, even if not a direct LTCF transfer) versus “any recent” (occurring within the prior 90 days, including current) (Supplemental Table 1). Analyses were performed using SAS version 9.4 software (SAS Institute, Cary, North Carolina).
Results
Source population and study cohort
During the 2016–2021 study period, there were 352,947 adult inpatient admissions across UMMS hospitals (source population). The validation (study) cohort comprised 1,020 randomly sampled admissions, which were drawn from all UMMS hospitals and study years (Table 1). Study patients had a median age of 58 years (IQR: 41–71), 48% were male, and 55% identified as White race. 227 (22%) study patients were admitted to a UMMS hospital in the 90 days preceding the index admission (Table 1).
Table 1. Patient, admission, and hospital characteristics in a validation cohort of 1,020 randomly sampled index admissions across 12 hospitals in the University of Maryland Medical System (UMMS) (2016–2021)
a American Indian or Alaskan Native, Native Hawaiian, or Other Pacific Islander, Other race, and Unknown race
b Labor and delivery (5%), urgent admissions (2%), “other” admissions (0.4%), rehabilitation admissions (0.3%), and unknown admissions (0.1%)
Performance of unstructured history & physical notes and electronic health record structured data fields for identifying long-term care facility-exposed inpatients
Across the 1,020 study admissions, 111 (11%) patients were classified as LTCF-exposed by either the structured EHR data fields or the unstructured H&P clinical notes. Stratifying by exposure timing, classifications and performance were as follows.
Identifying current LTCF exposure
Sixty-five patients were identified as having current LTCF exposure: 30 (3% of all admissions) by the EHR “Admission Source” for the index admission and 60 (6% of all admissions) by the H&P note for the index admission. Thirty-five patients were identified only by the H&P note, 25 patients were positive for current LTCF exposure by both their EHR-coded admission source and the H&P note, and 5 patients were positive only by admission source. The sensitivity of the EHR “Admission Source” field for detecting current LTCF exposure was 46% (30/65) (95% CI: 35%–58%) and of the H&P note was 92% (60/65) (95% CI: 83%–97%) (Figure 1).
Figure 1. Sensitivity of EHR data sources compared to the index admission’s history & physical (H&P) clinical note for identifying inpatients with long-term care facility (LTCF) exposure, across 1,020 University of Maryland Medical System adult admissions between 2016 and 2021. LTCF exposure was stratified by timing: “current” LTCF exposure (occurring immediately prior to admission or an LTCF resident) versus “any recent” (occurring within the prior 90 days, including current).
Identifying any recent long-term care facility exposure (within the prior 90 days, including current)
Expanding to include a 90-day lookback period, of 111 patients with any recent LTCF exposure, 35 (3% of all index admissions) were identified by the EHR “Admission Source” for the index or previous admissions, 63 (6% of all index admissions) by the “Discharge Location” for previous admissions, and 76 (7% of all index admissions) by the H&P note for the index admission.
For detecting any recent LTCF exposure within the prior 90 days, the sensitivity of the “Admission Source” field across the index or previous admissions was 32% (35/111) (95% CI: 24%–41%), of the “Discharge Location” field across previous admission(s) was 57% (63/111) (95% CI: 47%–66%), and of the index admission’s H&P note was 68% (76/111) (95% CI: 59%–76%). Overall, compared to the H&P notes, the combined sensitivity of the admission source and discharge location fields for detecting any recent LTCF exposure was 76% (84/111) (95% CI: 67%–83%) (Figure 1).
Characteristics of long-term care facility-exposed inpatients missed by the electronic health record admission and discharge location fields
Twenty-seven patients (24% of patients with any recent LTCF exposure) were identified as LTCF-exposed by the H&P note but missed by the admission and discharge location data fields. Twenty-two (81%) of these patients had current LTCF exposure, and 22 (81%) had not been recently hospitalized (the 22 patients in each of these groups did not fully overlap). Of the 27 missed patients, most had an admission source for their index admission of “Home, Self-Referred, Group Home, Congregate House, Foster” (n = 15, 56%) or “Transfer from Another Acute Care Hospital” (n = 7, 26%). Figure 2 reflects the distribution of LTCF exposure timing and facility types for missed patients.
Figure 2. Distribution of LTCF exposure timing and facility types for patients identified as LTCF-exposed by the H&P note but missed by the admission and discharge location data fields. Abbreviations: SNF, skilled nursing facility; ALF, assisted living facility.
Discussion
Residence or recent stay in an LTCF is one of the most important risk factors for MDRO carriage and infection, making reliable identification of LTCF-exposed inpatients a critical priority for infection control day-to-day practice and research (eg, to guide targeted screening decisions, contact isolation initiation, or conduct outbreak investigations). However, because most hospital EHRs do not include a dedicated field for documenting LTCF exposure, absent manual, and time-consuming review of patient charts, identifying LTCF-exposed inpatients is challenging. Two universally mandated EHR billing fields, hospitalization admission source and discharge location, offer a potential alternative strategy for identifying LTCF-exposed inpatients. Analyzing more than 1,000 randomly sampled admissions between 2016 and 2021 across 12 UMMS hospitals, we found that when used individually, each field misses many LTCF-exposed patients, but sensitivity increases to 76% compared to the clinical notes when evaluating admission source and discharge location together.
Our study was stratified by LTCF exposure timing. For identifying current LTCF exposure, ie, exposure that immediately preceded hospitalization, the H&P note significantly outperformed the admission source field. Specifically, the H&P clinical note identified most patients with current LTCF exposure (92%), double the identification rate (46%) of admission source. These results are consistent with findings from a smaller 2012 study by Prabaker et al Reference Prabaker, Hayden, Weinstein and Lin17 that also evaluated the H&P note and admission source field for identifying current LTCF exposure. That study of 523 inpatients at Rush University Medical Center similarly found that the H&P note outperformed the admission source field, with respective sensitivities of 71% and 50% compared to patient interview. Reference Prabaker, Hayden, Weinstein and Lin17
In manual review, we found that most currently LTCF-exposed patients missed by the admission source field had admission sources of “home” and “self-referral,” and some of these patients first presented to the Emergency Department (ED). A further 26% of missed patients were direct transfers from other acute care hospitals. These alternative admission sources help explain the admission source field’s poor sensitivity for identifying LTCF exposure, because preadmission visits to the ED or other hospitals “break the chain” of direct admission from an LTCF. Prabaker et al identified similar patterns of missingness in their earlier study, including that many LTCF patients were transported to the hospital by family members rather than as direct LTCF transfers via ambulance. Interestingly, they also found that one-third of LTCF patients were non-verbal and thus could not have answered an LTCF intake questionnaire, even had the hospital implemented one. Reference Prabaker, Hayden, Weinstein and Lin17 Taken together, both our and Prabaker et al’s findings suggest that absent a dedicated LTCF-exposure ascertainment mechanism (eg, a statewide database Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick20 or family interview), identifying current LTCF-exposed inpatients requires clinical documentation. Given the resource-intensiveness of manual chart review, the development of automated NLP classifiers for extracting LTCF information from free-text notes, if appropriately validated, could thus substantially assist hospital epidemiology and infection control efforts.
MDRO carriage generally persists for months or years, Reference Lima, Sousa and Filgueira21,Reference Weintrob, Roediger and Barber22 and any recent LTCF exposure may increase MDRO risk. Therefore, we also evaluated the admission and discharge locations for admissions occurring within 90 days preceding the index admission, which to our knowledge is the first validation of these data fields for identifying recent LTCF exposure. As with current LTCF exposure, we found that the admission source for the index admission and any recent admissions had poor sensitivity (32%) for identifying patients with any recent LTCF exposure. These findings are broadly consistent with published literature for non-LTCF healthcare facilities, which documents that many inpatients with recent, previous hospitalizations are not admitted as direct hospital transfers. Reference Lee, McGlone and Song23 Evaluating the discharge location for previous admission(s) identified a greater percentage—57%—of LTCF-exposed patients. However, an approach that evaluated the admission source and discharge location fields together was the highest-performing strategy, achieving 76% sensitivity compared to the H&P note for detecting any recent LTCF exposure (95% CI: 67%–83%). These results suggest that patients admitted from and discharged to LTCFs represent largely distinct populations, a finding that is clinically intuitive insofar as many patients discharged to rehabilitation and SNFs do not enter from LTCFs and are not permanent LTCF residents.
Although the admission source and discharge location fields together identified 76% of LTCF-exposed patients compared to the H&P note, researchers and practitioners interested in using these variables to ascertain LTCF exposure should understand 2 important caveats. First, these data fields still missed most currently exposed LTCF patients, particularly those who had not been recently hospitalized. For projects aiming to identify current LTCF exposure, clinical note review would likely remain necessary. Second, identifying recent LTCF exposure from structured EHR data relies on “seeing” previous admissions and, in turn, the admission and discharge locations from these admissions. This visibility will vary by healthcare network size and geographic capture area. UMMS is a large network, increasing the likelihood that previous admissions were in-network and documented in our EHR. Even across our UMMS population, however, 22% of patients hospitalized in the previous 90 days had been hospitalized at a different UMMS hospital than for their index admission (data not shown). As these data underscore, patients do not always return to the same hospital, and had these previous admissions between out-of-network, our EHR would not have included them. When using the EHR admission and discharge location fields to identify LTCF-exposed inpatients in either smaller hospital networks or networks with less geographic capture, we would expect lower sensitivity compared to the H&P note than the 76% demonstrated in this study.
As a multi-center study spanning multiple years, this validation benefited from a large and diverse sample, increasing statistical power and generalizability. However, the retrospective study design that enabled these strengths also meant that our study is subject to several limitations, principally that we were unable to include a gold-standard comparator for assessing LTCF exposure (eg, family interview). Our study mirrors real-world infection control practice and clinical research, however, for which chart review generally constitutes the de facto reference standard for identifying LTCF-exposed patients. Reference Marchaim, Chopra and Bogan8,Reference Cunha, Kassakian and Chan24 Moreover, and importantly, the lack of a gold-standard comparator does not affect sensitivity comparisons between data sources (eg, between the H&P note and the admission source field). Nevertheless, because missingness of LTCF exposure in patient data may be non-random, calculating robust estimates of true LTCF exposure among inpatients would require prospective studies (eg, pilot deployments of intake questionnaires under quality improvement initiatives). Second, a review of clinical notes is labor- and time-intensive, and we prioritized reviewing index admission notes to maximize study sample size. Given resource constraints, we were unable to review H&P notes for hospitalizations that preceded the index admission or other note types. In practical effect, this decision decreased the sensitivity of the H&P notes for identifying any recent LTCF exposure, compared to the admission and discharge location fields from previous admissions. In future research, we plan to develop NLP classifiers for identifying LTCF exposure, which when executed across admissions and note types (and potentially patient addresses, cross-referenced to local LTCF addresses) should increase clinical note sensitivity without increasing human labor time. Third, LTCF definitions vary across research studies and data sources, for example, whether ALFs are included 25 or excluded. Reference Prabaker, Lin and McNally10,Reference Prabaker, Hayden, Weinstein and Lin17 Because human syntax is imprecise (eg, physicians may use the terms “assisted living” and “nursing home” interchangeably), we employed a broad LTCF definition that included ALFs and rehabilitation facilities. While this decision increased study internal validity for comparing the clinical notes to the structured EHR data fields, our validation results do not extend to narrower definitions of LTCF exposure. Finally, we selected a 90-day lookback period due to its clinical and infection control relevance, with research demonstrating that LTCF patients have an average MDRO colonization duration of approximately 3 months. Reference Lin, Huang, Pan, Chen and Li26 However, future validation of these EHR data fields for longer lookback periods would be an important extension of the current work.
Overall, in this multi-center validation study, we found that compared to the H&P note, the EHR admission source and discharge location fields identified three-quarters of inpatients with recent LTCF exposure but missed most patients with current LTCF exposure (eg, permanent LTCF residents). From a practical perspective, our findings suggest 3 take-home points for using EHR data to identify LTCF-exposed inpatients: (1) do not rely solely on the hospital “admission source” field, which misses most LTCF-exposed patients, (2) review the clinical notes to identify patients with current LTCF exposure, and (3) examine discharge locations from recent admissions to identify patients who were recently LTCF-exposed, even if reviewing the clinical notes. As technological advances ease the barriers to extracting information from free-text data, the ideal strategy for identifying LTCF-exposed inpatients will likely involve triangulating data from the unstructured notes and the structured admission source and discharge location fields. Given the importance of accurately ascertaining LTCF exposure, however, we also encourage hospitals to consider implementing intake questionnaires, with structured response fields, for documenting LTCF exposure in the EHR.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2024.37.
Financial support
This work was supported by funding from the Agency for Healthcare Research and Quality (K01HS028363 to K.E.G.) and by the University of Maryland Strategic Partnership: MPowering the State, a formal collaboration between the University of Maryland College Park and the University of Maryland Baltimore, awarded to K.E.G. and P.R.
Competing interests
A.D.H. reports receiving fees from UpToDate, outside the submitted work. All other study authors report no potential conflicts of interest.
Patient consent statement
This study does not include factors necessitating patient consent.
