Skip to main content Accessibility help
×
Home

Contents:

Information:

  • Access
  • Cited by 5

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology
        Available formats
        ×
Export citation

To the Editor—A number of hand hygiene monitoring technology (HHMT) options have become commercially available with varying degrees of capability, reflecting a growing market without clear evidence of the impact these systems have on hand hygiene (HH). A systematic review including 42 articles mentioning automated measurement found that fewer than 20% of studies identified calculations for accuracy; among them, the level of rigor was highly variable. 1 Manufacturers’ accuracy assessments are often inadequate and vary with physical location. Given increasing pressure from accreditation bodies to measure HH, 2 a methodology is needed to assess HHMT efficacy in a standardized way, allowing for comparison both with the gold standard, direct observation, and with each other. Validation of HHMT must be tested in actual clinical practice to avoid overestimation or underestimation of accuracy. 3 We propose a rigorous method for validation of HHMT in a medical setting.

HH compliance can be measured in a number of ways. A patient-encounter–based approach measures compliance from just before room entry to just after room exit. This approach requires the observation of the entire patient encounter, which may prove valuable for detection of healthcare worker (HCW) patterns. Alternatively, an independent-event approach treats room entry and exit as separate events, maximizing observer efforts. For HHMT validation, we recommend an extremely independent-event–based approach: each room entry, exit, soap dispenser actuation, and alcohol-based hand rub (ABHR) dispenser actuation should be treated as independent events. This method allows for real-time identification of inaccuracies.

HHMT requires significant investment of resources in proper calibration. Many systems use room “zones” that must be properly calibrated to capture room entries and exits while distinguishing a person walking in a hallway from room activity. In the most sophisticated systems, the interaction between an individual’s identification device and the entry/exit sensor adds additional complexity. Once calibration and structural assessment are deemed acceptable, the basic functionality of the system should be tested using a planned path.

A planned path is a route created throughout a clinical area that allows for systematic validation of HHMT through purposeful activation of every device. In this approach, a planned path is developed for each area, accounting for all soap dispensers, ABHR dispensers, and monitored rooms (Figure 1a). An investigator follows the planned path, activating each device while documenting deviations from the path. Detailed notes are essential for comparison of performed activity with system data. For example, if a room is skipped, it should be noted. Thus, investigators must consider each device activation as a unique encounter.

Figure 1 Comparison of a planned path to natural healthcare provider behavior.

After executing the planned path, data are pulled from the HHMT in the most raw format available. Devices often have a unique identifier, allowing for the association of each event with a specific device or location. Each device encounter recorded by direct observation is compared to data recorded by the system. This approach is valuable for both aggregate and individualized HHMT because it accurately quantifies both activity and attributes of that activity for comparison with performed behavior. For example, a system capable of monitoring individual behavior is assessed for its ability to both document room activity and attribute that activity to the correct individual. There are likely to be inaccuracies identified that require repositioning of devices. After these adjustments, devices can be assessed again using the planned path.

While the planned path quantifies a system’s accuracy in detecting purposeful behavior, behavioral validation quantifies a system’s ability to accurately detect real-world behaviors, which can be chaotic and unpredictable (Figure 1b). Behavioral validation requires trained observers to document all room activity, dispenser actuations, and, for individualized systems, the individual performing each behavior. Observers document unusual behaviors (eg, a patient requiring immediate resuscitative measures) or those that pose difficulty for the technology. For instance, many HHMT observers struggle to accurately distinguish multiple, simultaneous room entries/exits. Thus, when groups enter a patient room, it should be documented.

Once investigators have reached the behavioral validation step of this protocol, basic functionality of the system has been deemed acceptable through the planned path. This allows flexibility in observing natural behavior, with the goal of observing a representative sample of HCWs, locations, and time of day. However, this step should be purposefully planned and systematically executed, ensuring standardization of observations and documentation. Each device activation is again considered a unique encounter with observed behaviors compared to system data.

Accuracy of HHMT can be calculated using fundamental epidemiologic approaches. Sensitivity of HHMT is calculated using direct observation as the accurate data source. This allows identification of true-positive events (those both captured through direct observation and counted by HHMT), false-positive events (events that are not observed but are counted by HHMT), and false-negative events (those which are observed but are not captured by HHMT). Because it would not be feasible to quantify the number of nonevents (ie, how many times a person did not walk into a room or did not perform HH), true-negative events are not captured in this approach. Sensitivity and positive predictive value are then calculated as the measures of system accuracy.

A standardized approach to validating HHMT is the next logical step to establishing standards that allow for comparison of available technologies by the healthcare industry. We recommend the following steps: (1) manufacturers should share the burden of performing validation steps with hospitals; (2) hospitals should require transparency of any accuracy data provided by manufacturers; and (3) either the hospital or a third party should be actively involved in assessing accuracy of HHMT in each physical location where it is installed. The protocol described above provides a method for moving toward these recommendations.

Acknowledgments

We would like to acknowledge the ongoing contributions of the Hand Hygiene Leadership Committee at the University of Chicago Medicine. Emily Landon has been an invited speaker for Proventix, EcoLab, and GOJO. Dr. Landon has also provided consulting services for Proventix.

Financial support: No financial support was provided relevant to this article.

Potential conflicts of interest: Emily Landon has been an invited speaker for Proventix, EcoLab, and GOJO. Dr. Landon has also provided consulting services for Proventix. All other authors have no conflicts of interest or financial disclosures to report.

References

1. Ward, MA, Schweizer, ML, Polgreen, PM, Gupta, K, Reisinger, HS, Perencevich, EN. Automated and electronically assisted hand hygiene monitoring systems: a systematic review. Am J Infec Control 2014 May;42:472478.
2. National Patient Safety Goal: Monitoring of Hand Hygiene. Goal 7- 07.01.01. Comprehensive Accreditation Manual for Hospitals Standards FAQs. Joint Commission website. http://www.jointcommission.org/assets/1/6/2015_NPSG_HAP.pdf. Accessed February 3, 2015.
3. Pineles, LA, Morgan, DJ, Limper, HM, et al. Accuracy of a radiofrequency identification (RFID) badge system to monitor hand hygiene behavior during routine clinical activities. Ame J Infect Control 2014;42:144147.