Skip to main content Accessibility help

Inter-Agency Communication and Operations Capabilities during a Hospital Functional Exercise: Reliability and Validity of a Measurement Tool

  • Elena Savoia (a1), Paul D. Biddinger (a1) (a2), Jon Burstein (a1) (a3) and Michael A. Stoto (a1) (a4)



As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise.


The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement.


Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self-reported scores and external evaluators' scores ranged from moderate to good.


The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.


Corresponding author

Center for Public Health PreparednessDivision of Public Health PracticeHarvard School of Public Health677 Huntington AvenueBoston, MassachusettsUSA E-mail:


Hide All
1.Dausey, DJ, Buehler, JW, Lurie, N: Designing and conducting tabletop exercises to assess public health preparedness for manmade and naturally occurring biological threats. BMC Public Health 2007;7:92.
2.Asch, SM, Stoto, MA, Mendes, M, Valdez, M, Gallagher, ME, Halverson, P, Lurie, N: A review of instruments assessing public health preparedness. Public Health Rep 2005;120(5):532542.
3.Kaji, AH, Lewis, RJ: Assessment of the reliability of the Johns Hopkins/Agency for Healthcare Research and Quality hospital disaster drill evaluation tool. Ann Emerg Med 2008;52(3):204210.
4.Savoia, E, Testa, MA, Biddinger, PD, et al. : Assessing public health capabilities during emergency preparedness tabletop exercises: Reliability and validity of a measurement tool. Public Health Rep 2009;124(1):138148
5.Biddinger, PD, Cadigan, R, Auerbacj, B, et al. : Using tabletop exercises to identify systems-level changes for improving preparedness. Public Health Rep 2008;123(1):96101.
6.Cronbach, LJ: Coefficient alpha and the internal structure of tests. Psykometrika 1951;16:297333.
7.Hair, JF, Anderson, RE, Tatham, RL, et al. : Multivariate Analysis: With Readings. 4th ed.Englewood Cliffs, New Jersey: Prentice-Hall, 1995.
8.Duteman, GH: Principal Components Analysis. Newbury Park, California: Sage Publications, 1989.
9.SPSS for Windows, Release 16.0.1 2007. Chicago: SPSS Inc.
10.Agency for Healthcare Research and Quality: Optimizing Surge Capacity: Hospital Assessment and Planning. Bioterrorism and Health System Preparedness, Issue Brief No.3. AHRQ Publication No. 04-P008. Available at


Related content

Powered by UNSILO

Inter-Agency Communication and Operations Capabilities during a Hospital Functional Exercise: Reliability and Validity of a Measurement Tool

  • Elena Savoia (a1), Paul D. Biddinger (a1) (a2), Jon Burstein (a1) (a3) and Michael A. Stoto (a1) (a4)


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.