Skip to main content Accessibility help

Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers

  • Shoukat H. Qari (a1), Hussain R. Yusuf (a2), Samuel L. Groseclose (a1), Mary R. Leinhos (a1) and Eric G. Carbone (a1)...



The US Centers for Disease Control and Prevention (CDC)-funded Preparedness and Emergency Response Research Centers (PERRCs) conducted research from 2008 to 2015 aimed to improve the complex public health emergency preparedness and response (PHEPR) system. This paper summarizes PERRC studies that addressed the development and assessment of criteria for evaluating PHEPR and metrics for measuring their efficiency and effectiveness.


We reviewed 171 PERRC publications indexed in PubMed between 2009 and 2016. These publications derived from 34 PERRC research projects. We identified publications that addressed the development or assessment of criteria and metrics pertaining to PHEPR systems and describe the evaluation methods used and tools developed, the system domains evaluated, and the metrics developed or assessed.


We identified 29 publications from 12 of the 34 PERRC projects that addressed PHEPR system evaluation criteria and metrics. We grouped each study into 1 of 3 system domains, based on the metrics developed or assessed: (1) organizational characteristics (n = 9), (2) emergency response performance (n = 12), and (3) workforce capacity or capability (n = 8). These studies addressed PHEPR system activities including responses to the 2009 H1N1 pandemic and the 2011 tsunami, as well as emergency exercise performance, situational awareness, and workforce willingness to respond. Both PHEPR system process and outcome metrics were developed or assessed by PERRC studies.


PERRC researchers developed and evaluated a range of PHEPR system evaluation criteria and metrics that should be considered by system partners interested in assessing the efficiency and effectiveness of their activities. Nonetheless, the monitoring and measurement problem in PHEPR is far from solved. Lack of standard measures that are readily obtained or computed at local levels remains a challenge for the public health preparedness field. (Disaster Med Public Health Preparedness. 2019;13:626-638)


Corresponding author

Correspondence and reprint requests to Shoukat H. Qari, Centers for Disease Control and Prevention, Office of Public Health Preparedness and Response, 1600 Clifton Rd. NE, MS K-72, Atlanta, GA 30333 (e-mail:


Hide All
1. US Centers for Disease Control and Prevention. Public Health Emergency Preparedness (PHEP) CooperativeAgreement. Accessed August 9, 2018.
2. US Centers for Disease Control and Prevention. Public Health Preparedness Capabilities: National Standards for State and Local Planning. Published March 2011. Accessed August 9, 2018.
3. Nelson, C, Lurie, N, Wasserman, J. Assessing public health emergency preparedness: concepts, tools, and challenges. Annu Rev Public Health. 2007;28:1-18.10.1146/annurev.publhealth.28.021406.144054
4. Altevogt, BM, Pope, AM, Hill, MN, et al. Research priorities in emergency preparedness and response for public health systems: a letter report. Washington DC: National Academies Press; 2008.
5. Qari, SH, Abramson, DM, Kushma, JA, et al. PERRCs: early returns on investment in evidence-based public health systems research. Public Health Rep. 2014;129(Suppl 4):1-4.10.1177/00333549141296S401
6. Leinhos, M, Qari, SH, Williams-Johnson, M. Preparedness and emergency response research centers: using a public health systems approach to improve all-hazards preparedness and response. Public Health Rep. 2014;129(Suppl 4):8-18.10.1177/00333549141296S403
7. Savoia, E, Rodday, AM, Stoto, MA. Public health emergency preparedness at the local level: results of a national survey. Health Serv Res. 2009a;44(5 Pt 2):1909-1924.10.1111/j.1475-6773.2009.01009.x
8. Leep, CJ. 2005 National profile of local health departments. J Public Health Manag Pract. 2006 Sep-Oct; 12(5):496-498.10.1097/00124784-200609000-00019
9. Davis, MV, Mays, GP, Bellamy, J, et al. Improving public health preparedness capacity measurement: development of the local health department preparedness capacities assessment survey. Disaster Med Public Health Prep. 2013 Dec;7(6):578-584. doi: 10.1017/dmp.2013.108.
10. Davis, MV, Bevc, CA, Schenck, AP. Declining trends in local health department preparedness capacities. Am J Public Health. 2014a Nov;104(11):2233-2238.10.2105/AJPH.2014.302159
11. Bevc, CA, Davis, MV, Schenck, AP. Temporal trends in local public health preparedness capacity. Front Public Health Serv Syst Res. 2014;3(3).
12. Davis, MV, Bevc, CA, Schenck, AP. Effects of performance improvement programs on preparedness capacities. Public Health Rep. 2014b;129(Suppl 4):19-27.10.1177/00333549141296S404
13. Dalnoki-Veress, F, McKallagat, C, Klebesadal, A. Local health department planning for a radiological emergency: an application of the AHP2 tool to emergency preparedness prioritization. Public Health Rep. 2014;129(Suppl 4):136-144.10.1177/00333549141296S418
14. Hunter, JC, Yang, JE, Petrie, M, et al. Integrating a framework for conducting public health systems research into statewide operations-based exercises to improve emergency preparedness. BMC Public Health. 2012a Aug 20;12:680.10.1186/1471-2458-12-680
15. Agboola, F, Bernard, D, Savoia, E, et al. Development of an online toolkit for measuring performance in health emergency response exercises. Prehosp Disaster Med. 2015;30(5):503-508.10.1017/S1049023X15005117
16. Glik, DC, Eisenman, DP, Donatello, I, et al. Reliability and validity of the Assessment for Disaster Engagement with Partners Tool (ADEPT) for local health departments. Public Health Rep. 2014; 129(Suppl 4):77-86.10.1177/00333549141296S411
17. Hunter, JC, Crawley, AW, Petrie, M, et al. Local public health system response to the tsunami threat in coastal California following the Tōhoku Earthquake. PLoS Curr. 2012b;4:e4f7f57285b804. doi: 10.1371/4f7f57285b804
18. Hunter, JC, Yang, JE, Crawley, AW, et al. Public health response systems in-action: learning from local health departments’ experiences with acute and emergency incidents. PLoS One. 2013 Nov 13; 8(11):e79457. doi:10.1371/journal.pone.0079457
19. Hunter, MD, Hunter, JC, Yang, JE, et al. Public health system response to extreme weather events. J Public Health Manag Pract. 2016 Jan-Feb;22(1):E1-10. doi: 10.1097/PHH.0000000000000204
20. Stoto, M. Measuring and assessing public health emergency preparedness. J Public Health Manag Pract. 2013;19(Suppl 2):S16-21.10.1097/PHH.0b013e318294b0e3
21. Zhang, Y, Lopez-Gatell, H, Alpuche-Aranda, CM, et al. Did advances in global surveillance and notification systems make a difference in the 2009 H1N1 pandemic?–A retrospective analysis. PLoS ONE. 2013;8(4):e59893.10.1371/journal.pone.0059893
22. Stoto, MA. The effectiveness of U.S. public health surveillance systems for situational awareness during the 2009 H1N1 pandemic: a retrospective analysis. PLoS ONE. 2012;7(8):e40984.10.1371/journal.pone.0040984
23. Klaiman, T, O’Connell, K, Stoto, M. Local health department public vaccination clinic success during 2009 pH1N1. J Public Health Manag Pract. 2013;19(4):E20-E26.10.1097/PHH.0b013e318269e434
24. Klaiman, T, O’Connell, K, Stoto, MA. Learning from successful school-based vaccination clinics during 2009 pH1N1. J Sch Health. 2014;84:63-69.10.1111/josh.12119
25. Piltch-Loeb, R, Kraemer, JD, Nelson, C, et al. A public health emergency preparedness critical incident registry. Biosecur Bioterror. 2014a May-Jun;12(3):132-143. doi: 10.1089/bsp.2014.0007
26. Piltch-Loeb, RN, Nelson, CD, Kraemer, JD, et al. A peer assessment approach for learning from public health emergencies. Public Health Rep. 2014b;129(Suppl 4):28-34.10.1177/00333549141296S405
27. Stoto, MA, Nelson, C, Higdon, MA, et al. Lessons about the state and local public health system response to the 2009 H1N1 pandemic: a workshop summary. J Public Health Manag Pract. 2013 Sep-Oct; 19(5):428-435. doi:10.1097/PHH.0b013e3182751d3e
28. Savoia, E, Agboola, F, Biddinger, PD. Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness. Int J Environ Res Public Health. 2012 Aug ; 9(8):2949-2963. doi: 10.3390/ijerph9082949. Epub 2012 Aug 16.
29. Barnett, DJ, Balicer, RD, Thompson, CB, et al. 2009 assessment of local public health workers’ willingness to respond to pandemic influenza through application of the extended parallel process model. PLoS ONE. 2009;4(7):e6365.10.1371/journal.pone.0006365
30. Witte, K. Putting the fear back into fear appeals: The extended parallel process model. Commun Monogr. 1992;59:329-349.10.1080/03637759209376276
31. Barnett, DJ, Thompson, CB, Errett, NA, et al. Determinants of emergency response willingness in the local public health workforce by jurisdictional and scenario patterns: a cross-sectional survey. BMC Public Health. 2012;12:164.10.1186/1471-2458-12-164
32. Barnett, DJ, Thompson, CB, Semon, NL, et al. EPPM and willingness to respond: the role of risk and efficacy communication in strengthening public health emergency response systems. Health Commun. 2014;29(6):598-609.10.1080/10410236.2013.785474
33. Errett, NA, Barnett, DJ, Thompson, CB, et al. Assessment of psychological preparedness and emergency response willingness of local public health department and hospital workers. Int J Emerg Ment Health. 2012;14(2):125-133.
34. Savoia, E, Massin‐Short, S, Higdon, MA, et al. A toolkit to assess Medical Reserve Corps Units’ performance. Disaster Med Public Health Prep. 2010;4(3):213-219.10.1001/dmp.2010.17
35. Horney, JA, Markiewicz, M, Meyer, AM. Regional public health preparedness teams in North Carolina: an analysis of their structural capacity and impact on services provided. Am J Disaster Med. 2011 Mar-Apr;6(2):107-117.10.5055/ajdm.2011.0050
36. Schuh, RG, Eichelberger, TR, Stebbins, S, et al. Developing a measure of local agency adaptation to emergencies: a metric. Eval Program Plann. 2012;35(4):473-480.10.1016/j.evalprogplan.2012.02.003
37. Potter, MA, Schuh, RG, Pomer, B, et al. The adaptive response metric: toward an all hazards tool for planning, decision support, and after‐action analytics. J Public Health Manag Pract. 2013 Sep-Oct;19(Suppl 2):S49-54. doi:10.1097/PHH.0b013e318296214c
38. Asch, SM, Stoto, M, Mendes, M, et al. A review of instruments assessing public health preparedness. Public Health Rep. 2005;120(5):532-542.10.1177/003335490512000508
39. Kirschenbaum, A. Measuring the effectiveness of disaster management organizations. Int J Mass Emerg Disasters. 1994;22(1):75-102.
40. Links, JM, Schwartz, BS, Lin, S, et al. COPEWELL: a conceptual framework and system dynamics model for predicting community functioning and resilience. Disaster Med Public Health Prep. 2017 Jun 21:1-11. doi: 10.1017/dmp.2017.39
41. Savoia, E, Agboola, F, Biddinger, PD. A conceptual framework to measure systems’ performance during emergency preparedness exercises. Int J Environ Res Public Health. 2014 Sep 17;11(9):9712-9722. doi: 10.3390/ijerph110909712
42. Jhung, MA, Swerdlow, D, Olsen, SJ, et al. Epidemiology of 2009 pandemic influenza A (H1N1) in the United States. Clin Infect Dis. 2011;52(Suppl 1):S13-26.


Related content

Powered by UNSILO

Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers

  • Shoukat H. Qari (a1), Hussain R. Yusuf (a2), Samuel L. Groseclose (a1), Mary R. Leinhos (a1) and Eric G. Carbone (a1)...


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.