Hostname: page-component-848d4c4894-v5vhk Total loading time: 0 Render date: 2024-06-14T02:59:00.716Z Has data issue: false hasContentIssue false

Measuring the Efficacy of a Pilot Public Health Intervention for Engaging Communities of Puerto Rico to Rapidly Write Hurricane Protection Plans

Published online by Cambridge University Press:  26 November 2020

Mark E. Keim*
DisasterDoc LLC, Atlanta, Georgia
Laura A. Runnels
LARC Consulting LLC, Pittsburgh, Pennsylvania
Alexander P. Lovallo
DisasterDoc LLC, Atlanta, Georgia
Margarita Pagan Medina
Puerto Rico Department of Health, San Juan, Puerto Rico
Eduardo Roman Rosa
Puerto Rico Department of Health, San Juan, Puerto Rico
Maximiliano Ramery Santos
Puerto Rico Department of Health, San Juan, Puerto Rico
Mollie Mahany
National Center for Environmental Health, Centers for Disease Control and Prevention, Atlanta, Georgia
Miguel A. Cruz
National Center for Environmental Health, Centers for Disease Control and Prevention, Atlanta, Georgia
Correspondence: Mark E. Keim, MD, MBA 141 Chantilly Lane Lawrenceville, GeorgiaUSA30043 E-mail:



The efficacy is measured for a public health intervention related to community-based planning for population protection measures (PPMs; ie, shelter-in-place and evacuation).


This is a mixed (qualitative and quantitative) prospective study of intervention efficacy, measured in terms of usability related to effectiveness, efficiency, satisfaction, and degree of community engagement.


Two municipalities in the Commonwealth of Puerto Rico are included.


Community members consisting of individuals; traditional leaders; federal, territorial, and municipal emergency managers; municipal mayors; National Guard; territorial departments of education, health, housing, public works, and transportation; health care; police; Emergency Medical Services; faith-based organizations; nongovernmental organizations (NGOs); and the private sector.


The intervention included four community convenings: one for risk communication; two for plan-writing; and one tabletop exercise (TTX). This study analyzed data collected from the project work plan; participant rosters; participant surveys; workshop outputs; and focus group interviews.

Main Outcome Measures:

Efficacy was measured in terms of ISO 9241-11, an international standard for usability that includes effectiveness, efficiency, user satisfaction, and “freedom from risk” among users. Degree of engagement was considered an indicator of “freedom from risk,” measurable through workshop attendance.


Two separate communities drafted and exercised ~60-page-long population protection plans, each within 14.5 hours. Plan-writing workshops completed 100% of plan objectives and activities. Efficiency rates were nearly the same in both communities. Interviews and surveys indicated high degrees of community satisfaction. Engagement was consistent among community members and variable among governmental officials.


Frontline communities have successfully demonstrated the ability to understand the environmental health hazards in their own community; rapidly write consensus-based plans for PPMs; participate in an objective-based TTX; and perform these activities in a bi-lingual setting. This intervention appears to be efficacious for public use in the rapid development of community-based PPMs.

Original Research
© The Author(s), 2020. Published by Cambridge University Press on behalf of the World Association for Disaster and Emergency Medicine

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Intergovernmental Panel on Climate Change (IPCC). Climate Change 2007: Impacts, Adaptation, and Vulnerability. New York USA: Cambridge University Press; 2007.Google Scholar
O’Brien, G, O’Keefe, P, Rose, J, Wisner, B. Climate change and disaster management. Disasters. 2006;30(1):6480.CrossRefGoogle ScholarPubMed
Thomalla, F, Downing, T, Spanger-Siegfried, E, Han, G, Rockstrom, J. Reducing hazard vulnerability: towards a common approach between disaster risk reduction and climate adaptation. Disasters. 2006;30(1):3948.CrossRefGoogle ScholarPubMed
Schipper, L, Pelling, M. Disaster risk, climate change and international development: scope for, and challenges to, integration. Disasters. 2006;30(1):1938.10.1111/j.1467-9523.2006.00304.xCrossRefGoogle ScholarPubMed
National Academies of Sciences, Engineering, and Medicine (NASEM). Exploring disaster risk reduction through community-level approaches to promote healthy outcomes: proceedings of a workshop. NASEM: 2017. Accessed August 2, 2020.Google Scholar
Institute of Medicine. Engaging the Public in Critical Disaster Planning and Decision Making: Workshop Summary. Washington, DC USA: The National Academies Press; 2013. Accessed August 2, 2020.Google Scholar
Wilson, E. Research Methods in Health Social Sciences. Singapore: Springer; 2018:1–15.Google Scholar
Brownson, RC, Eyler, AA, Harris, JK, Moore, JB, Tabak, RG. Getting the word out: new approaches for disseminating public health science. JPHMP. 2018;24(2):102111.Google ScholarPubMed
Kirsch, TD, Keim, M. US governmental spending for disaster-related research, 2011-2016: characterizing the state of science funding across 5 professional disciplines. Disaster Med Public Health Prep. 2019;13(5-6):912919.CrossRefGoogle ScholarPubMed
Keim, M, Kirsch, T, Alleyne, O, et al. The need for a national strategy to assess and reduce disaster-related mortality in the US. AJPH. 2019;109(4):539540.10.2105/AJPH.2019.304975CrossRefGoogle Scholar
Environmental Disasters, Keim M.. In: Frumkin, H, (ed). Environmental Health from Global to Local. Third Ed. San Francisco, California USA: John Wiley and Sons; 2016:667692.Google Scholar
Kishore, N, Mahmud, A, Kiang, M, et al. Mortality in Puerto Rico after Hurricane Maria. NEJM. 2018;379(2):162170.CrossRefGoogle ScholarPubMed
Webb Hooper, M, Napoles, AM, Pérez-Stable, EJ. COVID-19 and racial/ethnic disparities. JAMA. 2020;323(24):24662467.10.1001/jama.2020.8598CrossRefGoogle ScholarPubMed
Vanagas, G, Bala, M, Lhachimi, SK. Evidence-Based Public Health. Biomed Res Int. 2017:2607397.Google ScholarPubMed
Brownson, RC, Fielding, JE, Maylahn, CM. Evidence-based public health: a fundamental concept for public health practice. Ann Rev Public Health. 2009;30:175201.CrossRefGoogle ScholarPubMed
DeSalvo, KB, Wang, YC, Harris, A, Auerbach, J, Koo, D, O’Carroll, P. Public Health 3.0: a call to action for public health to meet the challenges of the 21st century. Prev Chronic Dis. 2017;14:170017.10.5888/pcd14.170017CrossRefGoogle Scholar
Lurie, N, Manolio, T, Patterson, AP, Collins, F, Frieden, T. Research as a part of public health emergency response. NEJM. 2013;368(13):12511255.CrossRefGoogle ScholarPubMed
Institute of Medicine. Enabling Rapid and Sustainable Public Health Research During Disasters: Summary of a Joint Workshop by the Institute of Medicine and the U.S. Department of Health and Human Services. Washington, DC USA: National Academies Press; 2015.Google Scholar
Ågerfalk, PJ, Eriksson, O. Socio–instrumental usability: IT is all about social action. J Inf Technol. 2006;21(1):2439.CrossRefGoogle Scholar
International Organization of Standards (ISO). Ergonomics of Human-System Interaction. Geneva, Switzerland: ISO; 2018.Google Scholar
Victora, CG, Habicht, JP, Bryce, J. Evidence-based public health: moving beyond randomized trials. AJPH. 2004;94(3):400405.CrossRefGoogle ScholarPubMed
Keim, ME. An innovative approach to capability-based emergency operations planning. Disaster Health. 2013;1(1):5462.10.4161/dish.23480CrossRefGoogle ScholarPubMed
Keim, ME. O2C3: a unified model for emergency operations planning. Am J Disaster Med. 2010;5(3):169179.CrossRefGoogle ScholarPubMed
Bazeyo, W, Mayega, RW, Orach, GC, et al. Regional approach to building operational level capacity for disaster planning: the case of the Eastern Africa region. East Afr J Public Health. 2013;10(2):447458.Google ScholarPubMed
Orach, GC, Mamuya, S, Mayega, RW, et al. Use of the automated disaster and emergency planning tool in developing district level public health emergency operating procedures in three East African countries. East Afr J Public Health. 2013;10(2):439446.Google ScholarPubMed
Yi, H, Za, Y, Fan, W, et al. Public health preparedness for the world’s largest mass gathering: 2010 World Exposition in Shanghai, China. Prehosp Disaster Med. 2012;27(6):589594.CrossRefGoogle ScholarPubMed
US Department of Homeland Security. Planning Considerations: Evacuation and Shelter-in-Place Guidance for State, Local, Tribal, and Territorial Partners. Washington, DC USA: US Department of Homeland Security; 2019.Google Scholar
Emergency Management Agency, Federal. Facilitator and Evaluator Handbook, Whole Community Exercise: Hurricane. Washington, DC USA: FEMA; 2020.Google Scholar
Secon HWA. A map of the US cities and states under lockdown — and those that are reopening. Business Insider. May 8, 2020. Accessed August 2, 2020.Google Scholar
Cobb, J. Protagonist-driven urban ethnography. City & Community. 2015;14(4):4.10.1111/cico.12136CrossRefGoogle Scholar
Rychetnik, L, Frommer, M, Hawe, P, Shiell, A. Criteria for evaluating evidence on public health interventions. J Epidemiol Comm Health. 2002;56(2):119127.CrossRefGoogle ScholarPubMed