Skip to main content Accessibility help
×
Home

Enhancing Reporting of After Action Reviews of Public Health Emergencies to Strengthen Preparedness: A Literature Review and Methodology Appraisal

  • Robert Davies (a1), Elly Vaughan (a2), Graham Fraser (a3), Robert Cook (a2), Massimo Ciotti (a3) and Jonathan E. Suk (a3)...

Abstract

Objective

This literature review aimed to identify the range of methods used in after action reviews (AARs) of public health emergencies and to develop appraisal tools to compare methodological reporting and validity standards.

Methods

A review of biomedical and gray literature identified key approaches from AAR methodological research, real-world AARs, and AAR reporting templates. We developed a 50-item tool to systematically document AAR methodological reporting and a linked 11-item summary tool to document validity. Both tools were used sequentially to appraise the literature included in this study.

Results

This review included 24 highly diverse papers, reflecting the lack of a standardized approach. We observed significant divergence between the standards described in AAR and qualitative research literature, and real-world AAR practice. The lack of reporting of basic methods to ensure validity increases doubt about the methodological basis of an individual AAR and the validity of its conclusions.

Conclusions

The main limitations in current AAR methodology and reporting standards may be addressed through our 11 validity-enhancing recommendations. A minimum reporting standard for AARs could help ensure that findings are valid and clear for others to learn from. A registry of AARs, based on a common reporting structure, may further facilitate shared learning. (Disaster Med Public Health Preparedness. 2019;13:618-625)

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Enhancing Reporting of After Action Reviews of Public Health Emergencies to Strengthen Preparedness: A Literature Review and Methodology Appraisal
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Enhancing Reporting of After Action Reviews of Public Health Emergencies to Strengthen Preparedness: A Literature Review and Methodology Appraisal
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Enhancing Reporting of After Action Reviews of Public Health Emergencies to Strengthen Preparedness: A Literature Review and Methodology Appraisal
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

Correspondence and reprint requests to Dr Jonathan E. Suk, European Centre for Disease Prevention and Control (ECDC), Gustav III:s Boulevard 40, 16973 Solna, Sweden (e-mail: Jonathan.Suk@ecdc.europa.eu).

References

Hide All
1. ECDC. Meeting report: ad hoc advisory meeting on preparedness Stockholm, 15-16 May 2014. Stockholm: European Centre for Disease Prevention and Control. 2014. http://ecdc.europa.eu/en/publications/Publications/preparedness-meeting-2014.pdf. Accessed May 25, 2018.
2. Stoto, MA. Measuring and assessing public health emergency preparedness. J Public Health Manag Pract. 2013;19:S16S21.
3. Geary, K. (International SOS). Personal communication. Rob Davies (Bazian Ltd). 2015.
4. Flanaghan, JC. The critical incident technique. Psychol Bull. 1954;51(4):327358.
5. Piltch-Loeb, RN, Nelson, C, Kraemer, J, et al. Peer assessment of public health emergency response toolkit. Cambridge (MA): Harvard School of Public Health. 2014. https://blogs.commons.georgetown.edu/lamps/files/2013/10/Peer-Assessment-toolkit-Updated-1-17-14.pdf. Accessed May 25, 2018.
6. Piltch-Loeb, RN, Nelson, CD, Kraemer, JD, et al. A peer assessment approach for learning from public health emergencies. Improv Sys Pract. 2014; 129(Suppl 4):2834.
7. Schwester, RW. Handbook of Critical Incident Analysis. New York (NY): Routledge (Taylor & Francis Group). 2014.
8. Serrat, O. The critical incident technique. Washington, DC: Asian Development Bank. 2010.
9. Stoto, MA. Getting from what to why: using qualitative methods in public health systems research. Orlando (FL): PHSR IG Methods Panel. 2012. http://www.academyhealth.org/files/phsr/Qual-Stoto.pdf. Accessed May 26, 2018.
10. ECDC. Best practices in ranking emerging infectious disease threats: a literature review. Stockholm European Centre for Disease Prevention and Control. 2015. http://ecdc.europa.eu/en/publications/Publications/emerging-infectious-disease-threats-best-practices-ranking.pdf. Accessed May 26, 2018.
11. O’Brien, EC, Taft, R, Geary, K, et al. Best practices in ranking communicable disease threats: a literature review, 2015. Euro Surveill. 2016; 21(17):pii=30212.
12. Woloshynowych, N, Rogers, S, Taylor-Adams, S, et al. The investigation and analysis of critical incidents and adverse events in healthcare. Health Technol Assess. 2005; 9(19):1143, iii.
13. CGOES. Standardized emergency management system: after action report. California: California Governor’s Office of Emergency Services. 2013. http://www.caloes.ca.gov/PlanningPreparednessSite/Documents/Local%20Government%20After%20Action%20Report%20Template.doc. Accessed May 26, 2018.
14. HSEEP. After-action report/improvement plan. Washington (DC): Homeland Security Exercise and Evaluation Program. 2013. http://www.in.gov/dhs/files/AAR-IP_Template_Apr-13-2_Clean.docx. Accessed May 26, 2018.
15. Taylor-Adams, S, Vincent, C. Systems analysis of clinical incidents: the London protocol. Imperial College London. 2004. https://www1.imperial.ac.uk/resources/C85B6574-7E28-4BE6-BE61-E94C3F6243CE/londonprotocol_e.pdf. Accessed May 27, 2018.
16. ISOS. Significant event analysis reporting form. London: International SOS. 2015.
17. MDU. Medico-legal guide to serious event analysis. London: Medical Defence Union. 2014. http://www.themdu.com/~/media/files/mdu/publications/guides/significant%20event%20analysis/medico_-_legal_guide_to_significant_event_analysis.pdf. Accessed May 27, 2018.
18. NPSA. A quick guide to conducting a significant event audit. London: National Patient Safety Agency. 2008. http://www.nrls.npsa.nhs.uk/EasySiteWeb/getresource.axd?AssetID=61502&type=full&servicetype=Attachment. Accessed May 27, 2018.
19. NPSA. Significant event audit: guidance for primary care teams. London: National Patient Safety Agency. 2008. http://www.nrls.npsa.nhs.uk/EasySiteWeb/getresource.axd?AssetID=61501. Accessed May 27, 2018.
20. HIROC. Critical incidents and multi-patient events: risk resource guide. Toronto (ON): Healthcare Insurance Reciprocal of Canada. 2015. https://www.hiroc.com/getmedia/c110b394-c5a6-4e93-80b3-73da14436dbd/HIROC-Management-of-Critical-Incidents-April-2015.pdf.aspx?ext=.pdf. Accessed May 27, 2018.
21. Masotti, P, Green, ME, Birtwhistle, R, et al. PH1N1: a comparative analysis of public health responses in Ontario to the influenza outbreak, public health and primary care: lessons learned and policy suggestions. BMC Public Health. 2013; 13(1):687.
22. DSB. New influenza A virus (H1N1): a summary of a study on the national response in Norway. Tonsberg. 2011. http://www.dsb.no/Global/Publikasjoner/2011/Rapport/summary_new%20influenza_virus_H1N1.pdf. Accessed May 25, 2018
23. Socialstyrelsen, A(H1N1) 2009: an evaluation of Sweden’s preparations for and management of the pandemic. Stockholm. 2011. http://www.socialstyrelsen.se/Lists/Artikelkatalog/Attachments/18398/2011-8-4.pdf. Accessed May 25, 2018.
24. Hine, D. The 2009 influenza pandemic: an independent review of the UK response to the 2009 influenza pandemic. London. 2010. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/61252/the2009influenzapandemic-review.pdf. Accessed May 26, 2018.
25. WHO. Global survey on national vaccine deployment and vaccination plans for pandemic A(H1N1) 2009 vaccine – 2010: report of findings. Geneva. 2013. http://www.who.int/influenza_vaccines_plan/resources/2010_H1N1_NVDP_WHO_Survey.pdf. Accessed May 26, 2018.
26. EC. Assessment report on EU-wide pandemic vaccine strategies. Brussels. 2010. http://ec.europa.eu/health/communicable_diseases/docs/assessment_vaccine_en.pdf. Accessed May 26, 2018.
27. HPA. Assessment report on the EU-wide response to pandemic (H1N1) 2009. London. 2010. http://ec.europa.eu/health/communicable_diseases/docs/assessment_response_en.pdf. Accessed May 26, 2018.
28. WHO. Recommendations for good practice in pandemic preparedness: identified through evaluation of the response to pandemic (H1N1) 2009. Copenhagen. 2010. http://www.euro.who.int/__data/assets/pdf_file/0017/128060/e94534.pdf. Accessed May 28, 2018.
29. MEMA. After action report for the response to the 2013 Boston marathon bombings. Framingham (MA). 2014. http://www.mass.gov/eopss/docs/mema/after-action-report-for-the-response-to-the-2013-boston-marathon-bombings.pdf. Accessed May 28, 2018.
30. Goralnick, E, Halpern, P, Loo, S, et al. Leadership during the Boston marathon bombings: a qualitative after-action review. Disaster Med Public Health Prep. 2015;9(10):6.
31. Little, M, Cooper, J, Gope, M, et al. “Lessons learned”: a comparative case study analysis of an emergency department response to two burns disasters. Emerg Med Australas. 2012;24(4):420429.
32. Aylwin, CJ, König, TC, Brennan, NW, et al. Reduction in critical mortality in urban mass casualty incidents: analysis of triage, surge, and resource use after the London bombings on July 7, 2005. Lancet. 2006;368(9554):22192225.
33. OEM. After action report: Alfred P. Murrah Federal Building bombing 19 April 1995 in Oklahoma City, Oklahoma. Oklahoma (OK). 1995. https://www.ok.gov/OEM/documents/Bombing%20After%20Action%20Report.pdf. Accessed May 28, 2018.
34. HSE. The Buncefield incident 11 December 2005: the final report of the major incident investigation board volume 2. Liverpool. 2008. http://www.hse.gov.uk/comah/buncefield/miib-final-volume2a.pdf.
35. HSE. The Buncefield incident 11 December 2005: the final report of the major incident investigation board volume 1. Liverpool. 2008. http://www.hse.gov.uk/comah/buncefield/miib-final-volume1.pdf.
37. SQW. Buncefield social impact assessment: annex A key information sources. London. 2007. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/78984/buncefield-social-impact-assessment-final-report-annexes.pdf.
38. Tapster, C. Buncefield: multi-agency debrief report and recommendations. Hertford. 2007. http://www.hertsdirect.org/infobase/docs/pdfstore/bunrepdebrief.pdf.
39. Paltrinieri, N, Dechy, N, Salzano, E, et al. Lessons learned from Toulouse and Buncefield disasters: from risk analysis failures to the identification of atypical scenarios through a better knowledge management. Risk Anal. 2012;32(8):14041419.
40. Knox, CC. Analyzing after-action reports from Hurricanes Andrew and Katrina: repeated, modified, and newly created recommendations. J Emerg Manage. 2013;11(2):160168.
41. Brevard, SB, Weintraub, SL, Aiken, JB, et al. Analysis of disaster response plans and the aftermath of Hurricane Katrina: lessons learned from a level I trauma center. J Trauma Injury Infect Crit Care. 2008;65(5):11261132.
42. Terenzini, C. The report of the Blue Ribbon Committee on the water emergency of April 25-27, 2007 in Spencer, Massachusetts. Spencer (MA). 2007. http://www.spencerfire.org/WaterEmergency.pdf.
43. Adrot, A. Crisis response, organizational improvisation and the dispassionate communicative genre during the 2003 French heat wave. Paris. 2011. https://basepub.dauphine.fr/bitstream/handle/123456789/7969/Adrot_aim2011.PDF?sequence=1.
45. SIESWE. Evaluation of an innovative method of assessment: critical incident analysis. Glasgow: Scottish Institute for Excellence in Social Work Education. 2005. https://pure.strath.ac.uk/portal/files/11177868/sieswe_nam_evaluation_critical_incident_analysis_2005_02.pdf.
46. Berry, K, Krizek, B. Root cause analysis in response to a “near miss.” J Healthc Qual. 2000;22(2):1618.
47. Iedema, R, Jorm, C, Braithwaite, J. Managing the scope and impact of root cause analysis recommendations. J Health Organ Manag. 2008;22(6):569585.
48. Singleton, CM, Debastiani, S, Rose, D, et al. An analysis of root cause identification and continuous quality improvement in public health H1N1 after-action reports. J Public Health Manage Pract. 2014;20(2):197204.
49. Aledort, JE, Lurie, N, Ricci, KA, et al. Facilitated look-backs: a new quality improvement tool for management of routine annual and pandemic influenza. RAND Corporation. 2006. http://www.rand.org/pubs/technical_reports/TR320.html.
50. OFSTED. Learning lessons from serious case reviews 2009-2010: Ofsted’s evaluation of serious case reviews from 1 April 2009 to 31 March 2010. The Office for Standards in Education, Children’s Services and Skills. 2010. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/381110/Learning_20lessons_20from_20serious_20case_20reviews_202009-2010.pdf.
51. Savoia, E, Agboola, F, Biddinger, PD. Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness. Int J Environ Res Public Health. 2012;9:14.

Keywords

Type Description Title
WORD
Supplementary materials

Davies et al. supplementary material
SI1

 Word (14 KB)
14 KB
WORD
Supplementary materials

Davies et al. supplementary material
SI2

 Word (17 KB)
17 KB
WORD
Supplementary materials

Davies et al. supplementary material
SI3

 Word (41 KB)
41 KB
WORD
Supplementary materials

Davies et al. supplementary material
SI4

 Word (16 KB)
16 KB

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed