Introduction
Over the last century, the number of disasters has increased,Reference Schiermeier1, Reference Bartels and VanRooyen2 indicating that future generations of physicians will be called upon to provide mass-casualty treatment to an even greater extent than before. A number of studies point to the lack of adequate training in the medical management of disaster response, a deficiency that has become apparent in the recent past.Reference Rosborough3, Reference Auf der Heide4 As a consequence, many governments and scientific institutions agree that disaster medicine education should be included in the standard medical curriculum.5–Reference Pelaccia17
The importance of enhancing education and training in disaster medicine has widely been perceived by medical students.Reference Kaji, Coates and Fung11, Reference Scott, Carson and Greenwell13, Reference Kaiser, Barnett, Hsu, Kirsch, James and Subbarao18, Reference Mortelmans, De Cauwer and Van Dyck19 Several studies have emphasized that students are willing to respond to public health emergencies and disasters,Reference Kaiser, Barnett, Hsu, Kirsch, James and Subbarao18–Reference Rega, Bork, Chen, Woodson, Hogue and Batten21 and that their active participation in these situations could be effective in supplementing workforce requirements.Reference Sapp, Brice, Myers and Hinchey22, Reference Katz, Gluck, Maurizio and DeLisi23
Students themselves have recently expressed the need to fill this formative gap, as demonstrated by the current policy statement on disasters and emergencies of the International Federation of Medical Students’ Associations (IFMSA)24 that calls on medical schools to “incorporate disaster medicine and health emergency management into the medical curricula.”25 The IFMSA is an organization active in over 70 countries around the world, acting as official partner of the United Nations and of the World Health Organization. With over 1,200,000 members, it is the largest university association in the world.
However, although several examples of curricular courses have been designed in the last few years,Reference Pfenninger, Domres, Stahl, Bauer, Houser and Himmelseher9–Reference Pelaccia17 Kaiser et alReference Kaiser, Barnett, Hsu, Kirsch, James and Subbarao18 and Smith et alReference Smith, Levy, Hsu and Lee Levy26 have recently demonstrated that only a small percentage of US medical schools include disaster medicine in their core curricula. The same applied to Canadian medical schools, as evidenced by Cummings et al.Reference Cummings, Della Corte and Cummings27 The Centro di Ricerca Interdipartimentale in Medicina di Emergenza e dei Disastri ed Informatica applicata alla didattica e alla pratica Medica (CRIMEDIM),28 an institution of the Università degli Studi del Piemonte Orientale “A. Avogadro,” Novara, Italy, aims to promote and foster research and education in emergency and disaster medicine, with particular interest in the application of new informatics technologies. Within CRIMEDIM, the current European situation, with respect to disaster preparedness and response, has been examined recently; preliminary results confirm a lack of education and training in this field at various academic levels.Reference Della Corte29
In Italy, most medical schools do not contemplate disaster medicine within existing curriculums. However, a recent survey-based studyReference Ragazzoni, Ingrassia, Gugliotta, Tengattini, Franc and Della Corte30 conducted by CRIMEDIM showed that a majority of students would welcome specific training in this area, as well as an innovative educational approach. In an attempt to bridge the gap between educational demands and current academic curricula, the Segetariato Italiano Studenti in Medicina (SISM)31 contacted CRIMEDIM with a proposal for a nationwide program in disaster medicine. As a member of the IFMSA, SISM lists over 5,000 registered members, and it is the largest academic association for medical students in Italy being present in 37 out of 40 universities. The SISM addresses important social issues of medical interest related to public health. It offers training for medical students and is involved in identifying and filling educational gaps in the standard curriculum.
To satisfy this request, a disaster medicine program called DisasterSISM was designed and based on blended learning and simulation tools with the intent to introduce the student population to the importance of disaster medicine and to propose formative experiences in this field to Italian medical students. This manuscript reports the stages of development and the preliminary results in terms of learning outcomes and participants’ satisfaction of this educational program. This experience could represent a valid and innovative solution for designing a flexible disaster medicine curriculum for medical students that could be inserted into already full medical school programs.
The Blended-learning Approach
Rationale
DisasterSISM was intended to provide students in the second triennium of the 6-year undergraduate program an initial competency-based course common to all three disciplines (clinical and psychosocial, public health, and emergency and risk management) of the core of “Disaster Health” according to the World Association for Disaster and Emergency Medicine (WADEM) international guidelines for disaster medicine education.Reference Archer and Seynaeve8 The level of learning was set at the second level of the WADEM multilevel scale for disaster medicine education and training programs.Reference Seynaeve, Archer and Fisher32 Accordingly, the learning objectives were fixed to give the students an understanding of the health aspects and the function of the emergency and public health systems in disasters and mass-casualty incidents with an all hazard approach and the basic competencies in terms of knowledge, skills, and attitudes to administer life-saving treatments during such events. The program was built around the model proposed by Pfenninger and colleagues,Reference Pfenninger, Domres, Stahl, Bauer, Houser and Himmelseher9 selected among other recent examples of standardized medical school curriculum, to fulfill the target level and learning objectives. Moreover, this model was also chosen because it is characterized by the following methodological features:Reference Pfenninger, Domres, Stahl, Bauer, Houser and Himmelseher9
1. developed according to the 6-step approach to curriculum developmentReference Kern, Thomas, Howard and Bass33
2. created in conformity with the WADEM guidelines previously mentioned
3. based on cross-cutting competencies for disaster training of health care professionals identified by a consensus-building approach
4. designed to be delivered through innovative didactic methodologies
5. adopted by nine German medical schools and approved by both the German Society of Disaster Medicine34 and by the German Civil Protection Committee35 and
6. adaptable to various locations and systems.
Development
Following the advice of Pfenninger and colleagues stating “Because blended curricular formats integrating e-methods may enhance inter-professional exchange without compromising pedagogy, they may be included in future curricular versions,”Reference Pfenninger, Domres, Stahl, Bauer, Houser and Himmelseher9 and also ensuring the recommendation contained in the WADEM guidelines to adopt “flexible delivery methods including the use of distance methods,”Reference Archer and Seynaeve8 the curriculum proposed by Pfenninger was reviewed for consensus by a committee of disaster medicine experts from CRIMEDIM with experience in e-learning and simulation-based medical education to accomplish these specific aims:
1. to create a blended-learning experience by combining traditional instructor-led teaching and innovative methods, such as technology-enhanced learning (TEL), problem-based learning (PBL), table-top exercises, and computerized simulations
2. to design a standardized format easily manageable by medical students and easily deliverable by all medical schools and
3. to limit the class attendance to a maximum of 12 hours in order for it to be flexible enough to insert into an already full medical school program.
As shown in Table 2, the work of the expert consensus committee led to the adaptation of Pfenninger's program to a 1-month blended-learning course covering the following topics:
Table 2 Blended-learning Course Matrix
Abbreviations: CS, computerized simulation; e-C, e-Content, as expected time students spend on independent study; ES, electronic simulation; L, lecture; PBL, problem-based learning; RW, residential workshop; TtS, table-top simulation.
1. introduction to disaster medicine and public health during emergencies
2. prehospital disaster management
3. specific disaster medicine and triage procedures in the management of disasters and mass-casualty incidents
4. definition and characteristics of hospital disaster plans for in-hospital and out-of-hospital disasters
5. clinical features and treatment of the specific health consequences of different disasters with an all-hazards approach
6. psychosocial care during emergencies and
7. presentation of past disasters and public health emergencies, and review of assistance experiences.
Among currently known TEL approaches, student-content interaction was chosen.Reference Abrami, Bernard, Bures, Borokhovski and Tamim36 The course format involved content-driven individual study, with the assistance of a tutor, using online materials. It was developed as a standard distance-learning environment using the Modular Object-Oriented Dynamic Learning Environment.Reference Maciel, Soares and Amaral37, Reference Bollela, Grec and Matias38
The web-based platform was an integrated set of tools that contained the general program overview, the e-contents, and an electronic simulation. The e-contents were divided into seven modules, one for each of the topics listed previously. Each module was a 2-part, self-contained unit comprised of original texts and Microsoft PowerPoint (Microsoft Corp., Redmond, Washington USA) presentations (the “basic resources” component) as well as reference manuals and peer-reviewed articles (the “further reading” component). The basic resources component required at least five hours of individual study with an estimated completion time of 35 hours distributed over one month. Module 3, which focused on mass-casualty triage definitions and procedures, included an electronic simulation designed using the web-based application tool Adobe Flash (Adobe Systems Incorporated, San Jose, California USA). The aim of the simulation, which was a car accident resulting in 30 victims, was to train students in mass-casualty triage using the Simple Triage and Rapid Treatment (START) algorithm.39 The same electronic scenario, with randomized victims from a database of 100 different casualties, was delivered twice over the 1-month course in order to intensify the efficacy of training and help assess change in knowledge (Table 2).
Halfway through the course, a short residential workshop (one and one-half days, 12 hours in total), was held. During day one (eight hours), the seven modules were revisited and expanded with lectures, PBL scenarios, and table-top exercises. When working on PBL scenarios (Modules 1, 2, 5, and 7), the participants were divided into groups. Each working group was presented with real-life disaster situations. The aim was to actively engage students in identifying solutions for the health management of these events. Questions were asked relative to the contents presented in each module. The educational goal of the scenarios was to develop transversal skills, such as problem-solving, critical thinking, and the ability to work in teams. For Modules 3 and 4, two different table-top simulations were implemented. The first involved a building collapse in which students were asked to triage a total of 15 victims projected on a whiteboard. The second simulation involved a chemical accident with 150 casualties. The participants had to work out the best possible strategy that would enable the emergency departments of three virtual hospitals to cope with the massive influx of contaminated victims. For this, students were given maps and an overview of hospital resources in printed form. A complete schedule of the residential workshop is shown in Table 2.
During the remaining half day (four hours), a computerized simulation was run using the Interactive Simulation Exercise for Emergencies (ISEE) Simulator (E-Semble, Delft, Netherlands) which is software developed in an EU Leonardo da Vinci Project.Reference Delooz, Debacker, Moens and Johannik40 The simulation-based learning environment was created in accordance with the four criteria for designing effective simulations for learning procedural skills, based on the theoretical framework defined by Kneebone.Reference Kneebone41 The ISEE virtual environment used for the course was specifically developed for practicing the logistic aspects of contingency planning across the entire emergency services chain. It realistically represented the configuration of Novara Province in terms of geographical aspects, number and characteristics of emergency services stations and hospitals, as well as specific features, such as the number of rescue vehicles, the type of equipment, and the availability of on-call personnel. These realistic configurations can be modified easily and adapted to different countries and regions, or from low to high-resource environments. The scenario centered on a car accident resulting in 73 casualties. The simulation was expressly designed for the training and the evaluation of the tactical and strategic aspects of hospital and prehospital medical disaster management. At the end of the computerized exercise, a structured debriefingReference Fanning and Gaba42 was delivered showing the objective data (ie, prehospital, in-hospital, dispatch center performance indicators,Reference Gryth, Rådestad and Nilsson43 and triage accuracy).
The rest of the course was delivered in an e-learning format.
Project Results
From January 2011 through May 2013, 21 editions of the course with the identical format were delivered to 21 different medical schools, representing 56.8% of universities where a SISM local committee was present and 52.5% of all Italian medical faculties. The minimum and maximum number of participants per class was 15 and 30, respectively. In total, 524 students attended the course; of these, 322 (61.5%) were female and 202 (38.5%) were male. Participants were evenly represented by fourth, fifth, and sixth year medical students (170, 32.4%; 179, 34.2%; and 175, 33.4%, respectively). The highest percentage of students (248, 47.3%) indicated internal medicine as their preferred specialty. Only 37 students (7.7%) had previously attended vocational courses on disaster medicine and none had received academic training in the field.
The e-learning platform was accessed a total of 82,556 times, with an average time spent online per day per student of 94 minutes.
Academic Accreditation
All courses were delivered as an elective didactic activity (ADE) and were completed in addition to the mandatory core curriculum. One month before the beginning, the SISM local committees contacted CRIMEDIM for the organization of the course. Then, medical students submitted the course program to the Faculty Board and Degree Committee (FBDC) of each university for approval as an ADE. Elective didactic activities are defined as learning or training activities chosen by students from a number of noncompulsory subjects or courses, each being assigned a number of credits (one credit corresponds to 25 hours of learning, including contact hours and the time set aside for independent study). To graduate, Italian medical students must collect a set number of credits during the 6-year program. These include credits from compulsory subjects, as well as ADEs. Most local committees (19, 90.5%) submitted the program to their FBDCs and 17 (89.5%) of them approved it, assigning from a minimum of one to a maximum of two ADE credits, based on the discretionary decision of each FBDC. Credits were assigned to each student upon submission of a certificate of attendance. This was granted conditional to having obtained a minimum score of six out of 10 on the post-test questionnaire, and a minimum score of 60% on the START triage simulation exercise. To assure confidentiality, the students downloaded the certificate of attendance directly from the e-learning platform.
Evaluation of Knowledge, Skills, and Satisfaction
For purposes of evaluation, the first two levels of the Kirkpatrick 4-level model, “Reaction - what participants thought and felt about the training” and “Learning - the resulting increase in knowledge and skills,” were used.Reference Kirkpatrick44
Before accessing the seven on-line modules, each student had to complete a pretest questionnaire. Four questions were included to measure the participants’ perceived baseline knowledge in disaster medicine, the perceived importance of the course for their future career, the perceived comfort with the teaching format, and the overall interest in attending the course. The answers were rated on a 5-point Likert scale (5 “Strongly Agree/Positive,” 4 “Agree/Positive,” 3 “Neither Agree nor Disagree,” 2 “Disagree/Negative,” and 1 “Strongly Disagree/Negative”) and median results for each of the four questions were 3, 4.5, 4, and 4 out of 5, respectively.
The core of the pretest questionnaire aimed to assess preliminary knowledge of the course topics. It consisted of 25 multiple-choice questions on general principles of disaster medicine, each question comprising four possible choices, only one of which was correct. A simulation exercise assessed students’ confidence with START triage before the course was administered using the electronic simulator described previously. The students were asked to assign the correct color code to each of a series of 30 victims, based on vital signs.
A post-test was administered after completion of the course. The test included 25 multiple-choice questions to assess content knowledge and a START triage simulation exercise with 30 victims, different from those proposed in the pretest.
A 100% response rate was achieved in both the pre and post-tests. In accordance with the grading system commonly applied in Italy, results for the multiple-choice questions were expressed as a score out of 10. Pre and post-test results were 3.95 (SD = 1.29) and 8.29 (SD = 1.76) out of 10, respectively, with a statistically significant improvement between pre and post-tests in each of the course editions (P < .01 for each course). Accuracy in performing the START triage simulation exercise was 45% and 78% in the pre and post-tests, respectively. There was a statistically significant improvement between preand post-test results in each of the course editions (P < .01 for each course).
At the end of the course, the students were asked to rate 10 items on a 5-point Likert scale. The aim of this section was to get a sense of participants’ satisfaction and to detect any opinion change regarding the perceived usefulness of the course. When asked whether they would like to include a similar course in their academic curriculum, most students strongly agreed (472, 90% positive answers). Almost all students (512, 98%) said they would recommend the course to a classmate. Answers to each of the 10 questions are presented in detail in Figure 1.
Figure 1 Overall Results of the Satisfaction Questionnaire
The students were also asked to indicate which aspect of disaster medicine they would like to focus on according to Bradt's disaster medicine frameworkReference Bradt, Abraham and Franks45, Reference Bradt and Drummond46 and to express preferences with regard to the teaching methods used in the course. Preferred topics were as follows: emergency department triage, care, and referral (24%); prehospital triage and transport (22%); incident command (17%); specialty coordination (15%); resource mobilization (11%); and site security (11%). Figure 2 shows the results regarding favored teaching methods.
Figure 2 Students’ Preferences with Regard to Teaching Methods
Written informed consent was obtained from each participant prior to the start of the course. To assure confidentiality, the students submitted the pre and post-test questionnaire, and the satisfaction questionnaire anonymously. Since all data were deidentified and reported in aggregate, the evaluation was deemed exempt from institutional review approval by the local ethics committee.
Data Analysis
Pre and post-test results were compared using Graph Pad Prism (Version 5.0, GraphPad Software, Inc., La Jolla, California USA). All data were tested for normality using the Kolgomorov-Smirnov's test and expressed as means and standard deviation (normally distributed data) or as median and interquartile range (non-normally distributed data). Triage information was analyzed using the chi-squared test or the Fisher's exact test, as appropriate. Numerical variables were compared using t tests for parametric data and the Mann Whitney U test for nonparametric data. The level of significance was set at α = 0.05.
Discussion
In little more than two years, the program DisasterSISM was launched in 21 medical schools evenly distributed across Italy, and a substantial number of students (524) attended the course achieving a considerable reputation in all universities in which it was implemented. This encouraging success seems to reveal that this proposal of a 1-month course based on blended learning and simulation tools could represent a valid and effective model for designing a disaster medicine curriculum for medical students, easily delivered by different institutions. The program was designed to give the students an understanding of the health aspects during disasters, public health emergencies, and mass-casualty incidents with an all-hazards approach, and the basic competencies in terms of knowledge, skills, and attitudes to administer life-saving treatments during such events. The structure of the course curriculum, as well as its distance-learning format, proved flexible enough to allow class attendance to be limited to a total of 12 hours. The curriculum committee consensus does not recommend a further reduction of course contents, or of the e-learning path, to maintain the target level and learning objectives at the second level of the WADEM scale. A possible solution to limit the impact on an already full standard curriculum could be the distribution of the program throughout different years of medical school, rather than presented as a one-time course. This perspective may be helpful as medical schools have different curricular structures, and for some, finding opportunities for integration across the years is easier than finding a concentrated block of time. On the other hand, if the target level is lowered to a community awareness, it is possible to reduce the content of the course materials and accordingly, the time of the entire course, maintaining the same topics of the “Disaster Health” core.
Regarding the evaluation of the program in terms of learning outcomes and participants’ satisfaction, post-test results showed a significant increase in content knowledge and improved triage accuracy after completion of the course. Central to the course was its innovative approach, based on problem-based and distance learning, table-top exercises, and computerized simulations, a mix that proved to improve the students’ experience.Reference Kneebone41, Reference Ruiz, Mintzer and Leipzig47–Reference Vincent, Sherstyuk, Burgess and Connolly65 From this point of view, the program validates the findings of previous studies, namely that computer-assisted learning and simulations effectively facilitate knowledge gain and achievement of course objectives.Reference Kneebone41, Reference Ruiz, Mintzer and Leipzig47 Overall scores on the satisfaction questionnaire showed high appreciation among participants. Despite the fact that very few students had previously been exposed to computer-aided learning,Reference Ragazzoni, Ingrassia, Gugliotta, Tengattini, Franc and Della Corte30 the vast majority felt comfortable with the teaching approach and appeared to particularly appreciate the ISEE simulator application.
However, these results must be tempered in light of some limitations. Apart from the theoretical knowledge acquired and the increase of mass-casualty triage skills, the students were not evaluated for an improvement in other medical disaster management competencies. Given the high appreciation from students, it is suggested that a preliminary computer-based simulation be added at the beginning of the residential workshop. The ISEE simulation could, in fact, be useful for assessing student's abilities in managing virtual events and detecting the increase of these medical expertise after the course. Additionally, it may also increase the level of participant satisfaction, giving students additional exposure to what appears to be the preferred course activity.
Curiously, table-top exercises were not cited by students among the preferred methods of instruction. This could suggest the need to improve the quality of the materials offered, a goal that it is set to achieve in the next editions of the course. It also appears that the e-learning component was not appreciated as much as the other delivery methods. The choice of student-content interaction among current TEL strategies was dictated by a lack of available tutors. Online tutoring was only designed to assist in the delivery of e-contents, while in actual fact, the learning approach was entirely content-driven with independent study featured as a prominent component. A networked collaborative learning (NCL) approach, based on the integration of individual study and collaborative group learning (student-student interaction), could have yielded better results in terms of students’ appreciation.Reference Abrami, Bernard, Bures, Borokhovski and Tamim36, Reference Ruiz, Mintzer and Leipzig47, Reference Wiecha and Barrie66, Reference Wiecha, Gramling, Joachim and Vanderschmidt67 However, because its efficacy is strictly dependent on tutor-to-student ratio,Reference Abrami, Bernard, Bures, Borokhovski and Tamim36 this option was ruled out in the design stage. In the future, while keeping the same 1-month duration, it would be advisable to implement a more effective combination of individual study, collaborative learning, and adaptive teaching by increasing the number of online tutors. Should the number of tutors remain inadequate for an NCL approach, a possible solution could be to add an electronic simulation to each module, so as to increase the exposure of students to computer-assisted interactive activities.
In several editions of the course, local disaster experts, including Emergency Medical Services personnel, emergency medicine physicians, and nurses, worked alongside tutors from CRIMEDIM during the residential workshop. The plan for the future is to develop a nationwide training of trainers program to enlarge the pool of teachers and tutors, and to create local faculty in each medical school reached. Moreover, it is planned to involve police and fire brigades, so as to give students a complete view of the rescue chain in mass-casualty incidents and disasters, and to open the possibility to attend the course to other health care students for enhancing interprofessional collaboration as recently highlighted by Barnett-Vanes and Guinto.Reference Barnett-Vanes and Luis Guinto68
It is extremely important to underline that the true genesis of this innovative program stemmed directly from medical students, who perceived the lack of a specific learning opportunity on disaster medicine as an important gap in the medical curriculum. The analysis of pretest data confirmed that knowledge of disaster medicine among Italian medical students is, on average, inadequate, a result that closely mirrored the students’ own perceptions. Only a very small percentage of participants had previously attended a curricular course on the medical management of mass-casualty incidents and disasters, and these findings confirm those presented in a previous CRIMEDIM study.Reference Ragazzoni, Ingrassia, Gugliotta, Tengattini, Franc and Della Corte30 Unfortunately, disaster medicine is currently under-represented at an academic level in several countries,Reference Kaiser, Barnett, Hsu, Kirsch, James and Subbarao18, Reference Smith, Levy, Hsu and Lee Levy26–28 despite the fact that its importance in this respect has been largely recognized. The vast majority of participants were strongly convinced that knowing at least the basics of disaster medicine is important, and even essential, for a clinician, and that there was a strong possibility that the skills learned through the course could be brought to bear in clinical practice. Additionally, most respondents expressed a view that disaster medicine ought to be included into the core academic curriculum. The areas that appealed the most to students were those related to emergency department triage, stabilization, and care and referral, because of their perceived usefulness for medical practice. These findings should stimulate the academic community to consider introducing at least some basic disaster medicine teaching in the medical school program.
Conclusions
This experience represents an example of a blended-learning curriculum in disaster medicine, easily delivered by medical schools. Attendance at the course significantly increased participants’ knowledge of disaster medicine and basic competencies in performing mass-casualty triage. The students were satisfied with the selection of topics and appeared to particularly appreciate those related to disaster hospital management. The blended-learning approach was also positively evaluated, particularly the computerized simulation. Finally, all participants, regardless of future career plans, believed that disaster medicine ought to be included in the core curriculum of medical schools.
DisasterSISM was elected as one of the most interesting projects created by, and for, medical students at the IFMSA General Assembly in 2011.69
