Skip to main content Accessibility help
×
Home

Information:

  • Access
  • Cited by 27

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        The use of free online educational resources by Canadian emergency medicine residents and program directors
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        The use of free online educational resources by Canadian emergency medicine residents and program directors
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        The use of free online educational resources by Canadian emergency medicine residents and program directors
        Available formats
        ×
Export citation

Abstract

Introduction

Online educational resources (OERs) are increasingly available for emergency medicine (EM) education. This study describes and compares the use of free OERs by the Royal College of Physicians and Surgeons of Canada (RCPSC) EM residents and program directors (PDs) and investigates the relationship between the use of OERs and peer-reviewed literature.

Methods

A bilingual, online survey was distributed to RCPSC-EM residents and PDs using a modified Dillman method. The chi-square test and Fisher’s exact test were used to compare the responses of residents and PDs.

Results

The survey was completed by 214/350 (61%) residents and 11/14 (79%) PDs. Free OERs were used by residents most frequently for general EM education (99.5%), procedural skills training (96%), and learning to interpret diagnostic tests (92%). OER modalities used most frequently included wikis (95%), file-sharing websites (95%), e-textbooks (94%), and podcasts (91%). Residents used wikis, podcasts, vodcasts, and file-sharing websites significantly more frequently than PDs. Relative to PDs, residents found entertainment value to be more important for choosing OERs (p<0.01). Some residents (23%) did not feel that literature references were important, whereas all PDs did. Both groups reported that OERs increased the amount of peer-reviewed literature (75% and 60%, respectively) that they read.

Conclusions

EM residents make extensive use of OERs and differ from their PDs in the importance that they place on their entertainment value and incorporation of peer-reviewed references. OERs may increase the use of peer-reviewed literature in both groups. Given the prevalence of OER use for core educational goals among RCPSC-EM trainees, future efforts to facilitate critical appraisal and appropriate resource selection are warranted.

Introduction

Most medical trainees are members of Generation Y. This cohort, born in the 1980s and 1990s, is characterized as being more technologically savvy than preceding generations. 1 They also increasingly use social media (defined as the sharing of information and ideas within an online community) to communicate. The health profession trainees in this cohort increasingly rely on the Internet and other technology to retrieve medical information. 1 5

Online educational resources (OERs) are becoming increasingly prominent in medical education. Initially, these resources were paid for by institutions (e.g., UpToDate) and individuals (e.g., the podcast EM:RAP); however, free OERs are growing in popularity. 6 8 Emergency medicine (EM) has become a leader in this area. 6 8

Understanding whether learners and their teachers differ in use and choice of free resources is an important next step to comprehending the impact of OERs on medical education. 9 , 10 Free OERs vary in content; some are highly referenced, registered with universities, incorporate peer-review processes, and offer continuing medical education credit, whereas others propagate inaccurate information. 11 , 12 Generational trends suggest that the current cohort of EM learners will be drawn to these nontraditional resources to a greater extent than their teachers, but how students and teachers choose resources has not been well defined. 1 3 To teach appropriate stewardship of this new and evolving repository of educational material, the use of free OER must first be described and quantified.

The primary purpose of this study was to quantitatively describe free OER use by the Royal College of Physicians and Surgeons of Canada (RCPSC) EM residents. Secondary objectives were to compare the use of OERs between RCPSC-EM residents and program directors (PDs) and to investigate the effect of OERs on the use of peer-reviewed literature. A survey methodology was selected to explore these questions and to allow data to be collected from this large, distributed population in a generalizable, cost-effective, and standardized fashion.

Methods

This study received ethical approval from the Research Ethics Board of McMaster University. Survey results were reported in accordance with best practices as outlined in Academic Emergency Medicine. 13

The survey was designed by consensus of the five authors who were informed by similar surveys in anesthesia and EM residents. 5 , 8 , 14 It was hosted on Fluid Surveys. The program director and a resident from postgraduate years one through four from the University of Saskatchewan RCSPC-EM residency program completed a pilot survey between September 27, 2013 and October 10, 2013. Their qualitative feedback was used to clarify questions and refine content. The final version of the survey was translated into French by a bilingual co-author (Migneault). The English version is available in Appendix 1.

A survey link was distributed by email to RCPSC-EM residents and PDs of the 14 training programs in Canada. Respondents were recruited using a modified Dillman method 15 that involved a maximum of five emails from multiple sources, including the director of the Canadian Association of Emergency Physicians resident section, chief residents, and PDs. Responses were collected from November 19 to December 31, 2013. All responses were anonymous, and completion of the survey was voluntary. Entry into a draw for a $500 gift certificate was used as an incentive for survey participation.

The statistical analysis was performed using SPSS Version 21.0 (SPSS IBM, New York, USA). The chi-square test and Fisher’s exact test (<5 variables) were used to calculate p-values for categorical variables. A p-value ≤0.05 was considered statistically significant. Data entered into incomplete surveys were included in the statistical analysis.

Results

As outlined in Figure 1, 214/350 (61%) of RCPSC residents and 11/14 PDs (79%) participated in the study. There were three incomplete responses included in the analysis, for which participants entered demographic information and answered some questions but did not reach the final page of the survey. The demographics of the participants are presented in Table 1.

Figure 1 Flow diagram of study participation and survey completion.

Table 1 Demographics

Free OER use: frequency, type, and comparison to traditional resources

Residents used wikis (95%), e-textbooks (93%), file-sharing sites (91%), podcasts (90%), video sharing sites (88%), medical blogs (80%), and vodcasts (71%) at least monthly. Textbooks (86%), subscription-based resources (65%), and podcasts/vodcasts (40%) were ranked as the top three resources that contributed most to a resident’s education, more frequently than primary literature (33%), e-textbooks (26%), medical blogs (26%), and wikis (11%).

Comparison of the types of OERs used by RCPSC-EM residents and PDs (Figure 2), respectively, showed that residents over PDs used podcasts (90% v. 45%, p<0.01), wikis (96% v. 73%, p<0.01), online file-sharing sites (91% v. 54%, p<0.01), and vodcasts (71% v. 27%, p<0.01) more frequently.

Figure 2 Monthly or more frequent use of OERs by residents compared to PDs (*p<0.05).

Reason for free OER use

OERs were used by residents and PDs, respectively, for a wide variety of purposes (Figure 3), including core EM education (98% v. 82%, p=0.01), procedural skills training (95% v. 73%, p=0.01), diagnostic/imaging interpretation (92% v. 73%, p>0.05), answering questions at the point of patient care (86% v. 82%, p>0.05), and investigating controversial topics in EM (82% v. 73%, p>0.05).

Figure 3 Purpose of OER use by residents compared to PDs (*p<0.05).

Resource selection

Figure 4 compares the factors that residents and PDs used for selecting resources. The factors that ranked more frequently as important or very important by residents when compared to PDs, respectively, were entertainment value (41% v. 9%, p=0.02) and peer referral (50% v. 36%, p=0.05). Inclusion of references was not statistically different between the groups, but only 77% of residents compared to 100% of PDs thought references were important or very important.

Figure 4 Perceived value of factors important for selecting OERs by residents compared to PDs (*p<0.05).

Free OERs and peer-reviewed literature use

Overall, residents (80%) and PDs (73%) self-reported that free OERs increased their use of primary literature. Both groups (72% residents, 55% PDs) felt that they read more studies in full as a result of OERs, and both groups (79% residents, 73% PDs) felt that they read more critical appraisals of primary studies. A minority of residents (25%) and PDs (27%) access more than half of the articles cited by OERs.

Discussion

This study found that RCPSC-EM residents and PDs use free OERs regularly. Residents used nearly all OERs more frequently than PDs and considered different factors when selecting resources. OERs increased self-reported primary literature use. Despite prevalent OER use in our study, residents ranked textbooks and subscription-based resources as their top information sources. These results are similar to other reports of OER use by residents. 5 , 8 , 14

There were several differences between the types of OERs used by residents and PDs. The use of podcasts was the greatest discrepancy with 90% of residents and 45% of PDs listening to them monthly. This difference is important because we also found that learners value direction from faculty in selecting resources. If PDs are not frequently using podcasts or other OERs, they may be ill equipped to suggest appropriate resources. It will be increasingly important for educators to understand the usage patterns and motivations behind learner use of OERs.

Residents’ and PDs’ opinions also differed on the importance of entertainment value and referenced content when selecting OERs. The increased value placed on entertainment by residents is congruent with a recent report highlighting how Generation Y EM learners are most receptive to education that is fast-paced and entertaining. 16 , 17 OERs more frequently offer this “edu-tainment” factor than traditional educational resources. The increasing importance of engaging and entertaining educational materials is a trend that warrants recognition by educational leaders. 17 , 18 The importance of references in the selection of OERs also differed. All PDs valued appropriately referenced OERs, whereas nearly a quarter of residents did not. Although this trend was not statistically significant, efforts to further understand this discrepancy should be a focus of both formal research and discussions between residents and faculty.

By recognizing these differences in OER resource selection, PDs may be better equipped to encourage high quality resource choice among their trainees. Interestingly, a recent survey of RCPSC-EM PDs endorsed the use of high quality OERs and highlighted the importance of developing residents’ abilities to critically appraise them. 19 These skills are important because the quality of information delivered in OERs can be a variable. For example, the reliability of wikis for medical information is unclear 20 , 21 with one study showing that EM trainees more frequently answered clinical questions incorrectly with greater confidence when using Google, which often links to wikis as a resource. 22

One of the strongest criticisms of traditional resources (i.e., textbooks, systematic reviews, and subscription-based) is that they are quickly outdated. Subscription-based resources and e-textbooks have considerable variability in the proportion of updated topics with 52% of 200 articles on UpToDate classified as out of date. 23 If even continuously updated resources are potentially expired, it follows that printed textbooks are even more outdated. Systematic reviews are subject to similar limitations, with 7% in need of updating at the time of publication. 24 OERs have the potential to address this limitation because they can be published quickly and are easily updated.

The delayed update of traditional resources would be less of a problem if it was possible to read all of the latest primary literature. However, this is time-consuming with estimates suggesting a number needed to read 13–14 articles from the top 20 journals to find one practice-changing article. 25 OERs may ameliorate this problem by delivering important information from and increasing awareness of the literature in a timely and digestible fashion. In our study, both residents and PDs reported increased use of primary literature related to OER use, which may be due to increased awareness of relevant articles.

Weighing the strengths and weaknesses of any learning tool should be the shared responsibility of residents and their teachers. Free OERs are no different, but our study demonstrates discrepancies in OER use and the factors that residents and PDs see as important in this decision-making process. OERs offer unique benefits over traditional resources, including ease of access, frequent updates, real-time feedback from a worldwide audience, and the opportunity for active learning through content creation and interaction with content producers. The asynchronous learning model, central to OERs, makes it a natural choice for Generation Y learners who value accessibility highly when seeking information. 1 , 18 Those involved with residency education need to be aware of these benefits and also be prepared find ways to help learners identify the highest quality resources. This will require knowledge of the ever-increasing body of available free OERs.

Limitations

Our survey methodology was susceptible to various forms of bias (e.g., nonresponse, recall, and social desirability); however, many forms of validity evidence were also inherent to our study design. The survey content was based on previously used instruments, the survey was piloted with a representative group of residents and educators, the study population was well defined, reliable contact information was available for all potential participants, the response rate was relatively high for a national survey, 3 , 5 , 8 and residents from all programs and regions of Canada were included.

A second limitation is the large number of comparisons between the two populations, which makes the results susceptible to a type I error. However, the majority of differences that we found were highly significant (generally p<0.01). The small population of PDs also put the study at risk of a type II error.

It is arguable that the chosen study population and focus on free OERs limit the generalizability of our results. Exclusive study of RCPSC-EM residents and PDs means that the results are not fully generalizable to other residency programs, but it did allow for specificity to our population of interest. PDs are a specific and unique population of educators and may not be representative of all teachers involved in the education of RCPSC-EM residents. It was not feasible to survey all educators, so PDs were chosen because they are educational opinion leaders who have consistent contact with the surveyed population of residents and who are responsible for developing longitudinal resident curricula. We focused on free OERs because paid OERs are better established, less accessible, seem to face less scrutiny, and have had less explosive growth.

We were unable to provide descriptive interpretation of our results because the survey included only closed-ended questions. Open-ended questions were not included because it was felt that they would decrease survey completion.

Conclusion

This study informs educators about residents’ OER use. Its findings should stimulate discussions about OER use and their critical appraisal within EM education. EM residents use OERs frequently for general EM education, procedural training, and learning interpretive skills. Relative to their PDs, residents use OERs more frequently and place greater value on entertainment. Self-reported data suggest that OERs increase the use of peer-reviewed literature for both residents and PDs. Future research that prospectively assesses residents’ use of OERs would deepen our understanding of their role in resident education.

Competing interests: None declared.

Supplementary material

To view supplementary material for this article, please visit http://dx.doi.org/10.1017/cem.2014.73

References

1. Weiler, A. Information-seeking behavior in Generation Y students: motivation, critical thinking, and learning theory. The Journal of Academic Librarianship. Elsevier 2005;31(1):46-53.
2. Canadian Medical Association. Social media use by physicians. 2011. Available at: http://www.cma.ca/advocacy/social-media-use-physicians (accessed February 2014).
3. Barker, AL, Wehbe-Janek, H, Bhandari, NS, et al. A national cross-sectional survey of social networking practices of U.S. anesthesiology residency program directors. J Clin Anesth 2012;24(8):618-624.
4. von Muhlen, M, Ohno-Machado, L. Reviewing social media use by clinicians. J Am Med Inform Assoc 2012;19(5):777-781.
5. Matava, CT, Rosen, D, Siu, E, et al. eLearning among Canadian anesthesia residents: a survey of podcast use and content needs. BMC Med Educ 2013;13(1):59.
6. Nickson, CP, Cadogan, MD. Free Open Access Medical education (FOAM) for the emergency physician. Emerg Med Australas 2014;26(1):76-83.
7. Cadogan, MD, Thoma, B, Chan, TM, et al. Free Open Access Meducation (FOAM): the rise of emergency medicine and critical care blogs and podcasts (2002-2013). Emerg Med J 2014;31(e1):e76-7; online first. Available at: http://emj.bmj.com/content/early/2014/02/19/emermed-2013-203502.abstract (accessed March 2014).
8. Mallin, M, Schlein, S, Stroud, S, et al. A survey of the current utilization of asynchronous education among emergency medicine residents in the United States. 2014;89(4):598-601.
9. George, DR, Green, MJ. Beyond good and evil: exploring medical trainee use of social media. Teach Learn Med 2012;24(2):155-157.
10. Cheston, CC, Flickinger, TE, Chisolm, MS. Social media use in medical education. Acad Med 2013;88(6):1.
11. Brabazon, T. The Google effect: googling, blogging, wikis and the flattening of expertise. Libri 2006;56(3):157-167.
12. Thoma, B, Chan, TM, Desouza, N, et al. Implementing peer review at an emergency medicine blog bridging the gap between educators and clinical experts. CJEM 2014;16(0):1-4.
13. Mello, MJ, Merchant, RC. Surveying emergency medicine. Acad Emerg Med 2013;20:409-412.
14. Wong, N. EM education survey. EM Resident 2013:8-9.
15. Hoddinott, SN, Bass, MJ. The Dillman total design survey method. Can Fam Physician 1986;32:2366-2368.
16. Santella, T, Kaneva, P, Petrucci, A, et al. Using social media for surgical education: distraction or opportunity? 2013: 1-2. Available at: http://www.sages.org/meetings/annual-meeting/abstracts-archive/using-social-media-for-surgical-education-distraction-or-opportunity/ (accessed February 2014).
17. Moreno-Walton, L, Brunett, P, Akhtar, S, et al. Teaching across the generation gap: a consensus from the Council of Emergency Medicine Residency Directors 2009 Academic Assembly. Acad Emerg Med 2009;16(2):S19-S24.
18. Sandars, J, Morrison, C. What is the Net Generation? The challenge for future medical education. Med Teach 2007;29(2-3):85-88.
19. Bednarczyk, J, Pauls, M, Fridfinnson, J, et al. Characteristics of evidence-based medicine training in Royal College of Physicians and Surgeons of Canada emergency medicine residencies—a national survey of program directors. BMC Med Ed 2014;14(1):57.
20. Lavsa, SM, Corman, SL, Culley, CM. Reliability of Wikipedia as a medication information source for pharmacy students. Current Pharm Teach Learn 2011;3(2):154-158.
21. Haigh, CA. Wikipedia as an evidence source for nursing and healthcare students. Nurse Educ Today 2011;31(2):135-139.
22. Krause, R, Moscati, R, Halpern, S, et al. Can emergency medicine residents reliably use the Internet to answer clinical questions? West J Emerg Med 2011;12(4):442-447.
23. Jeffery, R, Navarro, T, Lokker, C, et al. How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res 2012;14(6):e175.
24. Shojania, KG, Sampson, M, Ansari, MT, et al. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007;147(4):224-234.
25. McKibbon, KA, Wilczynski, NL, Haynes, RB. What do evidence-based secondary journals tell us about the publication of clinically important articles in primary healthcare journals? BMC Med 2004;2:33-46.