Hostname: page-component-84b7d79bbc-2l2gl Total loading time: 0 Render date: 2024-07-25T12:58:17.342Z Has data issue: false hasContentIssue false

Resident Match During the COVID Pandemic: How Have Neurology Programs Adapted? – A Survey

Published online by Cambridge University Press:  22 February 2022

Naomi Niznick*
Affiliation:
Division of Neurology, Department of Medicine, The Ottawa Hospital, Ottawa, ON, Canada
Ronda Lun
Affiliation:
Division of Neurology, Department of Medicine, The Ottawa Hospital, Ottawa, ON, Canada Ottawa Hospital Research Institute, Ottawa, ON, Canada
Ryan Gotfrit
Affiliation:
Division of Neurology, Department of Medicine, The Ottawa Hospital, Ottawa, ON, Canada
Marc-Olivier Deguise
Affiliation:
Ottawa Hospital Research Institute, Ottawa, ON, Canada Division of Pediatric Neurology, Children’s Hospital of Eastern Ontario, Ottawa, ON, Canada
Tim Ramsay
Affiliation:
Ottawa Hospital Research Institute, Ottawa, ON, Canada School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
Dylan Blacquiere
Affiliation:
Division of Neurology, Department of Medicine, The Ottawa Hospital, Ottawa, ON, Canada
Daniel Lelli
Affiliation:
Division of Neurology, Department of Medicine, The Ottawa Hospital, Ottawa, ON, Canada
*
Corresponding author: Naomi Niznick, MD, Division of Neurology, Department of Medicine, The Ottawa Hospital, The Ottawa Hospital Civic Campus, 1053 Carling Avenue, Ottawa, ON, Canada, K1Y4E9. Email: naniznick@toh.ca
Rights & Permissions [Opens in a new window]

Abstract:

Background:

We aimed to evaluate the perceived effectiveness of interventions implemented by Canadian neurology residency programs for the 2020–2021 iteration of the Canadian Resident Matching Service (CaRMS).

Methods:

A cross-sectional survey was distributed to Canadian neurology residency programs and final-year Canadian medical students who applied to at least one neurology program during the 2020–2021 match cycle. The surveys evaluated pre-interview and interview period interventions implemented by Canadian neurology residency programs and accessed by medical students.

Results:

Thirty-five medical students and 13 out of 15 institutions in Canada with neurology residency programs responded to the survey. Multiple adaptations were implemented, including social media advertisement, web-based platforms, pre-interview information sessions, and teaching sessions, with all surveyed programs implementing at least two virtual interventions. We found that all interventions were perceived as adequate by a majority (>60%) of medical students, with pre-interview period virtual information sessions perceived as effective by the largest proportion of respondents. All Canadian neurology residency programs held virtual interviews for the 2020–2021 cycle, and most programs utilized the same interview structure as prior years. There was discordance between residency program stakeholders and medical students on the most helpful interview period modality. Medical students found the hospital tours and information sessions most valuable, whereas program stakeholders perceived the virtual socials and interviews as most helpful.

Conclusion:

The COVID-19 pandemic has led to innovative adaptations implemented by Canadian neurology residency programs, which were seen as effective by both medical students and program stakeholders.

Résumé :

RÉSUMÉ :

Jumelage des résidents en neurologie : comment les programmes de neurologie se sont-ils adaptés au cours de la pandémie de COVID-19 ?

Contexte :

Nous avons voulu évaluer l’efficacité perçue des interventions mises en place par les programmes de résidence en neurologie au Canada dans le cadre de l’itération 2020-2021 du Service canadien de jumelage des résidents (« CaRMS » en anglais).

Méthodes :

Une enquête transversale a donc été distribuée aux programmes canadiens de résidence en neurologie ainsi qu’aux étudiants canadiens en médecine de dernière année ayant postulé à au moins un programme de neurologie pendant le cycle de jumelage 2020-2021. Notre enquête a cherché à évaluer les interventions mises en œuvre par les programmes canadiens de résidence en neurologie avant et pendant les entrevues auxquelles les étudiants en médecine ont eu accès.

Résultats :

En tout, 35 étudiants en médecine de même que 13 établissements sur 15 qui offrent des programmes de résidence en neurologie au Canada ont répondu à notre enquête. De multiples interventions ont été mises en œuvre, notamment de la publicité sur les médias sociaux, des plates-formes en ligne, des séances d’information préalables aux entrevues et des séances d’enseignement, tous les établissements interrogés ayant mis en œuvre au moins deux interventions virtuelles. À cet égard, nous avons constaté que toutes les interventions ont été perçues comme adéquates par une majorité (> 60 %) d’étudiants en médecine, les séances d’information virtuelles avant entrevue étant perçues comme efficaces par la plus grande proportion des répondants. Il est à noter que tous les programmes canadiens de résidence en neurologie ont organisé des entrevues virtuelles pour le cycle 2020-2021, la plupart des programmes ayant utilisé la même structure d’entrevue que les années précédentes. Une divergence entre les intervenants des programmes de résidence et les étudiants en médecine a été observée quant aux modalités d’intervention les plus utiles. Les étudiants en médecine ont ainsi trouvé les visites à l’hôpital et les séances d’information les plus utiles alors que les intervenants des programmes ont estimé que c’était le cas avec les rencontres sociales virtuelles et les entrevues.

Conclusion :

En somme, la pandémie de COVID-19 a donné lieu à des interventions innovantes mises en œuvre par les programmes canadiens de résidence en neurologie. Ces interventions ont été jugées efficaces tant par les étudiants en médecine que par les intervenants desdits programmes.

Type
Original Article
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of Canadian Neurological Sciences Federation

Introduction

For the past five decades, graduating medical students in Canada have sought their prospective residency training program through the Canadian Resident Matching Service (CaRMS).

The COVID-19 pandemic significantly affected the 2020–2021 CaRMS cycle and led to a multitude of challenges for medical students applying to residency programs. With physical distancing regulations and travel bans in place in Canada since March 2020, medical students across the country were not allowed to complete visiting electives at other institutions as part of the 2020–2021 application cycle. Hence, this had significant implications on the application process for candidates. Completing multiple electives across Canada has been a mainstay for medical students to declare their interest in a specific medical specialty. Reference Courneya, Cheung and McMillan1,Reference Clark, Shah, Kolla, Marshall, Bryson and Nair2 In the prior 2019–2020 CaRMS cycle, the average medical student completed eight clerkship electives in their matched discipline; for neurology specifically, the average number of neurology electives was 6. 3 It should be noted that the AFMC (Association of Faculties of Medicine of Canada) implemented a policy that now limits medical student clerkship electives to a maximum of 8 weeks in the same discipline. There is also evidence that during prior CaRMS cycles, most medical students had completed at least one elective at the school they subsequently matched to. 3 For example, in 2019–2020, 78.7% (37/47) of Canadian Medical Graduates (CMGs) who matched to neurology did at least one elective at the school where they matched. 3

Based on these statistics, multiple potential challenges can be identified for medical students who recently completed the 2020–2021 CaRMS cycle. These challenges include a lack of opportunities to explore other institutions, needing to find unconventional ways to demonstrate an interest in neurology, overcoming “home-school advantage” (defined as students being more likely to match to their home school due to increased exposure and interactions with residents/staffs at their local hospital), a limited number of neurology home elective opportunities, and a potential disadvantage for students at institutions where there are few neurology spots.

While this posed adversity for candidates, neurology programs also faced new challenges for similar reasons. Applicant file scoring had to rely more heavily on the applicant’s curriculum vitae (CV) rather than on direct clinical observation and interpersonal skills. In response, many neurology residency programs implemented creative methods to promote their programs to enhance visibility and recruitment. These pre-interview period adaptations were also accompanied by necessary changes to the interview process, given that interviews for the 2020–2021 CaRMS cycle were held virtually for the first time. Dilemmas included choosing a virtual platform for conducting interviews, finding ways to engage applicants on interview day, and hosting virtual social gatherings.

It is not known whether the measures implemented helped applicants to adequately learn about Canadian neurology residency programs before ranking for the CaRMS 2020–2021 match. Similarly, it is unknown which of these changes were the most helpful for neurology programs to become better acquainted with the applicants. In this study, we evaluated the interventions implemented by Canadian neurology residency programs and used by medical students, both during the pre-interview and interview period, to assess their perceived efficacy for the programs and medical students.

Methods

Study Design

We utilized a cross-sectional survey study design to analyze the usefulness of adaptations implemented by Canadian neurology residency programs and used by medical students. Two separate surveys were designed – one distributed to Canadian neurology residency programs and one to final-year medical students who applied to at least one neurology program during the 2020–2021 match cycle. Surveys were available in the official languages of Canada – French and English.

Standard Protocol Approvals, Registrations, and Patient Consents

The current study falls within the context of quality initiative, quality improvement, quality assurance, and/or program evaluation. Hence, as per the Tri-Council Policy Statement 2, Article 2.5, this project received an exemption from Research Ethics Board review by our local institution. Participation in the survey was voluntary, and consent was implied upon completion of the survey as indicated in the survey (“By completing this survey, you are providing consent to use the data provided for research purposes”). No identifying data was used for research purposes.

Study Population

Potential participants were eligible to complete the Canadian neurology residency program survey if they were either a program director, program administrator, neurology resident or staff physician affiliated with an accredited Canadian adult, or pediatric neurology residency program. Neurology residents and staff physicians had to be actively involved in the 2020–2021 CaRMS match process to be eligible to complete the survey. Medical students were eligible to complete the survey if they were final-year medical students at any of the 15 Canadian medical schools who applied to at least one Canadian adult or pediatric neurology residency program during CaRMS in the 2020–2021 cycle. Participants had to be comfortable completing the survey in either English or French. Any candidates who did not satisfy the above inclusion criteria were excluded.

Due to the descriptive nature of the study, a target sample size was not calculated. We utilized convenience and snowball sampling to identify potential respondents via multiple different methods. Potential program-related survey respondents were identified through an available database containing contact information for program directors, program administrators, and chief neurology residents. Snowball sampling was subsequently used to identify other staff physicians and neurology residents involved in the CaRMS process at each university. Potentially eligible medical students were identified through a combination of methods – email lists of applicants who self-identified as interested in applying to neurology, direct dissemination from Undergraduate Medical Education (UGME) offices, advertisement by neurology interest groups at medical schools, social media (Twitter and Instagram), and word of mouth.

Survey Content and Development

The survey was created specifically for this study using the methodology described by Burns. Reference Burns, Duffett and Kho4 Question development and item generation were completed by three members (NN, RL, RG). Pre-testing, focusing on ensuring questions met their intended purpose, was conducted by the rest of the research team. Piloting and clinical sensibility testing was subsequently completed by a sample population of 10 neurology program stakeholders, including both staff physicians and neurology residents, and 10 medical students interested in applying to neurology. The sample population of medical students was chosen from the local undergraduate neurology interest group at our affiliated medical school. Data collected during piloting were not utilized in the final analysis of the study and were used purely to inform the structure and design of the survey. Medical students who piloted the survey were not eligible to participate in the final study.

Two surveys with parallel questions were designed for this study, one for medical students and one for program stakeholders. The survey was divided into four sections: demographics, pre-interview period interventions, interview period interventions, and feedback. In total, there were 36 questions for the program stakeholder survey and 37 questions for the medical student survey. The program stakeholder and medical student surveys are both available in Supplemental Materials (Appendix 1).

Survey Administration

The survey was administered through SurveyMonkey®. This one-time survey was distributed to our target populations from April 1, 2021 to April 20, 2021. This specific timeline was selected to align with the end of the national interviewing period but before the release of the match results. The survey link was distributed to neurology residents, program administrators, and staff physicians by email via a direct survey link. The medical student survey was distributed through email and posting of the survey link on social media (Twitter and Instagram). To ensure respondents satisfied our eligibility criteria, the inclusion/exclusion criteria were embedded into the first four questions of both the program and medical student surveys. The questions were designed to prevent anyone who met any of our exclusion criteria from proceeding further.

Data Collection Methods

Data were collected solely through SurveyMonkey®. For the neurology program survey, respondents were asked to identify their home medical school. However, the information about their specific program (adult vs pediatric) or role within the program had the potential for unintentional identification and was therefore omitted to ensure anonymity. Anonymous data were exported to a CSV file.

Statistical Analysis

The data were analyzed using descriptive statistics given the hypothesis-generating nature of our study. We used a 5-point Likert scale to evaluate the perceived effectiveness of interventions implemented by programs or used by medical students, where one represented “not effective” and five represented “very effective.” Likert scales evaluating effectiveness were collapsed into dichotomous variables for analysis: effective (Likert scale scores of 4–5) and not ineffective (Likert scale scores of 1–3). We also asked all respondents to compare all pre-interview and interview period modalities using rankings. We subsequently scaled the data as a percentage scale (i.e. from 1 to 100) and the average ranking for each intervention was compared. All statistical analyses were performed using SPSS v27.0 (IBM, Armonk, NY).

Data Access and Availability

Supporting data for this study is not publicly available, as the distribution of responses based on location can unintentionally compromise respondent anonymity (as described in Data collection method section).

Results

Medical Student Survey

Demographics

There were 35 total responses collected for the medical student survey. Four responses were excluded (for not being in their final year of medical school), resulting in a total of 31 survey responses included for analysis. The distribution of respondents by their home medical school is displayed in Figure 1A. The majority of students (24/31; 77%) applied to nine or more adult neurology programs (Supplemental Materials Table 1), and 26/31 (84%) applied to 0–1 pediatric neurology programs (Supplemental Materials Table 2). In addition, 28/31 (90%) applied to less than five non-neurology residency programs (Supplemental Materials Table 1).

Figure 1: The number of survey respondents by academic institution affiliation: (A) represents medical student responses and (B) represents neurology residency program responses.

Before the onset of the COVID-19 pandemic in March 2020, 89% of medical students had done 0–4 weeks of electives in neurology, with relatively similar proportions of home school electives versus away electives (Supplemental Materials Figure 1). After the COVID-19 pandemic, only six medical students (18%) reported doing any weeks of neurology electives at an institution other than their home school. In contrast, most students (89%) did between 1 and 8 weeks of neurology electives at their home school (Supplemental Materials Figure 1).

Pre-Interview Modalities

Pre-interview modalities utilized by medical students were divided into four categories: social media, web-based platforms/websites, virtual information sessions, and virtual teaching sessions. The results are summarized in Table 1. Out of 31 survey respondents, 90% of medical students utilized some form of social media during the pre-interview period to learn more about residency programs of interest. The most used social media platform was Instagram, in which 77% of medical student respondents used, followed by Facebook (45%) and Twitter (32%). Only one medical student did not use web-based platforms during the pre-interview period; the majority of medical students used either the Association of Faculties of Medicine of Canada (AFMC) website (81%) or program-specific websites (87%). Only 3 (9.7%) medical students accessed the CaRMS website for the purpose of reading program descriptions. Next, virtual information sessions held by residency programs were attended by 29/31 survey respondents (94%). Large group information sessions were defined as more than 15 medical students per session, and small group information sessions were defined as 15 or fewer students. 81% of survey respondents attended at least one large group session, 74% attended at least one small group session, and 61% reported having had at least one virtual one-on-one session with someone affiliated with a neurology residency program. More than half of those that had a one-on-one meeting reported doing so with a nonchief resident (55%), while 45% reported meeting with the program director of at least one residency program of interest. Lastly, virtual teaching sessions were attended by 81% of respondents, with resident academic half-day reported as the most common form of teaching session attended (61%), followed by divisional grand rounds (45%) (Table 1).

Table 1: The number of pre-interview modalities used by medical student applicants or implemented by academic institutions affiliated with a Canadian neurology residency program during the 2020–2021 match cycle

* Description of residency program on the CaRMS website (https://www.carms.ca/program-descriptions/) is mandatory.

& No comments were made to elaborate further on “others”.

The perceived effectiveness of each modality was dichotomized into effective versus neutral/ineffective and presented as proportions of survey respondents who thought the intervention was effective. All interventions were perceived as effective by most survey respondents (i.e. >60% for all interventions) (Figure 2 – where blue represents medical student responses). One-on-one meetings were perceived by the highest proportion of respondents to be effective (95%), while virtual teaching sessions were perceived as effective by the smallest proportion of respondents (65%) (Figure 2).

Figure 2: Survey respondents that perceived the pre-interview modality as either “very effective” or “effective” in helping medical students get to know a residency program of interest. Results are presented as proportions and stratified by medical student versus program responses.

Respondents were also asked to compare all pre-interview period interventions against each other by ranking all interventions they used, from most effective to least effective in getting to know a residency program of interest. Since not all respondents were able to rank all interventions, individual respondents’ ranks were converted to a 100-point percentage scale. Based on the mean rank score for each intervention, it was found that medical students perceived teaching sessions to be the most effective, followed by social media, then large group sessions, web-based platforms, one-on-one meetings, and small group information sessions last (Supplemental Materials Table 3). This was counterintuitive given the results in Figure 2. Upon revisions of the raw data, there is a high suspicion that the question was poorly understood by survey takers.

Interview Period

In total, 50% of medical student respondents attended 10 or more virtual interviews for neurology programs (Supplemental Materials Figure 2). The most common virtual interview platform was Zoom, which was encountered by 81% of medical students (Table 2). In addition, 74% of all respondents reported participating in multiple mini-interviews (MMI), panel interviews, and one-on-one interviews (Table 2). All medical students (100%) attended at least one virtual social, while 97% attended at least one information session during the interview period, and 87% attended at least one hospital tour (Table 2).

Table 2: Descriptions of interview period modalities encountered by medical students or implemented by academic institutions affiliated with a Canadian neurology residency program during the 2020–2021 match cycle

The most effective interview period intervention for deciding if a program was a good fit for them was hospital tours, followed by information sessions, virtual socials, and lastly, the interview itself (Supplemental Materials Table 4, Table 3).

Table 3: Ranking of pre-interview period and interview period modalities by medical students and residency program survey respondents

Feedback

We used free-text boxes to gather comments from medical students regarding how their virtual CaRMS experience could have been improved; these results are summarized by theme in Supplemental Materials Table 5.

Neurology Program Survey

Demographics

There are 15 accredited adult neurology residency programs and 7 pediatric neurology residency programs in Canada. The distribution of respondents by their home institution is displayed in Figure 1B. We had a total of 48 responses – three responses were invalid since respondents reported not being directly involved with the match process this year. Therefore, 45 responses were included for analysis. Out of the valid responses, 35/45 (73%) were affiliated with an adult neurology program and 10/45 (21%) with a pediatric neurology program.

Preinterview Modalities

The reporting of interventions implemented is summarized by institutions (i.e. not by residency programs) (Table 1). 3/13 institutions reported not using social media to advertise their residency programs. The most common social media platform reported was Instagram (62%), followed by Facebook (31%) and Twitter (23%). All institutions reported using web-based platforms for medical student engagement, with the AFMC website reported as the most common website used (85%). In addition, all institutions hosted pre-interview period virtual information sessions, with the majority of institutions (77%) reporting the use of small group sessions (i.e. ≤15 students per session), followed by 69% reporting the use of large group sessions (i.e. more than 15 students) and 69% having participated in at least one one-on-one meeting with a medical student. Only 9/13 institutions (69%) used virtual teaching sessions as a way of medical student engagement: 6/13 institutions (46%) invited medical students to resident academic half-day and grand rounds, while 5/13 institutions (38%) organized nonacademic half-day virtual teaching sessions.

All program survey respondents thought small group information sessions and one-on-one meetings were effective interventions, and only 61% of program survey respondents thought virtual teaching sessions were effective (Figure 2 – depicted by orange bars). Upon conversion to numeric scales, we found programs perceived one-on-one meetings as the most effective intervention, followed by teaching sessions, large group information sessions, social media, web-based platforms, and small group information sessions as the least effective intervention (Supplemental Materials Table 3).

Interview Period

The most frequent platform used to conduct virtual interviews was Zoom (12/13 institutions – 92%) (Table 2). Other platforms utilized included Microsoft Teams (1/13) and WebEx (1/13). The majority of institutions (62%) utilized an MMI interview format followed by panel interview (46%). Compared to previous years, 11/13 institutions (85%) reported unchanged interview structure. Out of the 13 institutions represented, 11 hosted a virtual social, 10 hosted an information session, and 6 organized a virtual hospital tour. Softwares/applications such as Wonder, Kumospace, DoorDash, UberEats, and Gather/Town were used by programs to host virtual socials. Descriptions of socials were collected using free-text entry (Supplemental Materials Table 5).

Based on summary average ranks (Supplemental Materials Table 4) to compare interview innovations, programs perceived virtual socials as the most effective interview period intervention, followed by the interview itself, then hospital tours, and information sessions (Table 3). 1/28 respondents thought virtual interviews were more useful than in-person interviews, 14/28 (50%) thought virtual interviews were equivalent, and the remaining 13 (46%) perceived virtual interviews as less helpful than traditional in-person interviews.

Issues Encountered and Feedback

The issues encountered by residency programs are categorized by pre-interview period and interview period issues (Table 4). One common theme identified during the pre-interview period was the challenge in gauging medical student interest and evaluating clinical performance without visiting electives. Interview period difficulties included technology-related difficulties (i.e. moving people in and out of breakout rooms), and scheduling-related issues due to fewer students declining virtual interviews than traditional in-person interviews from previous years.

Table 4: Free-text responses to the question “Did you have any issues during this year’s CaRMS process?” from the neurology program survey

Lastly, feedback from the program survey in terms of how the CaRMS process could have been improved this year is summarized in Supplemental Materials Table 5. Medical students and program respondents emphasized the importance of not having programs overlap interview days and virtual socials for future cycles. Multiple respondents also wished to address technical difficulties by potentially familiarizing interviewers with the interview platform before the interview period and providing training on virtual interviews a priori. The most common response from the program survey in terms of how the experience could have been improved was “if interviews had been in person”.

Discussion

The COVID-19 pandemic has resulted in consequences at all levels of our society, including the way medical graduates identify and match to their future residency program. In this study, we found that all surveyed Canadian neurology programs implemented at least two virtual interventions in preparation for the CaRMS 2020-2021 cycle, which were accessed by the majority of applicants. Social media platforms were used by 76.9% of institutions, with Instagram being the most utilized platform. Use of social media platforms is common amongst younger generations of medical graduates and publications about social media in medical education have exponentially grown in the recent years. Reference Katz and Nandi5 Instagram was found to be the most common platform used in multiple recent studies, and our results support similar findings. Reference Bludevich, Fryer, Scott, Buettner, Davids and LaFemina6,Reference Clay Pruett, Deneen and Turner7 Most surveyed medical students also accessed web-based platforms (96.8%), and virtual information sessions (93.5%) at some point during the CaRMS cycle. Interestingly, only 3 (9.7%) medical students accessed the CaRMS website for the purpose of program descriptions despite the fact that it is mandatory for programs to update this information every application cycle. Overall, we found that all implemented interventions were perceived as adequate by the majority (>60%) of medical students and residency program stakeholders, with virtual information sessions (specifically – one-on-one meetings and small group information sessions) perceived as effective by the largest proportion of respondents.

All Canadian neurology residency programs held virtual interviews during the CaRMS 2020–2021 cycle. Despite requiring this adaptation, most institutions reported utilizing the same interview structure as prior years. In addition to virtual interviews, 84.6% of institutions hosted virtual socials, 76.6% organized virtual information sessions during the interview period, and 46.2% arranged virtual hospital tours. There was discordance between residency program stakeholders and medical students’ perceptions on the most helpful interview period modality in getting to know applicants and programs, respectively. Medical students found the hospital tours and information sessions most valuable, whereas program stakeholders perceived the virtual socials and interviews most helpful. This difference may stem from the unique goals of each modality: informative sessions (i.e. hospital tours, information sessions) may be perceived as more effective for accruing objective information about a program, given applicants’ lack of exposure to other institutions, while socials may be perceived as opportunities to socialize and form interpersonal connections. As such, medical students may seemingly be more focused on using the interview period as an opportunity to learn about institutions and specific program details. During past CaRMS cycles, medical students have traditionally used visiting electives to acquire this information. Due to the COVID-19 pandemic, most students were unable to complete away electives (Supplemental Materials Figure 1). This is further supported by medical students’ comments in the survey recommending that future virtual CaRMS cycles could implement more virtual hospital tours and ensuring that information sessions between institutions do not overlap to allow medical students to attend all sessions. Conversely, residency program stakeholders may be more focused on getting to know individual applicants to ensure that they are a good fit for their program. Programs may better acquire this information through socials and one-on-one interactions, such as interviews, which may be why programs ranked these modalities more favorably. To circumvent difficulties in learning about programs for future applicants, neurology programs may consider alternative ways to distribute program information – for example, the Canadian Association of Radiologists put forth an online series for medical students to learn about programs outside of their home institution, which was well received by most attendees. Reference Linehan, Katlariwala, Chahal, Khatchikian, Supersad and Dobson8 Medical students could also aim to optimize the time provided during their interviews to ask more in-depth questions. A commentary by Canadian neurology program stakeholders from Western University, Mirian et al. suggested several important topics that medical students should inquire about in order to aid with evaluating programs. Reference Mirian, Jenkins, Watling, Venance and Florendo-Cumbermack9 These suggestions included asking about a program’s mentorship culture, how a program supports residents’ career goals, and resident wellness initiatives. Using focused and goal-directed questioning may make the interviews more valuable to medical students.

This is the first research study to evaluate the adaptations made by Canadian neurology residency programs for CaRMS in response to the COVID-19 pandemic, and to assess their effectiveness both for medical students and program stakeholders. One study focusing on Canadian ophthalmology programs surveyed selection committee members and program directors and outlined their views on the essential aspects of CaRMS applications, while also describing the perceived effect of the pandemic on the CaRMS process. Reference Nguyen, Clark and Damji10 However, this study did not include resident or medical student perspectives and did not evaluate the effectiveness of changes to the process. This study did find that stakeholders viewed cancellation of visiting electives as likely to have the most significant negative impact of the CaRMS cycle. Our study supports this finding, as a reasonable proportion of both medical students and residency program stakeholders commented that the CaRMS process could be improved by reimplementing visiting electives. Currently, there is no plan by Canadian medical schools or the Association of Faculties of Medicine (AFMC) to reimplement visiting electives for the 2021–2022 cycle. The commentary by Mirian et al. also made several recommendations for prospective neurology residency applicants. Reference Mirian, Jenkins, Watling, Venance and Florendo-Cumbermack9 The findings of our study support their recommendation that medical students should attend virtual information sessions or academic half-days in order to learn more about a program’s academic environment. Other studies focused on evaluating the perception of virtual interviews by medical students, residents, and physicians in the United States. Reference Seifi, Mirahmadizadeh and Eslami11,Reference Shreffler, Platt, The and Huecker12 However, these surveys were disseminated before the interview period and therefore did not assess perspectives after completing interviews. The added advantage of virtual interviews may include cost- and time-saving benefits for medical students. For example, the average applicant in general surgery visited 10 cities with a total approximated cost of $4866. Reference Fung, Raiche, Lamb, Gawad, MacNeill and Moloo13 It should be noted that traveling for interviews also generatesalso excluded from this study, and therefore a significant carbon footprint and has, without a doubt, a negative impact on the environment. A recent study calculated that in 2020 the average CaRMS applicant had a carbon footprint of 1.44 tCO2e (tonnes of carbon dioxide equivalent), which is equivalent to 35.1% of the average annual carbon footprint of 1 Canadian household. Reference Fung, Raiche, Lamb, Gawad, MacNeill and Moloo13,Reference Liang, Dawson, Stoian, Clark, Wynes and Donner14 This could be substantially reduced by centralized interviews or virtual interviews, the latter achieving upwards of 98% reduction. Reference Liang, Dawson, Stoian, Clark, Wynes and Donner14 However, several program respondents commented that virtual interviews were inferior to in-person interviews for getting to know candidates. Several respondents also mentioned that they experienced technical difficulties during the interview period. Formal training on interview platforms would be helpful to ensure a smoother process for future virtual interviews. Similarly, unified, standardized use of the platform throughout Canada could facilitate the process.

The strengths of our study include the systematic methodology involving pretesting, piloting, and clinical sensibility testing before the dissemination of our survey. We also focused on involving a broad range of stakeholders, including medical students, residents, program directors, program administrators, and selection committee members to adequately represent different perspectives. We gathered responses from most institutions in Canada with a neurology residency program, with 13 out of 16 institutions responding to our survey. Our study is not without limitations. Our sample sizes were small, but more specifically, medical student responses only represented 10 out of 17 Canadian medical schools. Given that medical students were partially identified through social media and email lists of applicants who self-identified as interested in applying to neurology, there is a risk of sampling bias as medical students identified through these methods may represent a highly engaged group of medical students with a strong interest in neurology as compared to the general medical student population. Furthermore, given that social media was used for recruitment, this could have created bias as medical students involved in the study may have been more likely to use social media methods to learn about neurology residency program. International medical graduates (IMGs) were also excluded from this study and therefore, the results cannot be generalized to IMGs. Lastly, it should be noted that we may have encountered misinterpretation of survey instructions for the ranking questions: medical students were asked to rank pre-interviews modalities from most effective (rank of “7”) to least effective (rank of “1”). There was a discrepancy between the modalities perceived as effective by the largest proportion of medical students and the effectiveness rank order of these modalities, suggesting possible misinterpretation of ranking instructions (i.e. using the rank number of “1” as most effective rather than least effective). This raises concerns regarding the reliability of this specific question, and the medical student ranking data of pre-interview modalities should be interpreted with caution.

Conclusion

Overall, Canadian neurology residency programs have put significant effort into adapting to the ongoing changes to Canadian medical education, while also trying to address the needs of final year medical students applying to neurology. With the added effort of implementing these changes, it is crucial to continue to evaluate these measures to ensure that interventions are evidence-based and truly represent medical student needs while also balancing opportunity costs for residency programs. This study demonstrates that efforts made by Canadian neurology residency programs were seen as effective by both medical students and program stakeholders, which supports the continued use of these adaptations during future CaRMS cycles, especially since the 2021–2022 CaRMS cycle will again be virtual, with the cancellation of medical student visiting electives. Both medical students and program stakeholders had similar views on the most effective pre-interview modalities: neurology residency programs should focus on optimizing small group information sessions and increasing accessibility to one-on-one virtual meetings during future virtual CaRMS cycles. It is also important that the interview period focuses on both student and program needs, incorporating information sessions, hospital tours, and virtual social sessions. Canadian neurology residency programs should coordinate both virtual interviews and social dates to ensure that there is no overlap between sessions so that medical students interested in neurology can attend all relevant sessions. Once safe to do so, all involved parties support reinstituting visiting electives. More research is needed to evaluate the cost-saving and environment-friendly benefits of virtual interviews compared to the perceived benefit of getting to know applicants through in-person interviews.

Supplementary material

For supplementary material accompanying this paper visit https://doi.org/10.1017/cjn.2022.16

Acknowledgements

No acknowledgements.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of Interest

The authors have nothing to disclose.

Disclosures

None.

Ethical Approval

The study received exemption from the Ottawa Health Science Network Research Ethics Board as it falls within the context of quality initiative, quality improvement, quality assurance, and/or program evaluation.

Statement of Authorship

N.N. and R.L. conceptualized/designed the study, acquired the data, and analyzed and interpreted the data. R.G. and M.D. acquired and interpreted the data. T.R., D.B., and D.L. designed the study and interpreted the data. All authors were involved in drafting or revising the study for important intellectual content.

Footnotes

Dr. Niznick and Dr. Lun contributed equally as co-first authors.

References

Courneya, CA, Cheung, W, McMillan, MJ. High dives and parallel plans: relationships between medical student elective strategies and residency match outcomes. Can Med Educ J. 2020;11:e4e12. DOI 10.36834/cmej.53018.Google ScholarPubMed
Clark, M, Shah, S, Kolla, L, Marshall, S, Bryson, S, Nair, B. Post-CaRMS match survey for fourth year medical students. Can Med Educ J. 2020;11:e101–10. DOI 10.36834/cmej.69330.Google ScholarPubMed
CaRMS. Electives data - CaRMS. Available at: https://www.carms.ca/data-reports/r1-data-reports/electives/; accessed December 4, 2020.Google Scholar
Burns, KE, Duffett, M, Kho, ME, et al. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ. 2008;179:245–52. DOI 10.1503/cmaj.080372.CrossRefGoogle ScholarPubMed
Katz, M, Nandi, N. Social media and medical education in the context of the COVID-19 pandemic: scoping review. JMIR Med Educ. 2021;7:e25892. DOI 10.2196/25892.Google ScholarPubMed
Bludevich, BM, Fryer, M, Scott, EM, Buettner, H, Davids, JS, LaFemina, J. Patterns of general surgery residency social media use in the age of COVID-19. J Surg Educ. 2021;78:e218–25. DOI 10.1016/j.jsurg.2021.04.017.CrossRefGoogle ScholarPubMed
Clay Pruett, J, Deneen, K, Turner, H, et al. Social media changes in pediatric residency programs during COVID-19 pandemic. Acad Pediatr. 2021;21:1104–7. DOI 10.1016/j.acap.2021.06.004.10.1016/j.acap.2021.06.004CrossRefGoogle ScholarPubMed
Linehan, V, Katlariwala, P, Chahal, BS, Khatchikian, AD, Supersad, A, Dobson, JL. The online cross-country tour: evaluation of a novel online series for canadian radiology residency programs and future implications for the radiology CaRMS process. Acad Radiol. 2021;28:1313–20. DOI 10.1016/j.acra.2021.03.009.Google ScholarPubMed
Mirian, A, Jenkins, ME, Watling, CJ, Venance, SL, Florendo-Cumbermack, A. Finding the "right" Canadian neurology residency program during the COVID-19 era. Can J Neurol Sci. 2021;48:47–9. DOI 10.1017/cjn.2020.218.Google ScholarPubMed
Nguyen, AX, Clark, I, Damji, KF, et al. New virtual CaRMS: perspectives from residency programs. Can J Ophthalmol. 2021;56:273–6. DOI 10.1016/j.jcjo.2020.12.007.CrossRefGoogle ScholarPubMed
Seifi, A, Mirahmadizadeh, A, Eslami, V. Perception of medical students and residents about virtual interviews for residency applications in the United States. PLoS One. 2020;15:e0238239. DOI 10.1371/journal.pone.0238239.Google ScholarPubMed
Shreffler, J, Platt, M, The, S, Huecker, M. Planning virtual residency interviews as a result of COVID-19: insight from residency applicants and physicians conducting interviews. Postgrad Med J. 2021. DOI 10.1136/postgradmedj-2020-139182.Google ScholarPubMed
Fung, BS, Raiche, I, Lamb, T, Gawad, N, MacNeill, AJ, Moloo, H. A chance for reform: the environmental impact of travel for general surgery residency interviews. Can Med Educ J. 2021;12:818. DOI 10.36834/cmej.71022.Google ScholarPubMed
Liang, KE, Dawson, JQ, Stoian, MD, Clark, DG, Wynes, S, Donner, SD. A carbon footprint study of the Canadian medical residency interview tour. Med Teach. 2021;43:13021308. DOI 10.1080/0142159X.2021.1944612.10.1080/0142159X.2021.1944612CrossRefGoogle ScholarPubMed
Figure 0

Figure 1: The number of survey respondents by academic institution affiliation: (A) represents medical student responses and (B) represents neurology residency program responses.

Figure 1

Table 1: The number of pre-interview modalities used by medical student applicants or implemented by academic institutions affiliated with a Canadian neurology residency program during the 2020–2021 match cycle

Figure 2

Figure 2: Survey respondents that perceived the pre-interview modality as either “very effective” or “effective” in helping medical students get to know a residency program of interest. Results are presented as proportions and stratified by medical student versus program responses.

Figure 3

Table 2: Descriptions of interview period modalities encountered by medical students or implemented by academic institutions affiliated with a Canadian neurology residency program during the 2020–2021 match cycle

Figure 4

Table 3: Ranking of pre-interview period and interview period modalities by medical students and residency program survey respondents

Figure 5

Table 4: Free-text responses to the question “Did you have any issues during this year’s CaRMS process?” from the neurology program survey

Supplementary material: File

Niznick et al. supplementary material

Niznick et al. supplementary material 1

Download Niznick et al. supplementary material(File)
File 35.8 KB
Supplementary material: File

Niznick et al. supplementary material

Niznick et al. supplementary material 2

Download Niznick et al. supplementary material(File)
File 57.6 KB