Hostname: page-component-7c8c6479df-nwzlb Total loading time: 0 Render date: 2024-03-29T14:03:25.368Z Has data issue: false hasContentIssue false

Transition to practice: Evaluating the need for formal training in supervision and assessment among senior emergency medicine residents and new to practice emergency physicians

Published online by Cambridge University Press:  07 March 2019

Sarah Kilbertus*
Affiliation:
Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON
Kaif Pardhan
Affiliation:
Department of Emergency Services, Sunnybrook Health Sciences Centre, Toronto, ON Division of Pediatric Emergency Medicine, Department of Pediatrics, McMaster University, Hamilton, ON
Juveria Zaheer
Affiliation:
Institute for Mental Health Policy Research, Centre for Addiction and Mental Health, Toronto, ON
Glen Bandiera
Affiliation:
Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON Department of Emergency Medicine, St. Michael's Hospital, Toronto, ON
*
Correspondence to: Dr. Sarah Kilbertus, Department of Medicine, University of Toronto, 190 Elizabeth Street, R. Fraser Elliott Bldg. Rm. 3-805, Toronto, ON M5 G 2C4; Email: Skilb086@uottawa.ca

Abstract

Objectives

Emergency medicine residents may be transitioning to practice with minimal training on how to supervise and assess trainees. Our study sought to examine: 1) physician comfort with supervision and assessment, 2) what the current training gaps are within these competencies, and 3) what barriers or enablers might exist in implementing curricular improvements.

Methods

Qualitative data were collected in two phases through individual interviews from September 2016 to November 2017, at the University of Toronto and McMaster University after receiving ethics approval from both sites. Eligible participants were final year emergency medicine residents, residents pursuing an enhanced skills program in emergency medicine, and attendings within their first 3 years of practice. A semi-structured interview guide was developed and refined after phase one, to reflect content identified in the first set of interviews. All interviews were recorded, transcribed, coded, and collapsed into themes. Data analysis was guided by constructivist grounded theory.

Results

A thematic analysis revealed five themes: 1) Supervision and assessment skills were acquired passively through modelling, 2) the training available in these areas is variably used, creating a diversity of comfort levels, 3) competing priorities in the emergency department represent significant barriers to improving supervision and assessment; 4) providing negative feedback is difficult and often avoided; and 5) competence by design will act as an impetus for formal curriculum development in these areas.

Conclusions

As programs transition to competence by design, there will be a need for formal training in supervision and assessment, with a focus on negative feedback, to achieve a standardized level of competence among emergency physicians.

Résumé

Objectifs

Les résidents en médecine d'urgence (MU) peuvent passer de la formation à la pratique en recevant une formation minimale dans la supervision et l’évaluation des stagiaires. Aussi l’étude visait-elle à examiner : a) le degré d'aisance des médecins à superviser et à évaluer des étudiants; b) les lacunes actuelles en matière de formation en ce qui concerne ces deux compétences; c) les obstacles à la mise en œuvre des améliorations à apporter au programme, ou encore les facteurs de facilitation.

Méthode

L’étude consistait en la collecte de données qualitatives faite en deux étapes, au cours d'entretiens individuels menés de septembre 2016 à novembre 2017, à l'Université de Toronto et à l'Université McMaster, après l'approbation du projet par les comités d’éthique des deux établissements. Étaient admissibles à l’étude les résidents en dernière année de formation en MU, les résidents poursuivant une formation avancée en MU et les médecins traitants en pratique depuis 3 ans ou moins. Un guide d'entretien semi-dirigé a d'abord été élaboré, puis amélioré après la première phase afin de mieux refléter les contenus relevés au cours de la première série d'entretiens. Ceux-ci ont tous été enregistrés, transcrits, codés et divisés en thèmes. L'analyse des données a été réalisée selon la théorie fondée sur le constructivisme.

Résultats

Se sont dégagés de l'analyse thématique cinq grands thèmes : l'acquisition passive des compétences en matière de supervision et d’évaluation à l'aide de modèles; la formation offerte dans ces compétences varie énormément, ce qui entraîne des écarts importants du degré d'aisance en la matière; les priorités concurrentes au service des urgences représentent une source importante d'obstacles à l'amélioration de la supervision et de l’évaluation; la difficulté à exprimer de mauvaises rétroactions se traduit souvent par une tendance à les éviter; et l'approche selon la compétence par conception imprimera une impulsion à l’élaboration d'un programme structuré dans ces deux domaines.

Conclusion

Comme les différents programmes passeront progressivement à la formule de la compétence par conception, la nécessité d'une formation structurée en matière de supervision et d’évaluation, notamment de mauvaises rétroactions, s'imposera d'elle-même, ce qui permettra aux urgentologues d'acquérir un degré comparable de compétences.

Type
Original Research
Copyright
Copyright © Canadian Association of Emergency Physicians 2019 

CLINICIAN'S CAPSULE

What is known about the topic?

A gap exists in our understanding of how well-prepared newly certified emergency physicians feel with supervision and assessment of trainees.

What did this study ask?

How prepared do senior emergency medicine residents and new-to-practice physicians feel with supervision and assessment of trainees?

What did this study find?

Residents are transitioning to practice with little formal training on how to supervise and assess trainees.

Why does this study matter to clinicians?

Clinician educators will need to play a larger role in developing curricula to instruct residents on how to perform these skills.

INTRODUCTION

Multiple organizations have identified transitions from training to practice as unique periods in the medical life-cycle which can be stressful and burdensome.Reference Frank, Snell and Sherbino1Reference Westerman, Teunissen and van der Vleuten5 The current Canadian paradigm allows residents to progress through their training with graded, progressive responsibility,6 a model whose efficacy has not been demonstrated and whose expectations are at times ambiguous.Reference Kennedy, Regehr, Baker and Lingard7 While many training programs have commenced transitioning to a competency-based educational model, including emergency medicine (EM), this transition is in its infancy. However, regardless of the training paradigm used, inherent in the transition from training to practice is the change from being a supervised trainee to a supervising physician, often for former resident colleagues at senior training levels.

Research involving the medical life-cycle of EM residents has largely focused on the transition from medical student to resident, rather than from resident to independent practice.Reference Cohen-Schotanus, Schonrock-Adema and Bouwkamp-Timmer8Reference Prince, Van De Wiel and Scherpbier10 Supervision and assessment investigations have focused primarily on junior trainees,Reference Orman and Thornton11Reference Touchie, De Champlain and Pugh15 leaving a gap in our understanding of how comfortable senior trainees feel when supervising and assessing other senior trainees. While many programs have adopted “residents as teachers” programs, these programs focus primarily on the competencies of bedside teaching, not supervision or assessment.16Reference Olmos-Vega, Dolmans, Donkers and Stalmeijer18 There are some free open access medical education (FOAMed) materials that are trying to address this gap in training,Reference Chan and Sherbino1920 as well as curricula in the literature to ease the transition to independent practice.Reference Stodel, Wyand and Crooks21Reference Dahn, Watts, Best and Bowes23 However, within these materials, there are few, if any, mentions of developing or consolidating skills in supervision and assessment, and outcome measures are scarce.

Given that the Royal College of Physicians and Surgeons of Canada (RCPSC) and the College of Family Physicians of Canada (CFPC) identify supervision and assessment as core competencies,Reference Frank, Snell and Sherbino1Reference Tannenbaum, Konkin and Parsons2 our study sought to explore senior emergency residents’ and new-to-practice emergency physicians’ sense of preparedness to supervise and assess trainees. We also sought to understand whether there is a perceived need for more formal training in supervision and assessment, and to appreciate what barriers and/or enablers might exist in implementing it.

METHODS

Methodology overview

Our qualitative design was informed by constructivist grounded theory. Grounded theory is a methodology that emphasizes the generation of theory rooted in data.Reference Glaser and Strauss24 Constructivist grounded theory is a modernized version that takes on a relativist approach, recognizing the diverse viewpoints and realities of the investigators and the participants, and uses a reflexive stance towards situations, actions, and people – thereby allowing investigators to develop hypotheses to guide the qualitative interviewing and analysis.Reference Charmaz, Holstein and Gubrium25Reference Charmaz26 Constructivist grounded theory has been used broadly within healthcare because its guidelines can assist in the examination and subsequent understanding of social, psychological, and behavioural processes.Reference Braun and Clarke27 It allowed us to understand how emergency residents and new attendings developed skills and behaviours through their training history and professional experiences.

The study received ethics approval from the Sunnybrook Health Sciences Centre Research Ethics Board and the Hamilton Integrated Research Ethics Board. Data were collected between September 2016 and November 2017.

Participant criteria and recruitment

Eligible participants were from and/or affiliated with the University of Toronto or McMaster University and were either a fifth year RCPSC EM resident (PGY5), a resident completing an enhanced skills program in EM through the CFPC [CCFP(EM), certification in family medicine with added competency in emergency medicine], or an EM physician within his or her first 3 years of practice. An email invitation and Letter of Information were sent to all PGY5 EM and CCFP(EM) residents and to all EM attendings affiliated with either site by their respective program administrator or director. Potential participates self-identified to a study investigator to schedule an interview.

Semi-structured interviews

A semi-structured interview guide was developed based on pertinent areas of interest from the literature, while favouring open-ended questions designed to elicit experiential data. Interviews were conducted either in person or via Skype with the same co-investigator (SK). Each interview lasted 30–45 minutes, after completion of a verbal consent checklist.

Two phases of interviews were held to allow for refinements to the interview guide.

Phase one took place between September 2016 and July 2017 at the University of Toronto, and phase two between August 2017 and November 2017 at the University of Toronto and McMaster University. Due to scheduling conflicts, we were unable to conduct any interviews with participants from McMaster during phase one. Our final sample size of 18 participants was guided by thematic saturation; interviews were stopped after all investigators agreed that all themes had been fully captured, and that successive interviews did not reveal any further themes or expansions upon relevant themes.5,28–32 See Table 1 for participant distribution.

Table 1. Distribution of interview participants

CCFP(EM) = certification in family medicine with added competency in emergency medicine; PGY = postgraduate year.

Data analysis

A data analysis examined how participants constructed meanings and actions related to their experiences of supervision and assessment by coding for what was commonly occurring and by comparing responses between participants. Experiences and emotional responses involving supervision and assessment were of specific importance.

The constructivist grounded theory framework involved these steps: 1) simultaneous data collection and analysis, 2) exploration of emergent themes by a constant comparative method, 3) construction of abstract categories that explain these processes, and 4) integration of categories into a theoretical framework.Reference Braun and Clarke27 The analytic team consisted of the principal investigator (KP), a newly practicing RCPSC-certified emergency physician, co-investigator (SK), a senior RCPSC EM resident, and co-investigator (JZ), a psychiatrist and expert in qualitative research methodologies. Interviews were audio-recorded, deidentified, and transcribed verbatim by an external company and were uploaded into NVivo 11 (QRS International, Burlington, Massachusetts). An analysis was performed in parallel with data collection in two separate phases. Interview transcripts were coded by co-investigator SK who consolidated the data into broader themes using open-coding, memo-writing, and categorizing.Reference Plas and Kvale29 Principal investigator KP and co-investigator JZ met regularly with co-investigator SK to discuss the coding and emerging themes. JZ provided methodological support and coded several transcripts independently to ensure consistency and review themes. Responses from each academic institution were also compared to look for any differences that may have emerged.

RESULTS

Five major themes were identified: 1) Supervision and assessment skills were acquired passively through modelling; 2) the training available in these areas is variably used, creating a diversity of comfort levels; 3) competing priorities in the emergency department (ED) represent significant barriers to improving supervision and assessment; 4) providing negative feedback is difficult and often avoided; and 5) competence by design will act as an impetus for formal curriculum development in these areas. For a summary of these five themes and representative quotations, see Table 2.

Table 2. Summary of overarching themes with representative quotations

CCFP(EM) = certification in family medicine with added competency in emergency medicine; ED = emergency department; PGY = postgraduate year.

Modelling behaviours

Regardless of the academic centre, residents and attendings described learning how to supervise and assess trainees passively through observation of those more senior, modelling those behaviours, and then using a “trial and error” method of identifying which strategies worked best for them personally.

It was a little bit of trial and error in the sense of trying things and seeing what people would respond to, trying something different and seeing if that worked better. (Attending)

When asked whether this was useful or ideal, responses varied; some agreed, with the caveat that they did not see any other way, while others hoped for change. The consensus amongst most participants was that, while non-clinical skills like supervision and assessment are more challenging to teach in a didactic manner, there are opportunities – didactic or otherwise – to improve how we acquire them.

Training and comfort levels

Residents and attendings from both universities identified the scarcity of formal training on supervision and assessment during residency. Residents identified a maximum of two sessions within residency, and some external “residents as teachers” sessions; however, most agreed that the latter were not focused on supervision or assessment.

Participants also identified heterogeneity in exposure to informal learning opportunities where they might practise supervision and assessment such as on dedicated “supervising” shifts. Some residents started supervising trainees in their PGY3 year, whereas others started earlier, including as a senior clerk. There seemed to be greater opportunities to supervise at academic centres and relatively few opportunities at affiliated community sites. Residents specifically noted that they felt uncomfortable supervising fellow senior residents because they had little opportunity to do so. Some participants also mentioned a lack of clear expectations in terms of what the natural progression of these competencies look like.

It's easier to dichotomize an R5 versus an R1. But 2–3, 3–4, 4–5, those little differences in years, I think it's very tough. I don't know what normal is. (Attending)

Most residents who had undertaken extra courses in medical education, such as a master's degree, the ED STAT! course, or a certificate-based teaching program (“Master Teacher” or “University Teaching” program), noted feeling more comfortable with supervision and assessment due to having greater exposure to these topics within these programs.

Competing priorities in the ED

Residents and attendings made mention of the many competing priorities to balance during a shift. When asked where supervision and assessment sat within the priority hierarchy, and how these competencies were considered and managed during a shift, most residents mentioned splitting up responsibilities but felt they lacked knowledge on how to execute it. Some PGY5 residents identified being made responsible for all aspects of ED management without any formal training.

We're given options on shift to work on various roles. You could look at it as flow of the department, care of your own patients and then supervising trainees. We don't really get any formal training on how to balance those priorities. (PGY5)

Residents unanimously stated that responsibility for ensuring a well-balanced experience between teaching, supervising, and developing one's own clinical acumen, was theirs. However, not all residents or attendings felt a strong impetus to push themselves to improve or focus on these skills independently. Some residents identified other priorities in their senior years, such as certification examinations and future practice plans, as barriers to honing their skills, whereas attendings unanimously mentioned departmental flow as being the biggest barrier to being able to be an effective supervisor and teacher.

It can slow you down from a flow perspective…. One of the struggles of being a junior staff is I feel like I'm really expected to see a bunch of patients and work on my flow. (Attending)

Negative feedback and assessment

Residents and attendings unanimously chose providing negative feedback as the most challenging supervisor and assessor task, and all mentioned a lack of formal training in this. Participants brought up a challenge unique to the ED: the expectation of making a judgement regarding a trainee, despite working with them for only one shift. This led to feelings of awkwardness and guilt about not having the authority or the right to provide negative feedback after such short encounters.

Additionally, residents mentioned feeling inadequate in providing negative feedback based on their overall lack of experience in being a formal assessor, and not knowing how a trainee should be performing for their PGY-level. This was also brought up by some attendings who admitted that they were unsure of their ability to accurately identify a failing trainee.

If I had more experience with other learners… and I knew what [their] baseline was, then it [giving negative feedback] would be okay. But I haven't really seen dozens of learners where I can actually say this is the standard that you should be at. (Attending)

Other difficulties with providing negative feedback included potentially offending the trainee, a defensive trainee, and fear of affecting a trainee's progression. Attendings also discussed fear of academic reprisal and a lack of support from their respective departments.

People who have academic positions are working hard to go from being assistant to associate to full professors, and they're worried about teaching evaluations[sic] being negative. I do worry that if I have a very negative teaching evaluation [sic], that will impact my academic progression. (Attending)

Some attendings indicated that some degree of opinion about a trainee comes from hearsay. When assessors have limited interactions with trainees and a lack of clear expectations for where residents ought to be with respect to training level, assessors will establish an impression from informally asking colleagues “by the water cooler.”

The move towards competence by design

As programs transition to competence by design, supervision and assessment will be necessary to ensure that residents demonstrate competence in entrustable professional activities and milestones.33 Participants identified advantages and disadvantages in how competence by design could influence or change supervision and assessment.

The main benefit described by most participants was the hope that trainees would get more frequent and specific feedback.

You can go through residency getting the same evaluation [sic], and a lot of people that are weak don't get honest results. I think there will be a lot of benefit to having feedback that targets a specific behaviour that is observed and actionable. (Attending)

Additional benefits of competence by design included greater continuity of assessments and shifting the feedback atmosphere to one of coaching and entrustment. However, others were concerned that competence by design will not be able to capture all of the skills that an emergency physician needs and will mean an increased amount of work for physicians in an already overburdened system.

At the end of your training, you're going to have a couple of hundred data points, but is that sufficient to say that someone is competent in all of the major things that you need to be in order to perform EM? I'm not sure that it is. (Attending)

DISCUSSION

Our study used thematic analysis informed by the tenets of constructivist grounded theory to explore senior residents’ and new-to-practice emergency physicians’ sense of preparedness to supervise and assess trainees, and understand how current EM curricula support or detract from the development of these competencies.

Our findings suggest that residents are transitioning to practice with little formal training on how to effectively supervise and assess trainees. These results mirror those by Westerman et al., which demonstrated that, while transitioning internists and obstetricians felt prepared to fulfill their role clinically, they did not feel prepared for nonclinical tasks, including supervision and assessment.Reference Westerman, Teunissen and van der Vleuten5 Berendonk et al. also identified three areas of discomfort regarding assessment among supervisors: the importance of being a content expert to be a credible assessor, familiarity with the level of knowledge possessed by the trainee being assessed, and the self-efficacy of the assessor.Reference Berendonk, Stalmeijer and Schuwirth34 This aligns with our work in that participants in our study identified that determining the appropriate performance level of a trainee, particularly those in adjacent years, is difficult and presents a unique challenge.

Our study also reinforces the findings of Dudek et al. who demonstrated that there are multiple barriers to documenting poor performance and providing negative feedback, and that when there is virtually no continuity of feedback between assessors, it can be easy for a struggling resident to fall through the cracks.Reference Dudek, Marks and Regehr35 Our study also confirms that a substantial number of attendings avoid providing negative feedback for fear that a poor teaching evaluation could impact a promotion.Reference Dudek, Marks and Regehr3537

Competence by design has the potential to address these issues by creating a culture of coaching within medical education, rather than the current culture of staged supervision and assessment. Further, as competence by design assessment tools are developed, it will be critical that residents are afforded the opportunity to practise these skills. It was clear from our interviews that residents in the RCPSC program had significantly more opportunities to supervise and assess junior residents than their CCFP(EM) colleagues. However, both programs suffered from a lack of experience in supervising and assessing senior resident colleagues, a task expected of a new attending physician. It seems that more attention needs to be paid to exploring how educators can better prepare senior residents to supervise other senior resident colleagues.

We also found that role modelling regarding how to supervise and assess was universal. Our study begs the question: “Who is the role model?” Have they demonstrated adequate competence for this behaviour? Would this kind of learning meet the standards set out by a certification college? These are important questions that demand further research if we continue to rely on this as the primary learning method.

Role modelling may also be linked to supervising physicians acting as mentors and coaches. As described by Watling et al., the culture of medical education often involves residents working in parallel with supervisors, with discrete periods of direct observation that progressively decline over the course of training as residents become more independent.Reference Watling, Driessen and van der Vleuten38

Watling et al. also identified that medical trainees, as compared to musicians, seek out clinical excellence in their supervisors as a marker of their credibility, rather than excellence in instruction or coaching.Reference Watling, Driessen and van der Vleuten38 Reconciling these previous findings with the findings in our study that role modelling plays an important role in the development of skills in supervision and assessment will be an important area of further research.

Limitations

This small-scale study captured the opinions of emergency physicians and trainees at only two institutions, limiting its generalizability. Our convenience sampling technique resulted in our interviewing only two CCFP(EM) residents and only four residents from McMaster; this limits our ability to draw conclusions from the CCFP(EM) cohort and from McMaster. However, despite our small sample size at McMaster, we still were able to reach thematic saturation.

Our data sets were primarily coded by a single reviewer (SK), which could introduce bias into the themes. We attempted to mitigate this in the following ways: 1) The principal investigator and co-author JZ coded a minimum number of transcripts, 2) the coding done by SK was reviewed by the principal investigator and co-author JZ for accuracy, and 3) all three authors were involved in discussions on thematic generation. Lastly, whereas the principal investigator is an EM physician and co-author SK is an EM resident, co-investigator JZ is not and therefore assisted with engaging KP and SK in self-reflexivity to limit potential bias.

CONCLUSIONS

The transition from residency to independent practice is a challenging time for any physician. Our study sought to identify how prepared senior EM residents and new-to-practice physicians felt when supervising and assessing trainees. Our findings suggest that both groups feel they do not have adequate training in these areas. Our study also found that managing the competing priorities of an ED and providing negative feedback also lack formal training and need to be addressed in future curricula. We hope that our results will provide support to educators and spur the development of dedicated curricula to address these deficiencies.

Financial support

This work was generously supported by the Canadian Association of Emergency Physicians (CAEP) Junior Investigator Research Grant Program (Control ID: 2661311).

Competing interests

None declared.

References

REFERENCES

1.Frank, J, Snell, L, Sherbino, J. The Draft CanMEDS 2015 physician competency framework. Ottawa, ON: The Royal College of Physicians and Surgeons of Canada; 2015.Google Scholar
2.Tannenbaum, D, Konkin, J, Parsons, E, et al. CanMEDS – Family Medicine 2017: a competency framework for family physicians across the continuum. Mississauga, ON: The College of Family Physicians of Canada; 2017.Google Scholar
3.Health Force Ontario. Ontario: c2018. Transition into practice service (TiPS) toolkit; 2017. Available at: http://www.healthforceontario.ca/en/Home/All_Programs/Practice_Ontario/TiPS (accessed August 8, 2018).Google Scholar
4.PARO. Ontario: c2018. Starting residency; 2019. Available at: http://myparo.ca/After_Residency (accessed February 14, 2019).Google Scholar
5.Westerman, M, Teunissen, PW, van der Vleuten, CPM, et al. Understanding the transition from resident to attending physician: a transdisciplinary, qualitative study. Acad Med 2010;85(12):1914–9.Google Scholar
6.Royal College of Physicians and Surgeons of Canada. General standards applicable to all residency programs – B standards. Ottawa, ON: Royal College of Physicians and Surgeons of Canada; 2011.Google Scholar
7.Kennedy, TJ, Regehr, G, Baker, GR, Lingard, LA. Progressive independence in clinical training: a tradition worth defending? Acad Med 2005;80(10 Suppl):S106-11.Google Scholar
8.Cohen-Schotanus, J, Schonrock-Adema, J, Bouwkamp-Timmer, T, et al. One-year transitional programme increases knowledge to level sufficient for entry into the fourth year of the medical curriculum. Med Teach 2008;30:62–6.Google Scholar
9.van Hell, EA, Kuks, JB, Schonrock-Adema, J, et al. Transition to clinical training: influence of pre-clinical knowledge and skills, and consequences for clinical performance. Med Educ 2008;42(8):830–7.Google Scholar
10.Prince, KJ, Van De Wiel, M, Scherpbier, AJ, et al. A qualitative analysis of the transition from theory to practice in undergraduate training in a PBL-medical school. Adv Health Sci Educ Theory Pract 2000;5(2):105–16.Google Scholar
11.Orman, SA, Thornton, VJ. Analysis of junior doctor supervision in Australasian emergency departments. Emerg Med Australas 2010;22(4):301–39.Google Scholar
12.Craig, S, Braitberg, G, Nicolas, C, et al. Assessment and feedback in emergency medicine training: views of Australasian emergency trainees. Emerg Med Australas 2010;22(6):537–47.Google Scholar
13.Sacchetti, A, Carraccio, C, Harris, RH. Resident management of emergency department patients: is closer attending supervision needed? Ann Emerg Med 1992;21(6):749–52.Google Scholar
14.Jelinek, GA, Weiland, TJ, Mackinlay, C. Supervision and feedback for junior medical staff in Australian emergency departments: findings from the emergency medicine capacity assessment study. BMC Med Educ 2010;10:74.Google Scholar
15.Touchie, C, De Champlain, A, Pugh, D, et al. Supervising incoming first-year residents: faculty expectations versus residents’ experiences. Med Educ. 2014;48(9):921–9.Google Scholar
16.PARO. Ontario: c2018. Teaching during residency; 2019. Available at: http://myparo.ca/Teaching_During_Residency (accessed February 14, 2019).Google Scholar
17.Rogers, RL, Mattu, A, Winters, M, Martinez, J. Practical teaching in emergency medicine. Chichester, Sussex: Blackwell Publishing Ltd; c2009.Google Scholar
18.Olmos-Vega, F, Dolmans, D, Donkers, J, Stalmeijer, RE. Understanding how residents’ preferences for supervisory methods change throughout residency training: a mixed-methods study. BMC Med Educ 2015;15:177.Google Scholar
19.Chan, T, Sherbino, J. The McMaster Modular Assessment Program (McMAP): a theoretically grounded work-based assessment system for an emergency medicine residency program. Acad Med 2015;90(7):900–5.Google Scholar
20.ALiEM: Academic Life in Emergency Medicine. MedIC: Medical education in cases; 2018. Available at: https://www.aliem.com/category/non-clinical/medic-series/ (accessed December 12, 2018).Google Scholar
21.Stodel, EJ, Wyand, A, Crooks, S, et al. Designing and implementing a competency-based training program for anesthesiology residents at the University of Ottawa. Anesthesiol Res Pract. 2015;713038.Google Scholar
22.Lister, JR, Friedman, WA, Murad, GJ, et al. Evaluation of a transition to practice program for neurosurgery residents: creating a safe transition from resident to independent practitioner. J Grad Med Educ 2010;2(3):366–72.Google Scholar
23.Dahn, H, Watts, K, Best, L, Bowes, D. Transition to practice: creation of a transitional rotation for radiation oncology. Can Med Educ J 2018;9(3):e89e96.Google Scholar
24.Glaser, B, Strauss, A. The discovery of grounded theory: strategies for qualitative research. New York: Walter de Gruyter, Hawthorne; 1967.Google Scholar
25.Charmaz, K. Qualitative interviewing and grounded theory analysis. In: Inside Interviewing New Lenses, New Concerns (eds. Holstein, J, Gubrium, J). Thousand Oaks, CA: Sage; 2003.Google Scholar
26.Charmaz, K. Constructing grounded theory: a practical guide through qualitative analysis. Rohnert Park, CA: Sage; 2006.Google Scholar
27.Braun, V, Clarke, V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77101.Google Scholar
28.Cohen, D, Crabtree, B. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med 2008;6:331–9.Google Scholar
29.Plas, JM, Kvale, S. InterViews: an introduction to qualitative research interviewing. Thousand Oaks, CA: Sage; 1996.Google Scholar
30.Corbin, J, Strauss, A. Basics of qualitative research: techniques and procedures for developing grounded theory. 3rd ed. Los Angeles, CA: Sage; 2008.Google Scholar
31.Bauer, MW, Gaskell, G. Qualitative researching with text, image and sound: a practical handbook. London: Sage; 2000.Google Scholar
32.Creswell, JW. Qualitative inquiry and research design choosing among five traditions. Thousand Oaks, CA: Sage; 1998.Google Scholar
33.Royal College of Physicians and Surgeons of Canada. Entrustable professional activities for emergency medicine. Ottawa, ON: Royal College of Physicians and Surgeons of Canada; 2018.Google Scholar
34.Berendonk, C, Stalmeijer, RE, Schuwirth, LW. Expertise in performance assessment: assessors' perspectives. Adv Health Sci Educ 2012;18(4):559–71.Google Scholar
35.Dudek, N, Marks, MB, Regehr, G. Failure to fail: the perspectives of clinical supervisors. Acad Med 2005;80(10):S847.Google Scholar
36.University of Toronto, Faculty of Medicine. Manual for academic promotion to associate professor and professor; 2018. Available at: https://medicine.utoronto.ca/sites/default/files/July_2018%20Academic%20Promotions%20Manual%20-%20Final_2.pdf (accessed December 19, 2018).Google Scholar
37.McMaster University. McMaster University revised policy and regulations with respect to academic appointment, tenure and promotion; 2012. Available at: https://www.mcmaster.ca/policy/faculty/Appointments/Tenure_and_Promotion_January2012.pdf (accessed December 19, 2018).Google Scholar
38.Watling, C, Driessen, E, van der Vleuten, C, et al. Music lessons: revealing medicine's learning culture through a comparison with that of music. Med Educ 2013;47(8):842–50.Google Scholar
Figure 0

Table 1. Distribution of interview participants

Figure 1

Table 2. Summary of overarching themes with representative quotations