Introduction: In 2018, Canadian postgraduate specialist Emergency Medicine (EM) programs began implementing a competency-based medical education (CBME) assessment system. To support improvement of this assessment program, we sought to evaluate its short-term educational outcomes nationally and within individual programs. Methods: Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of Entrustable Professional Activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages was compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations. Results: Data from 15 of 17 (88.2%) EM programs containing 9,842 EPA observations from 68 of the 77 (88.3%) Canadian EM specialist residents in the 2018 cohort were analyzed. The average number of EPAs observed per resident in each program varied from 92.5 to 229.6 and correlated strongly with the number of blocks spent on EM and pediatric EM (r = 0.83, p < 0.001). Relative to the guidelines outlined by the specialty committee, residents were promoted later than expected and with fewer EPA observations than suggested. Conclusion: We present a new approach to the amalgamation of national and program-level assessment data. There was demonstrable variation in both EPA-based assessment numbers and promotion timelines between programs and with national guidelines. This evaluation data will inform the revision of local programs and national guidelines and serve as a starting point for further reaching outcome evaluation. This process could be replicated by other national assessment programs.

Introduction: Cricothyrotomy is an intervention performed to salvage “can't intubate, can't ventilate” situations. Studies have shown poor accuracy landmarking the cricothyroid membrane, particularly in female patients by surgeons and anesthesiologists. There is less data available about emergency physician performance. This study examines the perceived versus actual success rate of landmarking the cricothyroid membrane by resident and staff emergency physicians using obese and non-obese models. Methods: Five male and female volunteers were selected as models. Each model was placed supine, and a point-of-care ultrasound expert landmarked the borders of each cricothyroid membrane. 20 residents and 15 staff emergency physicians were given one attempt to landmark five models. Data was gathered on each participant's perceived likelihood of success and attempt difficulty. Overall accuracy and accuracy stratified by sex and obesity status were calculated. Results: Overall landmarking accuracy amongst all participants was 58% (SD 18%). A difference in accuracy was found for obese males (88%) versus obese females (40%) (difference = 48%, 95% CI = 30-65%, p < 0.0001); and non-obese males (77%) versus non-obese females (46%) (difference = 31%, 95% CI = 12-51%, p = 0.004). There was no association between perceived difficulty and success (correlation = 0.07, 95% CI=−0.081-0.214, p = 0.37). Confidence levels overall were higher amongst staff physicians (3.0) than residents (2.7) (difference = 0.3, 95% CI = 0.1-0.6, p = 0.02), but there was no correlation between confidence in an attempt and its success (p = 0.33). Conclusion: We found that physicians demonstrate significantly lower accuracy when landmarking cricothyroid membranes of females. Emergency physicians were unable to predict their own accuracy while landmarking, which can potentially lead to increased failed attempts and longer time to secure the airway. Improved training techniques and a modified approach to cricothyrotomy may reduce failed attempts and improve the time to secure the airway.

Introduction: Workplace based assessments (WBAs) are integral to emergency medicine residency training. However many biases undermine their validity, such as an assessor's personal inclination to rate learners leniently or stringently. Outlier assessors produce assessment data that may not reflect the learner's performance. Our emergency department introduced a new Daily Encounter Card (DEC) using entrustability scales in June 2018. Entrustability scales reflect the degree of supervision required for a given task, and are shown to improve assessment reliability and discrimination. It is unclear what effect they will have on assessor stringency/leniency – we hypothesize that they will reduce the number of outlier assessors. We propose a novel, simple method to identify outlying assessors in the setting of WBAs. We also examine the effect of transitioning from a norm-based assessment to an entrustability scale on the population of outlier assessors. Methods: This was a prospective pre-/post-implementation study, including all DECs completed between July 2017 and June 2019 at The Ottawa Hospital Emergency Department. For each phase, we identified outlier assessors as follows: 1. An assessor is a potential outlier if the mean of the scores they awarded was more than two standard deviations away from the mean score of all completed assessments. 2. For each assessor identified in step 1, their learners’ assessment scores were compared to the overall mean of all learners. This ensures that the assessor was not simply awarding outlying scores due to working with outlier learners. Results: 3927 and 3860 assessments were completed by 99 and 116 assessors in the pre- and post-implementation phases respectively. We identified 9 vs 5 outlier assessors (p = 0.16) in the pre- and post-implementation phases. Of these, 6 vs 0 (p = 0.01) were stringent, while 3 vs 5 (p = 0.67) were lenient. One assessor was identified as an outlier (lenient) in both phases. Conclusion: Our proposed method successfully identified outlier assessors, and could be used to identify assessors who might benefit from targeted coaching and feedback on their assessments. The transition to an entrustability scale resulted in a non-significant trend towards fewer outlier assessors. Further work is needed to identify ways to mitigate the effects of rater cognitive biases.

Introduction: The Emergency Medicine Specialty Committee of the Royal College of Physicians and Surgeons of Canada (RCPSC) has specified that resuscitation Entrustable Professional Activities (EPAs) can be assessed in either the workplace or simulation environments; however, there is minimal evidence that such clinical performance correlates. We sought to determine the relationship between assessments in the workplace versus simulation environments among junior emergency medicine residents. Methods: We conducted a prospective observational study to compare workplace and simulation resuscitation performance among all first-year residents (n = 9) enrolled in the RCPSC-Emergency Medicine program at the University of Ottawa. All scores from Foundations EPA #1 (F1) were collected during the 2018-2019 academic year; this EPA focuses on initiating and assisting in the resuscitation of critically ill patients. Workplace performance was assessed by clinical supervisors by direct observation during clinical shifts. Simulation performance was assessed by trained simulation educators during regularly-scheduled sessions. We present descriptive statistics and within-subjects analyses of variance. Results: We collected a total of 104 workplace and 36 simulation assessments. Interobserver reliability of simulation assessments was high (ICC = 0.863). We observed no correlation between mean EPA scores assigned in the workplace and simulation environments (Spearman's rho=−0.092, p = 0.813). Scores in both environments improved significantly over time (F(1,8) = 18.79, p < 0.001, ηp2 = 0.70), from 2.9(SD = 1.2) in months 1-4 to 3.5(0.2) in months 9-12 (p = 0.002). Workplace scores (3.4(0.1)) were consistently higher than simulation scores (2.9(0.2)) (F(1,8) = 7.16, p = 0.028, ηp2 = 0.47). Conclusion: We observed no correlation between EPA F1 ratings of resuscitation performance between the workplace and simulation environments. Further studies should seek to clarify this relationship to inform our ongoing use of simulation to assess clinical competence.

Introduction: The Ottawa Emergency Department Shift Observation Tool (O-EDShOT) was recently developed to assess a resident's ability to safely run an ED shift and is supported by multiple sources of validity evidence. The O-EDShOT uses entrustability scales, which reflect the degree of supervision required for a given task. It was found to discriminate between learners of different levels, and to differentiate between residents who were rated as able to safely run the shift and those who were not. In June 2018 we replaced norm-based daily encounter cards (DECs) with the O-EDShOT. With the ideal assessment tool, most of the score variability would be explained by variability in learners’ performances. In reality, however, much of the observed variability is explained by other factors. The purpose of this study is to determine what proportion of total score variability is accounted for by learner variability when using norm-based DECs vs the O-EDShOT. Methods: This was a prospective pre-/post-implementation study, including all daily assessments completed between July 2017 and June 2019 at The Ottawa Hospital ED. A generalizability analysis (G study) was performed to determine what proportion of total score variability is accounted for by the various factors in this study (learner, rater, form, pgy level) for both the pre- and post- implementation phases. We collected 12 months of data for each phase, because we estimated that 6-12 months would be required to observe a measurable increase in entrustment scale scores within a learner. Results: A total of 3908 and 3679 assessments were completed by 99 and 116 assessors in the pre- and post- implementation phases respectively. Our G study revealed that 21% of total score variance was explained by a combination of post-graduate year (PGY) level and the individual learner in the pre-implementation phase, compared to 59% in the post-implementation phase. An average of 51 vs 27 forms/learner are required to achieve a reliability of 0.80 in the pre- and post-implementation phases respectively. Conclusion: A significantly greater proportion of total score variability is explained by variability in learners’ performances with the O-EDShOT compared to norm-based DECs. The O-EDShOT also requires fewer assessments to generate a reliable estimate of the learner's ability. This study suggests that the O-EDShOT is a more useful assessment tool than norm-based DECs, and could be adopted in other emergency medicine training programs.

Introduction: Competency based medical education (CBME) has triggered widespread utilization of workplace-based assessment (WBA) tools in postgraduate training programs. These WBAs predominately use rating scales with entrustment anchors, such as the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE). However, little is known about the factors that influence a supervising physician's decision to assign a particular rating on scales using entrustment anchors. This study aimed to identify the factors that influence supervisors’ ratings of trainees using WBA tools with entrustment anchors at the time of assessment and to explore the experiences with and challenges of using entrustment anchors in the emergency department (ED). Methods: A convenience sample of full-time emergency medicine (EM) faculty were recruited from two sites within a single academic Canadian EM hospital system. Fifty semi-structured interviews were conducted with EM physicians within two hours of completing a WBA for an EM trainee. Interviews were audio-recorded, transcribed verbatim, and independently analyzed by two members of the research team. Themes were stratified by trainee level, rating and task. Results: Interviews involved 73% (27/37) of all EM staff and captured assessments completed on 83% (37/50) of EM trainees. The mean WBA rating of studied samples was 4.34 ± 0.77 (2 to 5), which was similar to the mean rating of all WBAs completed during the study period. Overall, six major factors were identified that influenced staff WBA ratings: amount of guidance required, perceived competence through discussion and questioning, trainee experience, clinical context, past experience working with the trainee, and perceived confidence. The majority of staff denied struggling to assign ratings. However, when they did struggle, it involved the interpretation of WBA anchors and their application to the clinical context in the ED. Conclusion: Several factors appear to be taken into account by clinical supervisors when they make decisions regarding the particular rating that they will assign a trainee on a WBA that uses entrustment anchors. Not all of these factors are specific to that particular clinical encounter. The results from this study further our understanding on the use of entrustment anchors within the ED and may facilitate faculty development regarding WBA completion as we move forward in CBME.

Germ plasm, a cytoplasmic factor of germline cell differentiation, is suggested to be a perspective tool for in vitro meiotic differentiation. To discriminate between the: (1) germ plasm-related structures (GPRS) involved in meiosis triggering; and (2) GPRS involved in the germ plasm storage phase, we investigated gametogenesis in the marine medaka Oryzias melastigma. The GPRS of the mitosis-to-meiosis period are similar in males and females. In both sexes, five events typically occur: (1) turning of the primary Vasa-positive germ plasm granules into the Vasa-positive intermitochondrial cement (IMC); (2) aggregation of some mitochondria by IMC followed by arising of mitochondrial clusters; (3) intramitochondrial localization of IMC-originated Vasa; followed by (4) mitochondrial cluster degradation; and (5) intranuclear localization of Vasa followed by this protein entering the nuclei (gonial cells) and synaptonemal complexes (zygotene–pachytene meiotic cells). In post-zygotene/pachytene gametogenesis, the GPRS are sex specific; the Vasa-positive chromatoid bodies are found during spermatogenesis, but oogenesis is characterized by secondary arising of Vasa-positive germ plasm granules followed by secondary formation and degradation of mitochondrial clusters. A complex type of germ plasm generation, ‘the follicle cell assigned germ plasm formation’, was found in late oogenesis. The mechanisms discovered are recommended to be taken into account for possible reconstruction of those under in vitro conditions.

Innovation Concept: The outcome of emergency medicine training is to produce physicians who can competently run an emergency department (ED) shift. While many workplace-based ED assessments focus on discrete tasks of the discipline, others emphasize assessment of performance across the entire shift. However, the quality of assessments is generally poor and these tools often lack validity evidence. The use of entrustment scale anchors may help to address these psychometric issues. The aim of this study was to develop and gather validity evidence for a novel tool to assess a resident's ability to independently run an ED shift. Methods: Through a nominal group technique, local and national stakeholders identified dimensions of performance reflective of a competent ED physician. These dimensions were included in a new tool that was piloted in the Department of Emergency Medicine at the University of Ottawa during a 4-month period. Psychometric characteristics of the items were calculated, and a generalizability analysis used to determine the reliability of scores. An ANOVA was conducted to determine whether scores increased as a function of training level (junior = PGY1-2, intermediate = PGY3, senior = PGY4-5), and varied by ED treatment area. Safety for independent practice was analyzed with a dichotomous score. Curriculum, Tool or Material: The developed Ottawa Emergency Department Shift Observation Tool (O-EDShOT) includes 12-items rated on a 5-point entrustment scale with a global assessment item and 2 short-answer questions. Eight hundred and thirty-three assessment were completed by 78 physicians for 45 residents. Mean scores differed significantly by training level (p < .001) with junior residents receiving lower ratings (3.48 ± 0.69) than intermediate residents who received lower ratings (3.98 ± 0.48) than senior residents (4.54 ± 0.42). Scores did not vary by ED treatment area (p > .05). Residents judged to be safe to independently run the shift had significantly higher mean scores than those judged not to be safe (4.74 ± 0.31 vs 3.75 ± 0.66; p < .001). Fourteen observations per resident, the typical number recorded during a 1-month rotation, were required to achieve a reliability of 0.80. Conclusion: The O-EDShOT successfully discriminated between junior, intermediate and senior-level residents regardless of ED treatment area. Multiple sources of evidence support the O-EDShOT producing valid scores for assessing a resident's ability to independently run an ED shift.

Introduction: Maintaining and enhancing competence in the breadth of Emergency Medicine (EM) is an ongoing challenge for all clinicians. In particular, resuscitative care in EM involves high-stakes clinical encounters that demand strong procedural skills, effective leadership, and up-to-date knowledge. However, Canadian emergency physicians are not required to complete any specific ongoing training for these encounters beyond general CPD requirements of professional colleges. Simulation-based medical education (SBME) is an effective modality for enhancing technical (e.g. procedural) and non-technical (i.e. Crisis Resource Management) skills in crisis situations, and has been embedded in undergraduate and postgraduate medical curricula worldwide. We present a novel comprehensive curriculum of simulation-based CPD designed specifically for academic emergency physicians (AEPs) at our centre. Methods: The curriculum development involved a departmental needs assessment survey, focus groups with AEPs, data from safety metrics and critical incidents, and consultations with senior departmental leadership. Institutional support was provided in the form of a $25,000 grant to fund a physician Program Lead, monthly session instructors, and simulation centre operating costs. Based on the results of the needs assessment, a two-year curriculum was mapped out and tailored to the available resources. Results: CPD simulation commenced in January 2017 and occurs monthly for three hours, immediately following departmental Grand Rounds to provide convenient scheduling. Our needs assessment identified two key types of educational needs: (1) Crisis Resource Management skills and (2) frequent practice of high-stakes critical care procedures (e.g. central lines). The first six months of implementation was dedicated to low-fidelity skills labs to facilitate the transition to SBME. After this, the program transitioned to a hybrid model involving two high-fidelity simulated resuscitations and one skills lab per session. Conclusion: We have introduced a comprehensive curriculum of ongoing simulation-based CPD in our department based on the educational needs of our AEPs. Key to our successful implementation has been support from educational and administrative leadership within our department. Ongoing challenges include securing adequate protected time from clinical duties for program facilitators and participants. Future work will include establishing permanent funding, CPD accreditation, and a formal program evaluation.

The flow before and after the partial impingement of a jet on a half plane has been studied. From the extensive mean velocity and turbulence intensity measurements the three basic flow regimes of the main jet flow, the flow over the flat plate and the redirected flow on the flat plate surface have been isolated. The inherited characteristics and axisymmetry of the upstream jet flow and their extent downstream of the flat plate, have also been established. The effect of the leading edge on the flow over the flat plate and the effect of friction on the surface on the redirected flow have also been studied.

Introduction: Much of the literature investigating the challenges associated with completing high quality work-based assessments (WBAs) have raised specific concerns over the appropriate documentation of assessments of underperforming trainees or trainees in difficulty. The purpose of this study was to examine the relationship between resident performance and the quality of assessments documented by supervisors on Daily Encounter Cards (DECs). The effect of trainee proximity (i.e. on-service versus off-service status) on this relationship was also examined. Methods: A series of DECs from the Department of Emergency Medicine at the University of Ottawa was scored by two raters using the Completed Clinical Evaluation Report Rating (CCERR). The CCERR is a 9-item instrument that has previously demonstrated reliable ratings and the ability to discriminate the quality of completed DECs. A proxy measure of resident performance was calculated by averaging the scores across performance items on the DEC to produce a “mean DEC rating”. Linear regression analysis was conducted with “mean DEC rating” as the independent measure and CCERR score as the dependent measure. Separate linear regression analyses were repeated for DECs completed for on-service versus off-service residents. Results: Linear regression analysis demonstrated a small but significant inverse relationship between mean DEC rating and CCERR score (p<0.001, r=-0.184), suggesting that when residents performed poorly, their supervisors tended to document higher quality assessments, and conversely, when residents performed well, their supervisors provided lower quality assessments. Further analysis demonstrated that this relationship was present for the on-service group (p<0.001, r=-0.24). However, no relationship was observed in the off-service group (p=0.62, r=-0.05). Conclusion: Resident performance and trainee proximity are important factors impacting the quality of documented clinical performance assessments. Greater attention needs to be given to determining ways of improving the quality of assessments reported for residents who are appropriately progressing in their clinical competence as well as for off-service trainees.

Introduction: Barriers to completing high quality work-based assessments (WBAs) include relational factors such as the episodic and fragmented interaction that often exists between clinical supervisors and trainees. In an effort to increase supervisor-trainee continuity, the Department of Emergency Medicine at the University of Ottawa created Clinical Teaching Teams (CTT) in which a resident and clinical supervisor work matched shifts together throughout the year. The aim of this study was to determine the impact of supervisor-trainee continuity on the quality of assessments documented on Daily Encounter Cards (DECs). Methods: DECs completed by 20 clinical supervisors were collected and sorted into three groups representing differing degrees of supervisor-trainee continuity (Group 1: CTT emergency resident; Group 2: non-CTT emergency resident; Group 3: non-CTT off-service resident). DECs were scored using the Completed Clinical Evaluation Report Rating (CCERR), a 9-item instrument that has been shown to have reliable ratings and the ability to discriminate the quality of completed DECs. Scores were analyzed using a univariate ANOVA with “mean CCERR score” as the dependent variable and “continuity group” and “supervisor” as between-subject variables. The relationship between CCERR scores and number of CTT encounters over time was examined using a repeated measures ANOVA with “encounter number” as the within-subject factor. Results: Mean CCERR scores for the CTT (21.0, SD=5.8), non-CTT (21.9, SD=4.2), and off-service (20.7, SD=4.0) groups differed (p=0.019). A subsequent pairwise comparison demonstrated a statistically significant difference in means between the non-CTT and off-service groups (p=0.04); however, this 1.2 difference on the 45-point CCERR scale is unlikely to be of any educational significance. The number of repeated encounters did not have a statistically significant effect on CCERR scores (p=0.43) indicating that DEC quality did not improve with greater supervisor-trainee interaction. Conclusion: DEC quality as scored by the CCERR was low for all three groups. Increasing supervisor continuity alone did not result in higher quality assessments of clinical performance. Additional research focusing on the educational alliance that develops between supervisor and trainee may hold greater promise.

Introduction: In response to concerns in the literature over the quality of completed work-based assessments (WBAs), faculty development and rater training initiatives have been developed. The Completed Clinical Evaluation Report Rating (CCERR) was designed to evaluate these interventions by providing a measure of the quality of documented assessments on In-Training Evaluation Reports (ITERs). Daily Encounter Cards (DECs) are a common form of WBA used in the Emergency Department setting. A tool to evaluate initiatives aimed at improving the quality of completion of this widely used WBA is also needed. The purpose of this study was to provide validity evidence to support using the CCERR to assess the quality of DEC completion. Methods: This study was conducted in the Department of Emergency Medicine at the University of Ottawa. Six experts in resident assessment grouped 60 DECs into three quality categories (high, average, poor) based on their perception of how informative each DEC was for reporting judgments of the resident’s performance. Eight clinical supervisors (blinded to the expert groupings) scored the 10 most representative DECs in each group using the CCERR. Mean scores were compared using a univariate ANOVA to determine if the CCERR was able to discriminate DEC quality. Reliability for the CCERR scores was determined using a generalizability analysis. Results: Mean CCERR scores for the high (37.3, SD=1.2), average (24.2, SD=3.3), and poor (14.4, SD=1.4) quality groups differed (p<0.001). A pairwise comparison demonstrated that differences between all three quality groups were statistically significant (p<0.001), indicating that the CCERR was able to discriminate DEC quality as judged by experts. A generalizability study demonstrated the majority of score variation was due to differences in DECs. The reliability with a single rater was 0.95. Conclusion: There is strong validity evidence to support the use of the CCERR to evaluate DEC quality. It can be used to provide feedback to supervisors for improving assessment reporting, and offers a quantitative measure of change in assessor behavior when utilized as a program evaluation instrument for determining the quality of completed DECs.

FePt:C films were prepared by filtered vacuum arc deposition. A strong dependence of coercivity and ordering of the face-central tetragonal structure on both C concentration and annealing temperature was observed. With C concentration of 21%, the sample with a coercivity of 5.7 kOe was obtained when annealing temperature was only 350 °C. Transmission electron microscope observations revealed that FePt grains with an average size of 4.1 nm were embedded in C and appeared to be well isolated.