Introduction
CHD is the most frequently diagnosed congenital disorder in newborns, occurring in around 1% of all live births. Reference Wu, He and Shao1 In addition to structural defects, infants and children can suffer from arrhythmias and cardiomyopathies. Reference Lipshultz, Cochran and Briston2,Reference Hanash and Crosson3 Electrocardiograms are recordings of the heart’s electrical activity and can provide important clues to underlying congenital or acquired heart disease. Reference Delany, Coffman, Shea and Jump4 Interpreting electrocardiograms is a complex and nuanced skill, and the literature demonstrates that when asked to read electrocardiograms, graduating medical students and residents of multiple specialties do not perform at the expected level. Reference Dickson, Chesser, Woods, Krug and Kellerman5–Reference Weinberg, Ottolini, Sestokas and Greene9 Electrocardiogram interpretation is made yet more complex in paediatric medicine by the need to understand how the changes in cardiac physiology as children age alter the expected electrocardiogram findings. Reference Weinberg, Ottolini, Sestokas and Greene9 Nevertheless, knowledge of paediatric electrocardiogram interpretation is a vital skill for paediatricians, as well as for many paediatric subspecialty services outside of paediatric cardiology, including hospital medicine, neonatal and paediatric intensive care, paediatric emergency medicine, paediatric primary care, and adolescent medicine. Reference Harris, Adler, Unti and McBride10
Because electrocardiogram interpretation is a complex skill, it is well suited to benefit from distributed practice. In educational theory, distributed practice refers to multiple short practice sessions spread out over time. Reference Gerbier and Toppino11 Distributed practice, which optimises processing time and allows for retrieval and reactivation of previously learned items, has been shown to improve retention of knowledge and skills. Reference Gerbier and Toppino11,Reference Toppino, Gerbier and Ross12 Distributed practice is a well-studied and often utilised concept in medical school curriculums. Reference Trumble, Lodge and Mandrusiak13
However, distributed practice of electrocardiogram interpretation can be difficult to achieve naturally during a paediatric residency. Residents move from rotation to rotation, and each block may utilise electrocardiograms as a diagnostic tool with different degrees of frequency and with variable supervisor understanding, guidance, and education. In addition, at the time this project was completed, cardiology was not a mandatory block at our centre, with the result that not all residents had the opportunity to rotate with the paediatric cardiology service. The specific aim of this work was therefore to address this educational gap by creating a longitudinal, email-based curriculum to increase resident comfort with paediatric electrocardiogram interpretation skills by utilising the principles of distributed practice. This work also sought to understand whether an email-based distributive practice curriculum would be feasible and effective for paediatric residents.
Method
Institutional review board approval was obtained for this study. At the start of the academic year, a pre-survey was administered to 118 paediatric and internal medicine/paediatric residents at our centre asking them to rate their confidence on a 5-point Likert scale in four electrocardiogram interpretation skills. The four skills were: How confident do you feel in your ability to: 1. Utilise a stepwise approach to interpret an electrocardiogram; 2. Recognise an abnormality on an electrocardiogram; 3. Recognise an arrhythmia on an electrocardiogram; and 4. Know when an electrocardiogram warrants urgent cardiology consultation. Basic demographic information was also collected.
During the academic year, electrocardiograms were emailed to residents one to two times per month with an associated clinical vignette. Participants were then asked to answer a multiple-choice question about that electrocardiogram. Typically, the questions asked participants to either describe a feature seen in the electrocardiogram or to choose a next step in management. For additional practice, participants could also opt to fill in answers regarding a stepwise approach to electrocardiogram interpretation—rate, rhythm, axis, QTc, presence of hypertrophy, and presence of interval or ST and T wave abnormalities. However, only answering the multiple-choice question was required. An example question is shown in Figure 1. After submitting their answer, the webpage redirected participants to an explanation of key concepts demonstrated in that week’s electrocardiogram. Questions were not distributed in December or May, as these months were focused on the six- and twelve-month post-survey data collection. All the questions and explanations were collated on a website that participants could continue to access for review throughout the year. After 6 and 12 months of electrocardiogram distribution, a survey was sent out again asking residents to rate their confidence in the same four electrocardiogram interpretation skills evaluated in the pre-survey. To incentivize participation, for each vignette and pre/post survey, a participant was randomly selected to win a Starbucks gift card.
Figure 1. Example of clinical vignette with associated ECG and multiple-choice question. Questions 2–8 demonstrate the optional section of each question reviewing a systematic approach to ECG interpretation. ECG = electrocardiogram.
Surveys were developed and implemented using REDCap. REDCap was also used to collect data in a deidentified manner, assigning an anonymous user ID to link answers from the same participant together over time. Statistics were analysed using Stata. Analyses included descriptive statistics, regression analyses, and correlation matrices. Differences in pre- and post-confidence levels were analysed using the Mann–Whitney U test.
Results
There are 118 paediatrics and internal medicine/paediatrics residents at our centre. Of these, 60 residents participated in the pre-survey (50.8%), 42 in the 6-month follow-up (35.5%) and 37 in the 12-month follow-up (31.3%). There were 31 residents who completed both the pre-survey and the 6-month post-survey, and 32 who completed either the pre- or the 6-month survey and the 12-month post-survey. There were 16 residents who participated in the pre-survey but did not participate in the distributed practice questions. The training year of the participants and their intended careers following residency are displayed in Table 1.
Table 1. Participant demographics
Fourteen practice questions were administered over the course of 12 months. Seventy-seven residents participated in at least one distributed practice question (65.2%), and on average those who did participate completed 5.3 questions each. There was a subset of more engaged learners who participated all or most of the time, with five residents completing 100% of practice questions. Participants in the 6- and 12-month post-survey were significantly more likely to have been engaged in the curriculum, with average participation improving from 31 to 72% of practice questions in those who completed the 6-month survey and to 70% of practice questions in those who completed the 12-month survey.
After 6 months, there was a statistically significant increase in confidence in three of the four skills: ability to use a stepwise approach to interpret an electrocardiogram (p-value 0.007), ability to recognise an arrhythmia (p-value 0.048), and ability to know when urgent cardiology consultation was required (p-value 0.016) (Figure 2, Table 2). Between 6 and 12 months, there was no significant change in confidence in any of the four skills, indicating that although there was no further increase in confidence, there was retention of confidence over time (Figure 2, Table 2). The highest degree of improvement was seen in confidence in ability to utilise a stepwise approach to interpret an electrocardiogram, and the smallest degree of improvement was seen in confidence in ability to recognise an abnormality on an electrocardiogram.
Figure 2. Likert scale data showing participant confidence in ECG interpretation skills during the pre-survey and at the 6- and 12-month follow up. ECG = electrocardiogram.
Table 2. Confidence in ECG skills over time
P-values obtained using the Mann–Whitney test.
* p-value < 0.05.
** p-value < 0.01.
At 6 months, more than 80% of participants reported having received no other formal electrocardiogram education during that study period. Of note, the paediatric residency program offers a series of protected subspecialty educational half days throughout the year, called “Academic Half Day.” Most residents had the opportunity to participate in the paediatric cardiology academic half day as further augmentation to education. However, this one-time educational experience did not occur until late in the second half of this study, in April 2023.
In the pre-survey as well as the 6- and 12-month post-surveys, the strongest predictor of confidence in one skill was confidence in the other three skills. Year in training, intended career after residency, and increased participation in practice questions were not significantly correlated with improved confidence scores in the four skills (Table 3).
Table 3. Relationships between participant demographics and confidence in ECG skills
Data represented as a correlation matrix, in which 1 represents a strong relationship, 0 a neutral relationship, and -1 a weak relationship.
Although not the primary outcome of this study, information on whether residents answered the distributed practice questions correctly was also collected. On average, participants selected the correct answer 47% of the time. However, this number varied widely from resident to resident (11–100%) and from week to week (21–76%), and it did not necessarily trend towards higher average percent correct over time. There were no significant predictors of percent of correct answers, including year of training, degree of participation in the curriculum, or stated post-residency career interests.
Comment
In this single centre study of paediatrics and internal medicine/paediatrics residency programs, initial confidence in paediatric electrocardiogram interpretation skills was relatively low. Year in training did not predict a higher level of confidence in these skills at the start of this study. This suggests that confidence in electrocardiogram interpretation skills does not necessarily improve naturally over time throughout residency and emphasises the need for curricular intervention in this area. Although this was a single centre study, the national literature also reflects that trainees in many specialties, including paediatrics, underperform at electrocardiogram interpretation and would benefit from additional specific education. Reference Dickson, Chesser, Woods, Krug and Kellerman5–Reference Weinberg, Ottolini, Sestokas and Greene9
The implementation of distributed electrocardiogram practice in our centre did result in a small but statistically significant increase in confidence in electrocardiogram interpretation skills after 6 months of practice that was durable over the course of the following 6 months. Nearly all participating residents rated this curriculum as an effective and educational use of their time and, crucially, indicated that they had used skills from the curriculum in their clinical practice. Notably, the most significant improvement was seen in ability to use a stepwise approach to interpret an electrocardiogram. This is a foundational skill in electrocardiogram interpretation which, when applied consistently over time, may result in improvement in other electrocardiogram skills, especially understanding when an electrocardiogram is abnormal and when and how to consult paediatric cardiology. It is well known that distributive practice works better than massed practice for nearly every type of learning, from memorising simple lists to performing complex skills. Reference Gerbier and Toppino11–Reference Trumble, Lodge and Mandrusiak13 It is therefore to be expected that this kind of learning also works well for electrocardiogram interpretation. However, because the structure of a paediatric residency does not allow for distributive practice of electrocardiogram interpretation to happen naturally, it is important to seek out educational tools that can bridge this gap. This study highlights that intermittent email-based education can be an effective form of distributive practice for electrocardiogram interpretation, resulting in durable change that trainees are using in their day-to-day practice.
Although initially surprising to note that degree of participation in the practice questions did not correlate with improved confidence at 6 and 12 months, this result must be interpreted with caution. It was most likely mediated by the fact that those who participated in the 6- and 12-month surveys were overall significantly more engaged with the curriculum and on average completed significantly more practice questions than those who did not participate in the post-surveys. It was also notable that confidence, while durable between 6 and 12 months, did not further improve. This may be related to the fact that overall participation declined in the second half of the year. However, it may also be related to the limits of what degree of mastery can be expected to occur within the framework of general paediatric training. Further mastery and confidence in electrocardiogram skills may only be possible with additional subspecialty training. Complete mastery of electrocardiogram interpretation was well beyond the scope of this project, which instead aimed to introduce a better framework for electrocardiogram interpretation and to increase trainee exposure to and familiarity with paediatric electrocardiograms.
Although this study was not powered to specifically address the question of whether participation in the curriculum resulted in objective improvements in electrocardiogram skills, it is interesting to note that increased participation did not predict improved performance on distributive practice questions. There may be several reasons for this. Difficulty of questions varied somewhat from week to week, and the content covered throughout the year was broad. Practice questions often introduced new concepts or diseases that did not always build on topics covered in previous weeks. Therefore, improvement over time may not be expected to be entirely linear. This voluntary study was also vulnerable to participation bias, resulting in some difficult to interpret trends that likely contributed to this result. For example, some participants had a high percent correct ratio but participated quite sporadically, suggesting that they may have only submitted answers when they felt sure they knew the correct response. Others participated nearly every week whether they knew the answer or not, and many participated not at all or so rarely it is difficult to trend their performance over time.
This study had several other limitations. As discussed, it largely relied on self-reported measures of confidence in skills rather than objective measures of skill improvement. In addition, there was a relatively high attrition rate over time, with participation in the post-survey nearly half that of participation in the pre-survey. Participation in the distributive practice questions varied from week to week, and total participation in each question also trended down over time, from 37% of residents in the first half of the study to 25% of residents in the second half of the study. Residents commonly reported email fatigue and lack of time as reasons for choosing not to participate.
Based on these initial data, there are several planned improvements to the curriculum. We aim to adjust the questions so that they are more interwoven, such that the knowledge from one week builds and can be applied more concretely to subsequent weeks. To more objectively assess improvement in electrocardiogram reading ability, we also plan to administer a short test as part of the pre- and post-surveys asking residents to interpret several electrocardiograms. Finally, we plan to look at the average percent correct on cardiology questions on the in-service training exam for the residency program before and after the implementation of this curriculum as an objective measure of knowledge retention.
Overall, this distributive practice electrocardiogram curriculum was effective in achieving the primary outcome of improving participating residents’ confidence in their electrocardiogram interpretation skills over time. The curriculum was nearly unanimously rated as a valuable use of time, and importantly, residents indicated they were using skills learned from this curriculum in their clinical practice. It appears that email-based distributive practice is a valuable and effective tool for teaching complex concepts in residency medical education, especially when scheduling constraints and clinical demands make it difficult to regularly gather a consistent group of residents in person. Finally, this type of curriculum is beneficial for its ease of distribution, and while this particular curriculum was focused on electrocardiogram education, this format could easily be adapted to other types of subspecialty learning.
Acknowledgements
We would like to thank the paediatric and internal medicine/paediatrics programs at the University of Colorado for their enthusiasm and participation in this project.
