Skip to main content Accessibility help
×
Home

A summative, Objective, Structured, Clinical Examination in ENT used to assess postgraduate doctors after one year of ENT training, as part of the Diploma of Otorhinolaryngology, Head and Neck Surgery

  • A B Drake-Lee (a1), D Skinner (a2), M Hawthorne (a3), R Clarke (a4) and the Examiners for the Diploma of Otorhinolaryngology, Head and Neck Surgery...

Abstract

Context:

‘High stakes’ postgraduate medical examinations should conform to current educational standards. In the UK and Ireland, national assessments in surgery are devised and managed through the examination structure of the Royal Colleges of Surgeons. Their efforts are not reported in the medical education literature. In the current paper, we aim to clarify this process.

Objectives:

To replace the clinical section of the Diploma of Otorhinolaryngology with an Objective, Structured, Clinical Examination, and to set the level of the assessment at one year of postgraduate training in the specialty.

Methods:

After ‘blueprinting’ against the whole curriculum, an Objective, Structured, Clinical Examination comprising 25 stations was divided into six clinical stations and 19 other stations exploring written case histories, instruments, test results, written communication skills and interpretation skills. The pass mark was set using a modified borderline method and other methods, and statistical analysis of the results was performed.

Results:

The results of nine examinations between May 2004 and May 2008 are presented. The pass mark varied between 68 and 82 per cent. Internal consistency was good, with a Cronbach's α value of 0.99 for all examinations and split-half statistics varying from 0.96 to 0.99. Different standard settings gave similar pass marks.

Conclusions:

We have developed a summative, Objective, Structured, Clinical Examination for doctors training in otorhinolaryngology, reported herein. The objectives and standards of setting a high quality assessment were met.

Copyright

Corresponding author

Address for correspondence: Mr Adrian Drake-Lee, Queen Elizabeth Hospital, Edgbaston, Birmingham B15 2TH, UK. E-mail: Adrakelee@aol.com

References

Hide All
1PMETB principles of an assessment system for postgraduate medical training, 2004. In: http://www.pmetb.org.uk/fileadmin/user/Standards_Requirements/PM_ETB_Scas_July2008_Final.pdf [23 June 2009]
2Calman, K. Hospital Doctors: Training for the Future: the Report of the Working Group on Specialist Medical Training. Heywood: Department of Health, 1993
3 Diploma in Otolaryngology – Head and Neck Surgery: Syllabus. In: http://www.intercollegiatemrcs.org.uk/dohns/syllabus_html [23 June 2009]
4Harden, R, Gleeson, F. Assessment of clinical competence using an Objective Structured Clinical Examination (OSCE). Med Educ 1979;13:1134–40
5Harden, R, Stevenson, M, Downie, W, Wilson, G. Assessment of clinical competence using objective structured clinical examination. BMJ 1975;1:447–51
6Schwartz, RW, Witzke, DB, Donnelly, MB, Stratton, T, Blue, AV, Sloan, DA. Assessing residents' clinical performance: cumulative results of a four-year study with the Objective Structured Clinical Examination. Surgery 1998;124:307–12
7Cohen, R, Reznick, RK, Taylor, B, Provan, J, Rothman, A. Reliability and validity of the objective structured clinical examination in assessing surgical residents. Am J Surg 1990;160:302–5
8MacRae, H, Regehr, G, Leadbetter, W, Reznick, R. A comprehensive examination for senior surgical resident. Am J Surg 2000;179:190–3
9Patil, NG, Saing, H, Wong, J. Role of OSCE in evaluation of practical skills. Medical Teacher 2003;25:271–2
10Brailovsky, C, Grand'Mason, P, Lescop, J. A large-scale multicentre objective structured clinical examination for licensure. Acad Med 1992;67:S37–9
11Hutchinson, L, Aitken, P, Hayes, T. Are medical postgraduate certification processes valid? A systematic review of the published evidence. Med Educ 2002;36:7391
12Drake-Lee, A, Skinner, D, Reid, A. An OSCE as part of the first-year SpR assessment in ENT. Ann R Coll Surg 2006;88(suppl):310–12
13Ebel, RL, Frisbie, DA. Essentials of Educational Measurement, 5 edn.Englewood Cliffs: Prentice Hall, 1991;370
15Frederiksen, J, Collins, A. A systems approach to educational testing. Educational Researcher 1989;18:2732
16Boulet, J, Champlain, A, McKinley, D. Setting defensible performance standards in OSCEs and standarized patient examinations. Medical Teacher 2003;25:245–9
17Norcini, JJ. Setting standards on educational tests. Med Educ 2003;37:464–9
18Kramer, A, Muijtjens, A, Jansen, K, Dusman, H, Tan, L, van der Vleuten, M. Comparison of a rational and empirical standard setting procedure for an OSCE. Med Educ 2003;37:132–9
19Davis, MH. OSCE: the Dundee experience. Medical Teacher 2003;25:255–61
20Hodges, B. Validity and the OSCE. Medical Teacher 2003;25:250–4
21Adamo, G. Simulated and standardized patients in OSCEs: achievements and challenges 1992–2003. Medical Teacher 2003;25:262–70
22Patil, NG, Cheng, SW, Wong, J. Surgical competence. World J Surg 2003;27:943–7
23Wilkinson, TJ, Newble, DI, Frampton, CM. Standard setting in an objective structured clinical examination: use of global ratings of borderline performance to determine the passing score. Med Educ 2001;35:1043–9
24Smee, S, Blackmore, D. Setting standards for an objective structured clinical examination: the borderline method gains ground on Angoff. Med Educ 2001;35:1109–1010
25Humphrey-Murto, S, MacFadyen, J. Standard setting: a comparison of case-author and modified borderline-group methods in a small scale OSCE. Acad Med 2002;77:729–32
26McKinlay, D, Boulet, J, Hambleton, R. A work-centered approach for setting passing scores on performance-based assessments. Evaluation and the Health Professions 2005;28:349–69

Keywords

A summative, Objective, Structured, Clinical Examination in ENT used to assess postgraduate doctors after one year of ENT training, as part of the Diploma of Otorhinolaryngology, Head and Neck Surgery

  • A B Drake-Lee (a1), D Skinner (a2), M Hawthorne (a3), R Clarke (a4) and the Examiners for the Diploma of Otorhinolaryngology, Head and Neck Surgery...

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed