Skip to main content Accessibility help
×
Home
Hostname: page-component-5959bf8d4d-4p99k Total loading time: 0.355 Render date: 2022-12-09T10:25:55.617Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study

Published online by Cambridge University Press:  20 April 2022

Emilia H. Fisher*
Affiliation:
Department of Emergency Medicine, Children’s Minnesota, Minneapolis, MinnesotaUSA
Ilene Claudius
Affiliation:
Harbor-UCLA Medical Center, Emergency Medicine, Torrance, CaliforniaUSA David Geffen School of Medicine at UCLA, Los Angeles, CaliforniaUSA The Lundquist Institute at Harbor-UCLA, Los Angeles, CaliforniaUSA
Amy H. Kaji
Affiliation:
Harbor-UCLA Medical Center, Emergency Medicine, Torrance, CaliforniaUSA David Geffen School of Medicine at UCLA, Los Angeles, CaliforniaUSA The Lundquist Institute at Harbor-UCLA, Los Angeles, CaliforniaUSA
Alaa Shaban
Affiliation:
Community Health Centers, Pediatrics, Santa Maria, CaliforniaUSA
Nicholas McGlynn
Affiliation:
Keck School of Medicine of University of Southern California, Los Angeles, CaliforniaUSA
Mark X. Cicero
Affiliation:
Yale University School of Medicine, New Haven, ConnecticutUSA
Genevieve Santillanes
Affiliation:
Department of Emergency Medicine, Keck School of Medicine of University of Southern California, Los Angeles, CaliforniaUSA
Marianne Gausche-Hill
Affiliation:
Harbor-UCLA Medical Center, Emergency Medicine, Torrance, CaliforniaUSA David Geffen School of Medicine at UCLA, Los Angeles, CaliforniaUSA The Lundquist Institute at Harbor-UCLA, Los Angeles, CaliforniaUSA Los Angeles County Emergency Medical Services Agency, Santa Fe Springs, CaliforniaUSA
Todd P. Chang
Affiliation:
Children’s Hospital Los Angeles, Los Angeles, CaliforniaUSA
J. Joelle Donofrio-Odmann
Affiliation:
University of California San Diego Departments of Pediatrics and Emergency Medicine, San Diego, CaliforniaUSA Rady Children’s Hospital of San Diego, San Diego, CaliforniaUSA
*
Correspondence: Emilia H. Fisher Children’s Minnesota - Department of Emergency Medicine 2525 Chicago Ave South Mailstop 32-1488 Minneapolis, Minnesota55404USA E-mail: Emilia.fisher@childrensmn.org

Abstract

Introduction:

Many triage algorithms exist for use in mass-casualty incidents (MCIs) involving pediatric patients. Most of these algorithms have not been validated for reliability across users.

Study Objective:

Investigators sought to compare inter-rater reliability (IRR) and agreement among five MCI algorithms used in the pediatric population.

Methods:

A dataset of 253 pediatric (<14 years of age) trauma activations from a Level I trauma center was used to obtain prehospital information and demographics. Three raters were trained on five MCI triage algorithms: Simple Triage and Rapid Treatment (START) and JumpSTART, as appropriate for age (combined as J-START); Sort Assess Life-Saving Intervention Treatment (SALT); Pediatric Triage Tape (PTT); CareFlight (CF); and Sacco Triage Method (STM). Patient outcomes were collected but not available to raters. Each rater triaged the full set of patients into Green, Yellow, Red, or Black categories with each of the five MCI algorithms. The IRR was reported as weighted kappa scores with 95% confidence intervals (CI). Descriptive statistics were used to describe inter-rater and inter-MCI algorithm agreement.

Results:

Of the 253 patients, 247 had complete triage assignments among the five algorithms and were included in the study. The IRR was excellent for a majority of the algorithms; however, J-START and CF had the highest reliability with a kappa 0.94 or higher (0.9-1.0, 95% CI for overall weighted kappa). The greatest variability was in SALT among Green and Yellow patients. Overall, J-START and CF had the highest inter-rater and inter-MCI algorithm agreements.

Conclusion:

The IRR was excellent for a majority of the algorithms. The SALT algorithm, which contains subjective components, had the lowest IRR when applied to this dataset of pediatric trauma patients. Both J-START and CF demonstrated the best overall reliability and agreement.

Type
Original Research
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of the World Association for Disaster and Emergency Medicine

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Koziel, JR, Meckler, G, Brown, L, et al. Barriers to pediatric disaster triage: a qualitative investigation. Prehosp Emerg Care. 2014;19(2):279286.10.3109/10903127.2014.967428CrossRefGoogle ScholarPubMed
Wallis, LA, Carley, S. Comparison of pediatric major incident primary triage tools. Emerg Med J. 2006;23(6):475478.10.1136/emj.2005.032672CrossRefGoogle ScholarPubMed
Cross, KP, Cicero, MX. Head-to-head comparison of disaster triage methods in pediatric, adult, and geriatric patients. YMEM. 2013;61(6):668676.Google ScholarPubMed
Heffernan, R, Lerner, EB, McKee, C, et al. Comparing the accuracy of mass casualty triage systems in a pediatric population. Prehosp Emerg Care. 2019;23(3):304308.10.1080/10903127.2018.1520946CrossRefGoogle Scholar
Cicero, MX, Overly, F, Brown, L, et al. Comparing the accuracy of three pediatric disaster triage strategies: a simulation-based investigation. Disaster Med Public Health Prep. 2016;10(2):253260.10.1017/dmp.2015.171CrossRefGoogle ScholarPubMed
Jenkins, JL, McCarthy, ML, Sauer, LM, et al. Mass-casualty triage: time for an evidence-based approach. Prehosp Disaster Med. 2008;23(1):38.10.1017/S1049023X00005471CrossRefGoogle ScholarPubMed
Nadeau, NL, Cicero, MX. Pediatric disaster triage system utilization across the United States. Pediatr Emerg Care. 2017;33(3):152155.10.1097/PEC.0000000000000680CrossRefGoogle ScholarPubMed
DeGeorge, L, Glick, R, Ackerman, A, Wu, Y. Pediatric triage decision making in a large-scale pediatric disaster. Ann Emerg Med. 2015;66(4):S143S144.10.1016/j.annemergmed.2015.07.433CrossRefGoogle Scholar
Cicero, MX, Riera, A, Northrup, V, Auerbach, M, Pearson, K, Baum, CR. Design, validity, and reliability of a pediatric resident JumpSTART disaster triage scoring instrument. Acad Pediatr. 2013;13(1):4854.10.1016/j.acap.2012.09.002CrossRefGoogle ScholarPubMed
Lerner, EB, Schwartz, RB, Coule, PL, et al. Mass casualty triage: an evaluation of the data and development of a proposed national guideline. Disaster Med Pub Health Prep. 2008;2 Suppl 1;S25S34.10.1097/DMP.0b013e318182194eCrossRefGoogle ScholarPubMed
Nocera, A, Garner, A. An Australian mass casualty incident triage system for the future based upon triage mistakes of the past: the Homebush Triage Standard. Aust NZ J Surg. 1999;69:603608.10.1046/j.1440-1622.1999.01644.xCrossRefGoogle ScholarPubMed
Lerner, EB, Cone, DC, Weinstein, ES, et al. Mass casualty triage: an evaluation of the science and refinement of a national guideline. Disaster Med Public Health Prep. 2011;5(2):129137.10.1001/dmp.2011.39CrossRefGoogle ScholarPubMed
Sacco, WJ, Navin, DM, Fiedler, KE, Waddell RK II, Long WB, Buckman RF. Precise formulation and evidence-based application of resource-constrained triage. Acad Emerg Med. 2005;12(8):759770.10.1197/j.aem.2005.04.003CrossRefGoogle Scholar

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Inter-Rater Reliability and Agreement Among Mass-Casualty Incident Algorithms Using a Pediatric Trauma Dataset: A Pilot Study
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *