Skip to main content Accessibility help
×
Home

When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory

  • Emily E. Haroz (a1), Jeremy C. Kane (a2), Amanda J. Nguyen (a3), Judith K. Bass (a4) (a5), Laura K. Murray (a4) and Paul Bolton (a1) (a4) (a5)...

Abstract

Background

There is a need for accurate and efficient assessment tools that cover a range of mental health and psychosocial problems. Existing, lengthy self-report assessments may reduce accuracy due to respondent fatigue. Using data from a sample of adults enrolled in a psychotherapy randomized trial in Thailand and a cross-sectional sample of adolescents in Zambia, we leveraged Item Response Theory (IRT) methods to create brief, psychometrically sound, mental health measures.

Methods

We used graded-response models to refine scales by identifying and removing poor performing items that were not well correlated with the underlying trait, and by identifying well-performing items at varying levels of a latent trait to assist in screening or monitoring purposes.

Results

In Thailand, the original 17-item depression scale was shortened to seven items and the 30-item Posttraumatic Stress Scale (PTS) was shortened to 10. In Zambia, the Child Posttraumatic Stress Scale (CPSS) was shortened from 17 items to six. Shortened scales in both settings retained the strength of their psychometric properties. When examining longitudinal intervention effects in Thailand, effect sizes were comparable in magnitude for the shortened and standard versions.

Conclusions

Using Item Response Theory (IRT) we created shortened valid measures that can be used to help guide clinical decisions and function as longitudinal research tools. The results of this analysis demonstrate the reliability and validity of shortened scales in each of the two settings and an approach that can be generalized more broadly to help improve screening, monitoring, and evaluation of mental health and psychosocial programs globally.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

Author for correspondence: Emily E. Haroz, E-mail: eharoz1@jhu.edu

References

Hide All
Achenbach, T (1991). Manual for the Youth Self-Report and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry.
American Psychiatric Association, DSM-5 Task Force (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5), 5th Edn. Arlington, VA: American Psychiatric Publishing.
Arroll, B, Goodyear-Smith, F, Crengle, S, Gunn, J, Kerse, N, Fishman, T, Falloon, K and Hatcher, S (2010). Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Annals of Family Medicine 8, 348353.
Baker, F and Kim, S (2004). Item Response Theory: Parameter Estimation Techniques, 2nd Edn. Boca Raton: CRC Press.
Barlow, DH and Farchione, TJ (2017). Applications of the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders. Oxford: Oxford University Press.
Bolton, P, Lee, C, Haroz, EE, Murray, L, Dorsey, S, Robinson, C, Ugueto, AM and Bass, J (2014). A transdiagnostic community-based mental health treatment for comorbid disorders: development and outcomes of a randomized controlled trial among Burmese refugees in Thailand. PLoS Medicine 11, e1001757. doi: 10.1371/journal.pmed.1001757.
Bolton, P and Tang, AM (2002). An alternative approach to cross-cultural function assessment. Social Psychiatry and Psychiatric Epidemiology 37(11), 537543.
Chorpita, BF, Daleiden, EL and Weisz, JR (2005). Identifying and selecting the common elements of evidence based interventions: a distillation and matching model. Mental Health Services Research 7, 520.
De Vet, H, Terwee, C, Mokkink, L and Knol, D (2011). Measurement in Medicine: A Practical Guide. Cambridge, MA: Cambridge University Press.
Diehr, P, Chen, L, Patrick, D, Feng, Z and Yasui, Y (2005). Reliability, effect size, and responsiveness of health status measures in the design of randomized and cluster-randomized trials. Contemporary Clinical Trials 26, 4558.
Doty, SB, Haroz, EE, Singh, NS, Bogdanov, S, Bass, JK, Murray, LK, Callaway, KL and Bolton, PA (2018). Adaptation and testing of an assessment for mental health and alcohol use problems among conflict-affected adults in Ukraine. Conflict and Health 12, 34.
Edelen, MO and Reeve, BB (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research 16(S1), 518.
Eisen, SV, Dickey, B and Sederer, LI (2000). A self-report symptom and problem rating scale to increase inpatients’ involvement in treatment. Psychiatric Services 51, 349353.
Embretson, SE (1996). The new rules of measurement. Psychological Assessment 8, 341349.
Farchione, TJ and Bullis, JR (2014). Addressing the global burden of mental illness: why transdiagnostic and common elements approaches to evidence-based practice might be our best bet. Cognitive and Behavioral Practice 21, 124126.
Foa, EB, Johnson, KM, Feeny, NC and Treadwell, KR (2001). The child PTSD symptom scale: a preliminary examination of its psychometric properties. Journal of Clinical Child Psychology 30, 376384.
Gutner, CA, Galovski, T, Bovin, MJ and Schnurr, PP (2016). Emergence of transdiagnostic treatments for PTSD and posttraumatic distress. Current Psychiatry Reports 18(10), 95.
Haroz, EE, Bass, JK, Lee, C, Murray, LK, Robinson, C and Bolton, P (2014). Adaptation and testing of psychosocial assessment instruments for cross-cultural use: an example from the Thailand Burma border. BMC Psychology 2, 31.
Haroz, EE, Bolton, P, Gross, A, Chan, KS, Michalopoulos, L and Bass, J (2016). Depression symptoms across cultures: an IRT analysis of standard depression symptoms using data from eight countries. Social Psychiatry and Psychiatric Epidemiology 51, 981991.
Haroz, EE, Ritchey, M, Bass, JK, Kohrt, BA, Augstinavicius, J, Michalopoulos, L, Burkey, MD and Bolton, P (2017). How is depression experienced around the world? A systematic review of qualitative literature. Social Science & Medicine 183, 151162.
Hays, RD, Morales, LS and Reise, SP (2000). Item response theory and health outcomes measurement in the 21st century. Medical Care 38(9 Suppl), II28II42.
Jordans, MJD, Pigott, H and Tol, WA (2016). Interventions for children affected by armed conflict: a systematic review of mental health and psychosocial support in low- and middle-income countries. Current Psychiatry Reports 18, 9.
Kane, JC, Bolton, PA, Murray, SM, Bass, JK, Lakin, D, Whetten, K, Skavenski, S and Murray, LK (2018). Psychometric evaluation of HIV risk behavior assessments using Audio Computer Assisted Self-Interviewing (ACASI) among orphans and vulnerable children in Zambia. AIDS Care 30, 160167.
Kane, JC, Murray, LK, Sughrue, S, DeMulder, J, Skavenski, S, Queenan, U, Tang, A and Bolton, P (2016). Process and implementation of Audio Computer Assisted Self-Interviewing (ACASI) assessments in low resource settings: a case example from Zambia. Global Mental Health 3, e24.
Kwan, B and Rickwood, DJ (2015). A systematic review of mental health outcome measures for young people aged 12 to 25 years. BMC Psychiatry 15, 279.
Lambert, MJ, Whipple, JL, Hawkins, EJ, Vermeersch, DA, Nielsen, SL and Smart, DW (2006). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice 10, 288301.
Lincare, JM (1994). Sample size and item calibration stability. Rasch Measurement Transactions 7, 328.
Marshall, GN and Edelen, MO (2002). Differential item functioning in a Spanish translation of the PTSD checklist. Psychological Assessment2 14, 5059.
Mollica, R, McDonald, L, Massagli, M and Silove, D (2004). Measuring Trauma, Measuring Torture. Cambridge, MA: Cambridge University Press.
Morris, DW and Trivedi, MH (2011). Measurement-based care for unipolar depression. Current Psychiatry Reports 13, 446458.
Murray, SM, Bolton, PA, Kane, JC, Lakin, D, Bass, JK and Murray, LK (2018 b). Measuring symptoms of psychopathology in Zambian orphans and vulnerable children: scale validation and psychometric evaluation. Assessment. epub. doi: 10.1177/1073191118780455.
Murray, LK, Dorsey, S, Haroz, E, Lee, C, Alsiary, MM, Haydary, A, Weiss, WM and Bolton, P (2014). A common elements treatment approach for adult mental health problems in low- and middle-income countries. Cognitive and Behavioral Practice 21, 111123.
Murray, LK, Haroz, EE, Doty, SB, Singh, NS, Bogdanov, S, Bass, J, Dorsey, S and Bolton, P (2018 a). Testing the effectiveness and implementation of a brief version of the Common Elements Treatment Approach (CETA) in Ukraine: a study protocol for a randomized controlled trial. Trials 19, 418.
Murray, LK and Jordans, MJD (2016). Rethinking the service delivery system of psychological interventions in low and middle income countries. BMC Psychiatry 16, 234.
Murray, LK, Skavenski, S, Kane, JC, Mayeya, J, Dorsey, S, Cohen, JA, Michalopoulos, LT, Imasiku, M and Bolton, PA (2015). Effectiveness of trauma-focused cognitive behavioral therapy among trauma-affected children in Lusaka, Zambia: a randomized clinical trial. JAMA Pediatrics 169, 761769.
Newby, JM, McKinnon, A, Kuyken, W, Gilbody, S and Dalgleish, T (2015). Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clinical Psychology Review 40, 91110.
Rathod, S, Pinninti, N, Irfan, M, Gorczynski, P, Rathod, P, Gega, L and Naeem, F (2017). Mental health service provision in low- and middle-income countries. Health Services Insights 10, 17. 1178632917694350.
Samejima, F (2016). Graded response models. In van der Linden, Wim J. and Hambleton, Ronald K. (eds), Handbook of Item Response Theory, pp. 123136. Springer-Verlag New York Inc.New York, NY.
Sauer-Zavala, S, Cassiello-Robbins, C, Conklin, LR, Bullis, JR, Thompson-Hollands, J and Kennedy, KA (2017). Isolating the unique effects of the unified protocol treatment modules using single case experimental design. Behavior Modification 41, 286307.
Scott, K and Lewis, CC (2015). Using measurement-based care to enhance any treatment. Cognitive and Behavioral Practice 22, 4959.
Seidman, G and Atun, R (2017). Does task shifting yield cost savings and improve efficiency for health systems? A systematic review of evidence from low-income and middle-income countries. Human Resources for Health 15, 29.
Singla, DR, Kohrt, BA, Murray, LK, Anand, A, Chorpita, BF and Patel, V (2017). Psychological treatments for the world: lessons from low- and middle-income countries. Annual Review of Clinical Psychology 13, 149181.
Smits, N, Cuijpers, P, Beekman, ATF and Smit, JH (2007). Reducing the length of mental health instruments through structurally incomplete designs. International Journal of Methods in Psychiatric Research 16, 150160.
Smits, N and Finkelman, MD (2014). Variable length testing using the ordinal regression model. Statistics in Medicine 33, 488499.
van Ginneken, N, Tharyan, P, Lewin, S, Rao, GN, Meera, SM, Pian, J, Chandrashekar, S and Patel, V (2013). Non-specialist health worker interventions for the care of mental, neurological and substance-abuse disorders in low- and middle-income countries. The Cochrane Database of Systematic Reviews 11, CD009149.
Weiss, WM, Murray, LK, Zangana, GAS, Mahmooth, Z, Kaysen, D, Dorsey, S, Lindgren, K, Gross, A, Murray, SM, Bass, JK and Bolton, P (2015). Community-based mental health treatments for survivors of torture and militant attacks in Southern Iraq: a randomized control trial. BMC Psychiatry 15, 249.
World Health Organization (2011). Mental Health Atlas 2011. World Health Organization, Geneva. Retrieved from http://whqlibdoc.who.int/publications/2011/9799241564359_eng.pdf?ua=1.
World Health Organization (2017). Depression and Other Common Mental Disorders: Global Health Estimates. World Health Organization, Geneva. Retrieved from https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf?sequence=1.
Xia, J, Tang, Z, Wu, P, Wang, J and Yu, J (2019). Use of item response theory to develop a shortened version of the EORTC QLQ-BR23 scales. Scientific Reports 9, 1764.
Yatham, S, Sivathasan, S, Yoon, R, da Silva, TL and Ravindran, AV (2017). Depression, anxiety, and post-traumatic stress disorder among youth in low and middle income countries: a review of prevalence and treatment interventions. Asian Journal of Psychiatry 38, 7891.

Keywords

Type Description Title
WORD
Supplementary materials

Haroz et al. supplementary materials
Haroz et al. supplementary materials

 Word (24 KB)
24 KB

When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory

  • Emily E. Haroz (a1), Jeremy C. Kane (a2), Amanda J. Nguyen (a3), Judith K. Bass (a4) (a5), Laura K. Murray (a4) and Paul Bolton (a1) (a4) (a5)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.