Hostname: page-component-848d4c4894-wg55d Total loading time: 0 Render date: 2024-05-21T04:05:56.377Z Has data issue: false hasContentIssue false
  • English
  • Français

Development of a model and method for hospice quality assessment from natural language processing (NLP) analysis of online caregiver reviews

Published online by Cambridge University Press:  14 July 2023

Jason T. Hotchkiss Open the ORCID record for Jason T. Hotchkiss [Opens in a new window] ,
Emily Ridderman  and
William Bufkin
Jason T. Hotchkiss*
Affiliation:
Cornerstone University, Grand Rapids, MI, USA
Emily Ridderman
Affiliation:
Cornerstone University, Grand Rapids, MI, USA
William Bufkin
Affiliation:
Cornerstone University, Grand Rapids, MI, USA
*
Corresponding author: Jason T. Hotchkiss; Email: jason.hotchkiss@cornerstone.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives

With a fraction of hospices having their Consumer Assessment of Healthcare Providers and Systems (CAHPS®) scores on Hospice Compare, a significant reservoir of hospice quality data remains in online caregiver reviews. The purpose of this study was to develop a method and model of hospice quality assessment from caregiver reviews using Watson’s carative model.

Methods

Retrospective mixed methods of pilot qualitative thematic analysis and sentiment analysis using NLP of Google and Yelp caregiver reviews between 2013 and 2023. We employed stratified sampling, weighted according to hospice size, to emulate the daily census of enrollees across the United States. Sentiment analysis was performed (n = 3393) using Google NLP.

Results

Two themes with the highest prevalence had moderately positive sentiments (S): Caring staff (+.47) and Care quality, comfort and cleanliness (+.41). Other positive sentiment scores with high prevalence were Gratitude and thanks (+.81), “Treating the patient with respect” (+.54), and “Emotional, spiritual, bereavement support” (+.60). Lowest sentiment scores were “Insurance, administrative or billing” (–.37), “Lack of staffing” (–.32), and “Communication with the family” (–.01).

Significance of results

In the developed quality model, caregivers recommended hospices with caring staff, providing quality care, responsive to requests, and offering family support, including bereavement care. All ten Watson's carative factors and all eight CAHPS measures were presented in the discovered review themes of the quality model. Close-ended CAHPS scores and open-ended online reviews have substantial conceptual overlap and complementary insights. Future hospice quality research should explore caregiver expectations and compare review themes by profit status.

Keywords

Online caregiver reviewsHospice caregiversGoogle Cloud NLPHospice qualityCAHPS® scoresConsumer Assessment of Healthcare Providers and Systems
Type
Original Article
Information
Palliative & Supportive Care , Volume 22 , Issue 1 , February 2024 , pp. 19 - 30
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Barry, CL, Carlson, MDA, Thompson, JW, et al. (2012) Caring for grieving family members: Results from a national hospice survey. Medical Care 50(7), 578–584. doi:10.1097/MLR.0b013e318248661dCrossRefGoogle ScholarPubMed
Blei, D, Ng, A and Jordan, M (2013) Latent Dirichlet allocation. Journal of Machine Learning Research 3, . doi:10.1016/B978-0-12-411519-4.00006-9Google Scholar
Brereton, EJ, Matlock, DD, Fitzgerald, M, et al. (2020) Content Analysis of Negative Online Reviews of Hospice Agencies in the United States. JAMA network open 3(2), . doi:10.1001/jamanetworkopen.2019.21130CrossRefGoogle ScholarPubMed
Cambria, E and Hussain, A (2012) Applications of sentiment analysis. In Hussain, VCA, Chang, EY and Meethan, J (eds.), Sentiment Analysis and Ontology Engineering. New York: Springer, 1–18.Google Scholar
Carlson, MDA, Gallo, WT and Bradley, EH (2004) Ownership status and patterns of care in hospice results from the national home and hospice care survey. Medical Care 42(5), 432–438. doi:10.1097/01.mlr.0000124246.86156.54CrossRefGoogle ScholarPubMed
CMS (2021) Hospice Quality Reporting Program. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Hospice-Quality-Reporting (accessed 11 February 2022).Google Scholar
CMS (2022) CMS Hospice care data archive. https://data.cms.gov/provider-data/archived-data/hospice-care (accessed 14 February 2022).Google Scholar
Hotchkiss, JT (2022) Hospice Glassdoor and CAHPS® scores—Glassdoor scores and hospice financial characteristics predict hospice consumer assessment of healthcare providers and systems scores. American Journal of Hospice and Palliative Medicine 40(3), 311–321. doi:10.1177/10499091221099475CrossRefGoogle ScholarPubMed
Hotchkiss, JT (2023) Overall US hospice quality according to decedent caregivers—Natural Language Processing and sentiment analysis of 3389 online caregiver reviews. American Journal of Hospice and Palliative Medicine 45(4), 1–18.Google Scholar
Johari, K, Kellogg, C, Vazquez, K, et al. (2018) Ratings game: An analysis of Nursing Home Compare and Yelp ratings. BMJ Quality & Safety 27(8), . doi:10.1136/bmjqs-2017-007301CrossRefGoogle ScholarPubMed
Liu, B (2012) Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies 5(1), 1–167. doi:10.2200/S00416ED1V01Y201204HLT016CrossRefGoogle Scholar
Liu, B (2015) Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Nasukawa, T and Yi, J (2003). Sentiment analysis: Capturing favorability using natural language processing.CrossRefGoogle Scholar
Pang, B and Lee, L (2008) Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval 2(1–2), 1–135. doi:10.1561/1500000011CrossRefGoogle Scholar
Parast, L, Elliott, MN, Hambarsoomian, K, et al. (2018a) Effects of survey mode on consumer assessment of healthcare providers and systems (CAHPS) Hospice Survey Scores. Journal of the American Geriatrics Society (JAGS) 66(3), 546–552. doi:10.1111/jgs.15265CrossRefGoogle ScholarPubMed
Parast, L, Haas, A, Tolpadi, A, et al. (2018b) Effects of caregiver and decedent characteristics on CAHPS hospice survey scores. Journal of Pain and Symptom Management 56(4), . doi:10.1016/j.jpainsymman.2018.07.014CrossRefGoogle ScholarPubMed
Perry, JE and Stone, RC (2011) In the business of dying: Questioning the commercialization of hospice. The Journal of Law, Medicine & Ethics 39(2), 224–234. doi:10.1111/j.1748-720X.2011.00591.xCrossRefGoogle ScholarPubMed
Rahman, A, Cardenas, V, Singleton, M, et al. (2021) What consumers say about hospices in online reviews. Journal of Palliative Medicine 24(2), . doi:10.1089/jpm.2019.0591CrossRefGoogle ScholarPubMed
Ranard, BL, Werner, RM, Antanavicius, T, et al. (2016) Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care. Health Affairs Web Exclusive 35(4), 697–705. doi:10.1377/hlthaff.2015.1030CrossRefGoogle ScholarPubMed
Raths, D (2016) The Yelp approach to healthcare quality improvement. https://www.hcinnovationgroup.com/home/blog/13026854/the-yelp-approach-to-healthcare-quality-improvement (accessed 18 April 2022).Google Scholar
Schapira, MM, Shea, JA, Duey, KA, et al. (2016) The Nursing Home Compare report card: Perceptions of residents and caregivers regarding quality ratings and nursing home choice. Health Services Research 51(S2), 1212–1228. doi:10.1111/1475-6773.12458CrossRefGoogle ScholarPubMed
Schwartz, HA, Eichstaedt, JC, Kern, ML, et al. (2013) Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS One 8(9), e73791–e73791. doi:10.1371/journal.pone.0073791CrossRefGoogle ScholarPubMed
Schwartz, HA and Ungar, LH (2015) Data-driven content analysis of social media: A systematic overview of automated methods. The Annals of the American Academy of Political and Social Science 659(1), 78–94. doi:10.1177/0002716215569197CrossRefGoogle Scholar
Scotty (2018) Yelp’s novel approach to healthcare quality improvement – patients review doctors. https://www.pyzdekinstitute.com/blog/healthcare-quality/yelps-novel-approach-to-healthcare-quality-improvement-patients-review-doctors.html (accessed 18 April 2022).Google Scholar
Shabbir, S (2021) LexisNexis Risk Solutions Announces Top U.S. Hospice, Home Health Providers. https://risk.lexisnexis.com/about-us/press-room/press-release/20210209-hom-health-hospice.Google Scholar
Sitzman, K and Watson, J (2014) Caring Science, Mindful Practice: Implementing Watson’s Human Caring Theory. New York: Springer.CrossRefGoogle Scholar
Wang, X, Beltran, S, Gammonley, D, et al. (2021) Examining the relationship between hospice agency characteristics and complaint deficiencies. Innovation in Aging 5(1), 1012–1012. doi:10.1093/geroni/igab046.3593CrossRefGoogle Scholar
Watson, J (1979) Nursing: The Philosophy and Science of Caring. Norwalk, Connecticut: Appleton & Lange.Google Scholar
Watson, J (1988) Nursing: Human Science and Human Care. A Theory of Nursing. New York: NLN publications (15-2236), .Google Scholar
Watson, J (2008) Nursing: The Philosophy and Science of Caring, Revised Edition. Boulder, CO: University Press of Colorado.Google Scholar
Watson, J (2021) Revisiting ‘discipline’ in relation to caring science as sacred science: Revisiting discipline of nursing. Journal of Holistic Nursing 40(3), 58–63. doi:10.1177/08980101211041187CrossRefGoogle ScholarPubMed
Yelp (2022) Yelp: Does Yelp recommend every review? https://www.yelp-support.com/article/Does-Yelp-recommend-every-review?l=en_US (accessed 18 April 2022).Google Scholar