Skip to main content Accessibility help

Validation of a tool to assess patient satisfaction, waiting times, healthcare utilization, and cost

  • Breda H. Eubank (a1), Mark R. Lafave (a1), Nicholas G. Mohtadi (a2), David M. Sheps (a3) and J. Preston Wiley (a4)...



Patients’ experience of the quality of care received throughout their continuum of care can be used to direct quality improvement efforts in areas where they are most needed. This study aims to establish validity and reliability of the Healthcare Access and Patient Satisfaction Questionnaire (HAPSQ) – a tool that collects patients’ experience that quantifies aspect of care used to make judgments about quality from the perspective of the Alberta Quality Matrix for Health (AQMH).


The AQMH is a framework that can be used to assess and compare the quality of care in different healthcare settings. The AQMH provides a common language, understanding, and approach to assessing quality. The HAPSQ is one tool that is able to assess quality of care according to five of six AQMH’s dimensions.


This was a prospective methodologic study. Between March and October 2015, a convenience sample of patients presenting with chronic full-thickness rotator cuff tears was recruited prospectively from the University of Calgary Sport Medicine Centre in Calgary, Alberta, Canada. Reliability of the HAPSQ was assessed using test–retest reliability [interclass correlation coefficient (ICC)>0.70]. Validity was assessed through content validity (patient interviews, floor and ceiling effects), criterion validity (percent agreement >70%), and construct validity (hypothesis testing).


Reliability testing was completed on 70 patients; validity testing occurred on 96 patients. The mean duration of symptoms was three years (SD: 5.0, range: 0.1–29). Only out-of-pocket utilization possessed an ICC<0.70. Patients reported that items were relevant and appropriate to measuring quality of care. No floor or ceiling effects were present. Criterion validity was reached for all items assessed. A priori hypotheses were confirmed. The HAPSQ represents an inexpensive, reliable, and valid approach toward collecting clinical information across a patient’s continuum of care.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Validation of a tool to assess patient satisfaction, waiting times, healthcare utilization, and cost
      Available formats

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Validation of a tool to assess patient satisfaction, waiting times, healthcare utilization, and cost
      Available formats

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Validation of a tool to assess patient satisfaction, waiting times, healthcare utilization, and cost
      Available formats


This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

Author for correspondence: Breda H. Eubank, Assistant Professor, Department of Health and Physical Education, Faculty of Health, Community, and Education, Mount Royal University, 4825 Mount Royal Gate SW, Calgary, AB, Canada T3E 6K6. E-mail:


Hide All

Cite this article: Eubank BH, Lafave MR, Mohtadi NG, Sheps DM, Wiley JP. (2019) Validation of a tool to assess patient satisfaction, waiting times, healthcare utilization, and cost. Primary Health Care Research & Development. 20(e47): 1–8. doi: 10.1017/S1463423619000094



Hide All
Ansell, D, Crispo, JAG, Simard, B Bjerre, LM (2017) Interventions to reduce wait times for primary care appointments: a systematic review. BMC Health Services Research 17, 295.10.1186/s12913-017-2219-y
Bartlett, JA (2002) Addressing the challenges of adherence. Journal of Acquired Immune Deficiency Syndrome 29 (Suppl 1), S2S10.10.1097/00126334-200202011-00002
Bokor, DJ, Hawkins, RJ, Huckell, GH, Angelo, RL Schickendantz, MS (1993) Results of nonoperative management of full-thickness tears of the rotator cuff. Clinical Orthopaedics and Related Research 294, 103110.10.1097/00003086-199309000-00013
Boorman, RS, More, KD, Hollinshead, RM, Wiley, JP, Brett, K, Mohtadi, NG, Nelson, AA, Lo, IK Bryant, D (2014) The rotator cuff quality-of-life index predicts the outcome of nonoperative treatment of patients with a chronic rotator cuff tear. Journal of Bone and Joint Surgery—American Volume 96, 18831888.10.2106/JBJS.M.01457
Brook, RH, McGlynn, EA Shekelle, PG (2000) Defining and measuring quality of care: a perspective from US researchers. International Journal of Quality Health Care 12, 281295.10.1093/intqhc/12.4.281
Buchert, AR Butler, GA (2016) Clinical pathways: driving high-reliability and high-value care. Pediatrics Clinics of North America 63, 317328.10.1016/j.pcl.2015.12.005
Cappelleri, JC, Gerber, RA, Kourides, IA Gelfand, RA (2000) Development and factor analysis of a questionnaire to measure patient satisfaction with injected and inhaled insulin for type 1 diabetes. Diabetes Care 23, 17991803.10.2337/diacare.23.12.1799
Chehade, MJ, Burgess, TA Bentley, DJ (2011) Ensuring quality of care through implementation of a competency-based musculoskeletal education framework. Arthritis Care Research (Hoboken) 63, 5864.10.1002/acr.20329
Donabedian, A (1988) The quality of care. How can it be assessed? JAMA 260, 17431748.10.1001/jama.1988.03410120089033
Eubank, B, Lafave, M, Wiley, JP, Sheps, D, Bois, A Mohtadi, N (2018) Evaluating quality of care for patients with rotator cuff disorders. BMC Health Services Research 18, 569.10.1186/s12913-018-3375-4
Eubank, BH, Mohtadi, NG, Lafave, MR, Wiley, JP, Bois, AJ, Boorman, RS Sheps, DM (2016) Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Medical Research Methodology 16, 56.10.1186/s12874-016-0165-8
Fournier, J, Heale, R Rietze, LL (2012) I can’t wait: advanced access decreases wait times in primary healthcare. Healthcare Quality 15, 6468.
Frank, C, Marshall, D, Faris, P Smith, C (2011) Essay for the CIHR/CMAJ award: improving access to hip and knee replacement and its quality by adopting a new model of care in Alberta. Canadian Medical Association Journal 183, E347E350.10.1503/cmaj.110358
Golin, CE, DiMatteo, MR Gelberg, L (1996) The role of patient participation in the doctor visit. Implications for adherence to diabetes care. Diabetes Care 19, 11531164.10.2337/diacare.19.10.1153
Gooch, KL, Smith, D, Wasylak, T, Faris, PD, Marshall, DA, Khong, H, Hibbert, JE, Parker, RD, Zernicke, RF, Beaupre, L, Pearce, T, Johnston, DW Frank, CB (2009) The Alberta Hip and Knee Replacement Project: a model for health technology assessment based on comparative effectiveness of clinical pathways. International Journal of Technology Assessment in Health Care 25, 113123.10.1017/S0266462309090163
Hall, RH Hanna, P (2004) The impact of web page text-background colour combinations on readability, retention, aesthetics and behavioural intention. Behaviour and Information Technology 23, 183195.10.1080/01449290410001669932
Health Council, of Canada (2012) Measuring and reporting on health system performance in Canada: Opportunities for improvement. Retrieved 24 May 2017 from
Health Quality Council, of Alberta (2003) Alberta quality matrix for health user guide. Health Quality Council of Alberta. Retrieved 5 May 2017 from file:///D|/Gowtham/01_CUP/CUP/01_AOP/PHC/PHC_EA/1900009/
Hertzog, M (2008) Considerations in determining sample size for pilot studies. Research in Nursing and Health 31, 180191.10.1002/nur.20247
Institute for Clinical, Evaluative Sciences (2012) Quality monitor. 2012 report on Ontario’s health system. Health quality Ontario. Retrieved 24 May 2017 from
Jo, YH, Lee, KH, Kim, SJ, Kim, J Lee, BG (2017) National trends in surgery for rotator cuff disease in Korea. Journal of Korean Medical Science 32, 357364.10.3346/jkms.2017.32.2.357
Jonsson, A Svingby, G (2007) The use of scoring rubrics: reliability, validity, and educational consequences. Educational Research Review 2, 130144.10.1016/j.edurev.2007.05.002
Kane, RL (2006) Understanding health care outcomes research. Sudbury, MA: Jones and Bartlett.
Kemp, KA, Sheps, DM, Luciak-Corea, C, Styles-Tripp, F, Buckingham, J Beaupre, LA (2011) Systematic review of rotator cuff tears in workers’ compensation patients. Occupational Medicine (Lond) 61, 556562.10.1093/occmed/kqr068
Kuhn, JE, Dunn, WR, Sanders, R, An, Q, Baumgarten, KM, Bishop, JY, Brophy, RH, Carey, JL, Holloway, BG, Jones, GL, Ma, CB, Marx, RG, McCarty, EC, Poddar, SK, Smith, MV, Spencer, EE, Vidal, AF, Wolf, BR Wright, RW (2013) Effectiveness of physical therapy in treating atraumatic full-thickness rotator cuff tears: a multicenter prospective cohort study. Journal of Shoulder and Elbow Surgery 22, 13711379.10.1016/j.jse.2013.01.026
Kujala, J, Lillrank, P, Kronstrom, V Peltokorpi, A (2006) Time-based management of patient processes. Journal of Health Organization and Management 20, 512524.10.1108/14777260610702262
Kukkonen, J, Joukainen, A, Lehtinen, J, Mattila, KT, Tuominen, EK, Kauko, T Aarimaa, V (2015) Treatment of nontraumatic rotator cuff tears: a randomized controlled trial with two years of clinical and imaging follow-up. Journal of Bone and Joint Surgery—American Volume 97, 17291737.10.2106/JBJS.N.01051
Lau, B (2009) Development and implementation of a Healthcare Access and Patient Satisfaction Questionnaire (HAPSQ) for measuring wait times, satisfaction, and costs with acute knee injury care in Alberta, Masters of Science Thesis. Faculty of Kinesiology. Calgary, Alberta: University of Calgary.
Lau, B, Lafave, M, Mohtadi, N Butterwick, D (2012) Utilization and cost of a new model of care for managing acute knee injuries: the Calgary Acute Knee Injury Clinic. BMC Health Services Research 12, 445.10.1186/1472-6963-12-445
Leatherman, S Sutherland, K (2010) Quality of healthcare in Canada: A chartbook. Canadian Foundation for Healthcare Improvement. Retrieved 24 May 2017 from
Lewis, CL, Wickstrom, GC, Kolar, MM, Keyserling, TC, Bognar, BA, DuPre, CT Hayden, J (2000) Patient preferences for care by general internists and specialists in the ambulatory setting. Journal of General Internal Medicine 15, 7583.10.1046/j.1525-1497.2000.05089.x
Marshall, DA, Christiansen, T, Smith, C, Squire, HJ, Werle, J, Faris, P Frank, C (2015) Continuous quality improvement program for hip and knee replacement. American Journal of Medical Quality 30, 425431.10.1177/1062860614540512
McGlynn, EA (1997) Six challenges in measuring the quality of health care. Health Affairs (Millwood) 16, 721.10.1377/hlthaff.16.3.7
McGlynn, EA, Asch, SM, Adams, J, Keesey, J, Hicks, J, DeCristofaro, A Kerr, EA (2003) The quality of health care delivered to adults in the United States. New England Journal of Medicine 348, 26352645.10.1056/NEJMsa022615
Mohtadi, N, Chan, D, Lau, B Lafave, M (2012) An innovative Canadian solution for improved access to care for knee injuries using “Non-Physician Experts”: the Calgary Acute Knee Injury Clinic. Rheumatology S2,
Mokkink, LB, Terwee, CB, Knol, DL, Stratford, PW, Alonso, J, Patrick, DL, Bouter, LM de Vet, HC (2010) The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Medical Research Methodology 10,
Nunally, J Bernstein, I (1994) Pscychometric theory, 3rd edn.. New York: McGraw-Hill.
Peacock, S, Chan, C, Mangolini, M Johansen, D (2001) Techniques for measuring efficiency in health services. Staff Working Paper. Productivity Commission. Retrieved 24 May 2017 from
Rozario, PA, Morrow-Howell, N Proctor, E (2004) Comparing the congruency of self-report and provider records of depressed elders’ service use by provider type. Medical Care 42, 952959.10.1097/00005650-200410000-00003
Schippits, M Schippits, M (2013) Clinical pathways leading healthcare reform: transformational strategies for oncology and beyond. Journal of Medicine and the Person 11, 6268.10.1007/s12682-013-0151-4
Schull, MJ, Guttmann, A, Leaver, CA, Vermeulen, M, Hatcher, CM, Rowe, BH, Zwarenstein, M Anderson, GM (2011) Prioritizing performance measurement for emergency department care: consensus on evidence-based quality of care indicators. CJEM 13, 300343.10.2310/8000.2011.110334
Shrout, PE Fleiss, JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychology Bulletin 86, 420428.10.1037/0033-2909.86.2.420
Sofaer, S Firminger, K (2005) Patient perceptions of the quality of health services. Annual Review of Public Health 26, 513559.10.1146/annurev.publhealth.25.050503.153958
SPSS, Inc (2007) Statistical Software: Release 17.0. Chicago, IL: SPSS Inc. [computer program].
Swan, BA Boruch, RF (2004) Quality of evidence: usefulness in measuring the quality of health care. Medical Care 42, II12II20.10.1097/01.mlr.0000109123.10875.5c
Tashjian, RZ (2016) The natural history of rotator cuff disease: evidence in 2016. Techniques in Shoulder and Elbow Surgery 17, 132138.10.1097/BTE.0000000000000109
Tisnado, DM, Adams, JL, Liu, H, Damberg, CL, Chen, WP, Hu, FA, Carlisle, DM, Mangione, CM Kahn, KL (2006) What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Medical Care 44, 132140.10.1097/
United States Bone, and Joint Initiative (2014) The burden of musculoskeletal diseases in the United States (BMUS), third edition. Rosemont, IL. Retrieved from 19 January 2018
Yamaguchi, K (2011) New guideline on rotator cuff problems. AAOS Now 5, 14.
Zukerberg, AL, Von Thurn, DR Moore, JC (1995) Practical considerations in sample size selection for behavior coding pretests. Proceedings of the Section on Survey Research Methods. American Statistical Association. Retrieved from 19 January 2018



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed