Skip to main content Accessibility help
×
Home

Association between methodological characteristics and outcome in health technology assessments which included case series

  • Ken Stein (a1), Kim Dalziel (a1), Ruth Garside (a1), Emanuela Castelnuovo (a1) and Ali Round (a1)...

Abstract

Objectives: Case series constitute a weak form of evidence for effectiveness of health technologies. However, for a variety of reasons, such studies may be included in health technology assessments. There are no clear criteria for assessing the quality of case series. We carried out an empirical investigation of the association between outcome frequency and methodological characteristics in a sample of health technology assessments.

Methods: Systematic reviews of functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, and percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic stable angina were identified as containing more than forty case series. Data were extracted by one reviewer and checked by a second on population characteristics, outcomes, and the following methodological features: sample size, prospective/retrospective approach, consecutive recruitment, multi- or single-center organization, length of follow-up, independence of outcome measurement, and date of publication. Association between methodological features and outcome were explored in univariate and multivariate analyses using parametric and nonparametric tests and robust regression or analysis of variance/analysis of covariance, as appropriate.

Results: Included reviews contained between forty-two and seventy-six case series studies, involving 5 to 172,283 participants. Reporting of methodological features was poor and limited the analyses. In general, we found little evidence of any association between methodological characteristics and outcome. Sample size is used as an inclusion criterion in many reviews of case series but was consistently shown to have no relationship to outcome in all analyses. A prospective approach was not associated with outcome. Insufficient data were available to explore consecutive recruitment. Mixed results were shown for length of follow-up, independence of outcome measurement, and publication date.

Conclusion: We found little evidence to support the use of many of the factors included in tools used for quality assessment of case series. Importantly, we found no relationship between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, for example, blinding of outcome measurement, but these examples were not shown consistently across the small number of examples studied. Further research into the determinants of quality in case series studies is required to support health technology assessment.

Copyright

References

Hide All
Bryant J, Cave C, Mihaylova B, et al. 2002 Clinical effectiveness and cost-effectiveness of growth hormone in children: A systematic review and economic evaluation. Health Technol Assess. 6: 1168.
Clarke M, Oxman A. 2002. Cochrane Reviewers' Handbook 4.1.5 [updated April 2002]. The Cochrane Library. Oxford: Update Software;
Dalziel K, Garside R, Stein K, Round A. 2003. Use of case series evidence in National Institute for Clinical Excellence (NICE) health technology assessments—why, how and did it matter? Canmore, Alberta: International Society for Technology Assessment in Health Care (ISTAHC);
Dalziel K, Stein K, Round A, Garside R. 2003 Systematic review of endoscopic sinus surgery for nasal polyps. Health Technol Assess. 7: 1159.
Fitzpatrick R, Shortall E, Sculpher M, et al. 1998 Primary total hip replacement surgery: A systematic review of outcomes and modelling of cost-effectiveness associated with different prostheses. Health Technol Assess. 2: 164.
Grodstein F, Stampfer MJ, Colditz GA. 1997 Postmenopausal hormone therapy and mortality. N Engl J Med. 336: 17691775.
Hulley S, Grady D, Bush T, 1998 for the Heart and Estrogen/Progestin Replacement Study (HERS) Research group. Randomised trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA. 280: 605613.
Hyde C, Wake B, Bryan S, et al. 2002 Fludarabine as second-line therapy for B-cell chronic lymphocytic leukaemia: A technology assessment. Health Technol Assess. 6: 189.
Jobanputra P, Parry D, Fry-Smith A, Burls A. 2001 Effectiveness of autologous chondrocyte transplantation for hyaline cartilage defects in knees: A rapid and systematic review. Health Technol Assess. 5: 157.
Juni P, Altman D, Egger M. 2000 Assessing the quality of randomised controlled trials. In: Egger M, Davey-Smith G, Altman D, eds. Systematic reviews in health care. London: BMJ Publishing Group;
Sculpher M, Petticrew M, Kelland J, et al. 1998 Resource allocation for chronic stable angina: A systematic review of effectiveness, costs and cost-effectiveness of alternative interventions. Health Technol Assess. 2: 1176.
Taylor R, Van Buyten J-P, Buchser E, et al. 2004 Spinal cord stimulation for chronic low back pain/failed back surgery syndrome and complex regional pain syndrome–systematic review of the clinical effectiveness and cost-effectiveness literature. J Pain Symptom Manage. 27: 370378.
Vale L, Wyness L, McCormack et al. 2002 A systematic review of the effectiveness and cost-effectiveness of metal-on-metal hip resurfacing arthroplasty for treatment of hip disease. Health Technol Assess. 6: 1109.
Vardulaki KA, Bennett-Lloyd B, Parfitt J, et al. 2000. A systematic review of the effectiveness and cost-effectiveness of laparoscopic surgery for colorectal cancer. London: National Institute for Clinical Excellence;
Wake B, Hyde C, Bryan S, et al. 2002 Rituximab as third-line treatment for refractory or recurrent stage III or IV follicular non-Hodgkin's lymphoma: A systematic review and economic evaluation. Health Technol Assess. 6: 185.

Keywords

Association between methodological characteristics and outcome in health technology assessments which included case series

  • Ken Stein (a1), Kim Dalziel (a1), Ruth Garside (a1), Emanuela Castelnuovo (a1) and Ali Round (a1)...

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed