Skip to main content Accessibility help

Setting priorities in clinical and health services research: Properties of an adapted and updated method

  • Silvina Berra (a1), Emília Sánchez (a1), Joan M. V. Pons (a2), Cristian Tebé (a1), Jordi Alonso (a3) and Marta Aymerich (a4)...


Objectives: The objectives of this study is to review the set of criteria of the Institute of Medicine (IOM) for priority-setting in research with addition of new criteria if necessary, and to develop and evaluate the reliability and validity of the final priority score.

Methods: Based on the evaluation of 199 research topics, forty-five experts identified additional criteria for priority-setting, rated their relevance, and ranked and weighted them in a three-round modified Delphi technique. A final priority score was developed and evaluated. Internal consistency, test–retest and inter-rater reliability were assessed. Correlation with experts’ overall qualitative topic ratings were assessed as an approximation to validity.

Results: All seven original IOM criteria were considered relevant and two new criteria were added (“potential for translation into practice”, and “need for knowledge”). Final ranks and relative weights differed from those of the original IOM criteria: “research impact on health outcomes” was considered the most important criterion (4.23), as opposed to “burden of disease” (3.92). Cronbach's alpha (0.75) and test–retest stability (interclass correlation coefficient = 0.66) for the final set of criteria were acceptable. The area under the receiver operating characteristic curve for overall assessment of priority was 0.66.

Conclusions: A reliable instrument for prioritizing topics in clinical and health services research has been developed. Further evaluation of its validity and impact on selecting research topics is required.



Hide All
1. Asua Batarrita, J. Priorización de necesidades de evaluación en el País Vasco. Madrid: Universidad Complutense de Madrid, Instituto de Salud Carlos III; 1999.
2. Bernal-Delgado, E, Peiró, S, Sotoca, R. Prioridades de investigación en servicios sanitarios en el Sistema Nacional de Salud. Una aproximación por consenso de expertos. Gac Sanit. 2006;20:287294.
3. Black, N. A national strategy for research and development: Lessons from England. Annu Rev Public Health. 1997;18:485505.
4. Brown, P, Brunnhuber, K, Chalkidou, K, et al. How to formulate research recommendations. BMJ. 2006;333:804806.
5. Brown, K, Dyas, J, Chahal, P, et al. Discovering the research priorities of people with diabetes in a multicultural community: A focus group study. Br J Gen Pract. 2006;56:206213.
6. Buxton, M, Hanney, S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:3543.
7. Caron-Flinterman, JF, Broerse, JE, Teerling, J, Bunders, JF. Patients’ priorities concerning health research: The case of asthma and COPD research in the Netherlands. Health Expect. 2005;8:253263.
8. Carson, N, Ansari, Z, Hart, W. Priority setting in public health and health services research. Aust Health Rev. 2000;23:4657.
9. Claxton, K, Ginnelly, L, Sculpher, M, Philips, Z, Palmer, S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme. Health Technol Assess. 2004;8:1103.
10. Dault, M, Lomas, J, Barer, M, on behalf of the Listening for Direction II partners. Listening for direction II. National consultation on health services and policy issues for 2004–2007. Ottawa, Ontario: Canadian Health Services Research Foundation and Institute of Health Services and Policy Research; 2004.
11. Department of Health. Best research for best health: A new national health research strategy. The NHS contribution to health research in England. London: HMSO; 2006.
12. Department of Health. Research for health. London: HMSO; 1992.
13. Donaldson, MS, Sox, HC, eds. Setting priorities for health technology assessment: A model process. Washington DC: National Academy Press; 1992.
14. Eden, J, Wheatley, B, McNeil, B, Sox, H, eds. Knowing what works in health care. Washington DC: National Academy Press; 2008.
15. Evidence-Based Care Resource Group. Evidence-based care: 1. Setting priorities: How important is this problem? Can Med Assoc J. 1994;150:12491254.
16. Fleurence, RL, Torgerson, DJ. Setting priorities for research. Health Policy. 2004;69:110.
17. Fulop, N, Allen, P, Clarke, A, Black, N. From health technology assessment to research on the organisation and delivery of health services: Addressing the balance. Health Policy. 2003;63:155165.
18. Gross, CP, Anderson, GF, Powe, NR. The relation between funding by the National Institutes of Health and the burden of disease. N Engl J Med. 1999;340:18811887.
19. Hanney, S, Mugford, M, Grant, J, Buxton, M. Assessing the benefits of health research: Lessons from research into the use of antenatal corticosteroids for the prevention of neonatal respiratory distress syndrome. Soc Sci Med. 2005;60:937947.
20. Henshall, C, Oortwijn, W, Stevens, A, Granados, A, Banta, D. Priority setting for health technology assessment. Int J Technol Assess Health Care. 1997;13:144185.
21. Institute of Medicine. Initial national priorities for comparative effectiveness research. Washington, DC: Institute of Medicine; 2009.
22. Jones, R, Lamont, T, Haines, A. Setting priorities for research and development in the NHS: A case study on the interface between primary and secondary care. BMJ. 1995;311:10761080.
23. Lionis, C, Stoffers, HE, Hummers-Pradier, , et al. Setting priorities and identifying barriers for general practice research in Europe. Results from an EGPRW meeting. Fam Pract. 2004;21:587593.
24. Meneu, R. Variabilidad de las decisiones médicas y su repercusión sobre las poblaciones. Barcelona: Masson; 2002.
25. Michaud, CM, Murray, CJ, Bloom, BR. Burden of disease: Implications for future research. JAMA. 2001;285:535539.
26. Moreno-Casbas, T, Martin-Arribas, C, Orts-Cortes, I, Comet-Cortes, P, Investen-ISCIII Co-ordination and Development of Nursing Research Centre. Identification of priorities for nursing research in Spain: A Delphi study. J Adv Nurs. 2001;35:857863.
27. Oortwijn, WJ, Vondeling, H, van Barneveld, T, van Vugt, C, Bouter, LM. Priority setting for health technology assessment in The Netherlands: Principles and practice. Health Policy. 2002;62:227242.
28. Phelps, CE, Parente, ST. Priority setting in medical technology and medical practice assessment. Med Care. 1990;28:703723.
29. Sánchez, E, Solans, M, Jaldon, C. VI call for clinical and health services research CAHTA 2006. CAHTA's Newsletter 2006. (accessed August 14, 2009)
30. Streiner, DL, Norman, GR. Health measurement scales. A practical guide to their development and use. 3rd ed. New York: Oxford University Press; 2003.
31. Swedish Council on Technology Assessment in Health Care. The ECHTA/ECAHI Project. 1999. (accessed August 14, 2009)
32. Tunis, SR, Stryer, DB, Clancy, CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:16241632.
33. Welfare, MR, Colligan, J, Molyneux, S, Pearson, P, Barton, JR. The identification of topics for research that are important to people with ulcerative colitis. Eur J Gastroenterol Hepatol. 2006;18:939944.
34. Zerhouni, EA. US biomedical research: Basic, translational, and clinical sciences. JAMA. 2005;294:13521358.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed