The first decade of the 21st century has lifted HTA from an academic niche to a prominent and visible position. Meanwhile numerous national health ministries, the European Commission, and the World Health Organization (WHO) have all proposed health technology assessment (HTA) as an indispensable coping strategy to appropriately confront the influx of new technologies and rising costs. The experiences and options for institutionalizing HTA in different countries and health systems were explored. Moreover, a meeting in 2000 convened by the WHO Regional Office for Europe encouraged its further development (12).
The institutionalization of HTA had been defined in that context as “promoting the structures and processes suitable to produce technology assessments that will be powerful in guiding policy and clinical practice toward the best possible health and cost outcomes” (12). HTA is in the process of becoming established and institutionalized both in individual countries and at the international level (Reference Draborg and Gyrd-Hansen3). However, most (70 percent) of the countries in the European region, and more than a half of European Union (EU) countries, do not yet have formal HTA (Table 1).
Table 1. HTA Agencies in European Countries (November 2008)
Note. Table 1 lists the countries with and without HTA agencies that are a member of the International Network of Agencies for Health Technology Assessment (INAHTA), which is regarded as an indicator for the institutionalization of the Agency. INAHTA membership implies that the organization is non-for-profit, relates to a regional or national government and is funded by public sources for at least 50%.Formal HTA: Countries with HTA agencies that are INAHTA members.
aConsiderable activity in HTA but no INAHTA member agency.
As HTA agencies and programs have flourished, attention to coordinating HTA activities at the European and international levels has been increasing. In Europe, the need for coordination was subsequently addressed by the following consecutive projects commissioned by the EU: The EUR-ASSESS project (Reference Banta1) (1994–97); the HTA-Europe project (Reference Banta and Oortwijn2) (1997–99); and the ECHTA/ECAHI (1999–2001) (Reference Jonsson, Banta, Henshall and Sampietro-Colom5).
As mentioned above, the number of European countries that have formal HTA remains limited. Moreover, little is known about the current state of HTA activities and endeavors to establish formal HTA, especially as regards new European Member States, candidate and potential candidate countries, and other countries in the European region. Knowledge about ongoing activities in HTA, and the barriers and solutions involved in establishing HTA units and their daily work processes, builds the basis for effective measures for its institutionalization. EUnetHTA (Reference Kristensen, Lampe and Chase6;Reference Kristensen, Mäkelä and Allgurin Neikter7) Work Package 8 (WP8)—Systems to support HTA in EU Member States with limited institutionalization of HTA—intended to consider such countries that either did not have formal HTA or were in the process of establishing formalized HTA. The Handbook on HTA Capacity Building was the main deliverable in accomplishing the aim of WP8.
WP8 was composed of eleven Associated Partners and seventeen Collaborating Partners. The Catalan Agency for Health Technology Assessment and Research (CAHTA) was in charge of coordinating WP8 and was responsible for supervising the implementation of tasks, designing surveys, analyzing data, and preparing final reports. WP8 Partners met three times during the course of the Project.
AIM AND OBJECTIVES
The aim of WP8 was to support HTA capacity building in countries with limited experiences in HTA or without institutionalized HTA. Three objectives were delineated for WP 8: (i) Define the minimum components related to the scope, structure, process, and visibility of an HTA organization; (ii) Develop tools for information support to institutions or healthcare systems in the process of evolving to/building an HTA organization/capacity; and (iii) Produce a handbook on HTA capacity building.
METHODS
The methods used to address the objectives were as follows:
(i) Survey of HTA Organizations
Objective. To gather knowledge on the current state of HTA worldwide and its degree of institutionalization and to gain insight on the characteristics and processes of such organizations.
Design. Cross-sectional study by means of a semistructured questionnaire sent by means of email.
Period of study. November 2006 through January 2007. The questionnaire was designed after conducting a review of HTA organizations (2006), which included Web sites of existing HTA units and relevant publications to learn about their characteristics in relation to structure, process, and visibility.
Sample. 149 HTA organizations worldwide. The list of HTA organizations included was compiled by merging information from various sources: directory of EUnetHTA Partners; members of the International Network of Agencies for Health Technology Assessment (INAHTA); Web sites of existing HTA units; and HTA organizations that collaborated in previous projects and surveys.
Analysis. Standard descriptive analyses were conducted to characterize the organizations.
(ii) Survey on Information Management in HTA Organizations
Objective. To describe the current state of development, the processes, and resources used in HTA information management and the characteristics of personnel involved in HTA information services worldwide.
Design. A cross-sectional survey by means of a semistructured questionnaire sent by means of email to information specialists.
Period of study. January 2007 through March 2007.
Sample. A total of 137 HTA organizations internationally. The survey sample was the same used previously in the other survey of HTA organizations. However, duplicate entries (due to agency name changes, and English versus national language agency names) were removed.
Analysis. Standard descriptive analyses were conducted to characterize the organizations. Analysis of the response rate was considered by geographical region and by country.
(iii) Survey on HTA Educational Programs
Objective. To provide an updated overview of most relevant educational programs on HTA and HTA-related areas worldwide.
Design. Cross-sectional study by means of a Web-based survey.
Period of study. April 2008.
Sample. Eighty organizations (INAHTA members, some members of the Health Technology Assessment International [HTAi]; Cochrane Centers, and selected WP8 Partners outside of EU).
Analysis. Standard descriptive analyses were conducted, and the programs were classified into three categories: MSc in HTA; MSc HTA-related areas, and other courses part of a postgraduate HTA-related course.
(iv) Production of the Handbook on HTA Capacity Building
The main sources for the Handbook included findings from the international survey on HTA organizations (Reference Moharra, Kubesch, Estrada, Parada, Cortes and Espallargues10), the survey on information management units (Reference Kubesch, Parada, Moharra, Estrada, Cortés and Espallargues8), expert and stakeholder opinions, and the analysis of other relevant literature (4). Both surveys included international respondents because EUnetHTA involves not only EU Member States, but also other health-related organizations worldwide.
RESULTS
Survey of HTA Organizations
Fifty-two responses were received, representing 35 percent of the total sample (149 organizations). From the fifty-two responses, eleven were excluded from the analysis, as they reported not being active in HTA activities. Hence, the results were based on forty-one responses.
Establishment. The main initiative for establishing the organization was governmental (61.0 percent), followed by health researchers (29.3 percent), and decision makers (24.4 percent). Respondents provided data on the main barriers confronting the establishment of the organization; specifically 58 percent reported having faced barriers. The main barriers centered on acquiring trained staff (63.6 percent) and funding (45.5 percent).
Specific Background, Aims, and Scope. All respondent organizations were not-for-profit, and 42.5 percent described their profile as a governmental agency and 32.5 percent as academia/university. Most of the respondents reported collaboration (either at the national or international level). At the national level, they reported collaboration with academia/universities (97.4 percent) and with other governmental agencies (94.9 percent). Most reported having approved statutes (67.6 percent) and a strategic plan (82.9 percent).
The main lines of activity reported by the organizations were HTA (80.5 percent), followed by other types of health research (63.4 percent), and clinical practice guidelines (36.6 percent). The types of health technology (HT) most commonly assessed were pharmaceuticals (77.5 percent), medical (or surgical) procedures (75.0 percent), and medical devices (70.0 percent).
Assessment reports (97.6 percent) and academic and training activities (78.8 percent) were the two types of products that the organizations most frequently produced. The respondents indicated that an HTA report usually involves formulating recommendations (80.5 percent), targeted mainly at policy makers (90.9 percent) and public healthcare providers (81.9 percent).
Structure. Most HTA agencies reported having a director, president, or manager (73.2 percent) followed by having a scientific or advisory committee (46.3 percent). Most organizations had administrative staff (94.6 percent), collaborating researchers (75.0 percent), and research assistants (55.9 percent). The most important funding sources were the Government (80.5 percent), followed by research funding bodies (46.3 percent), and private industries (24.4 percent).
Work Process. The main responsibility for setting priorities in the organization rested with the department/ministry of health (53.7 percent), followed by the director (43.9 percent), and the executive board (29.3 percent). Approximately half (52.3 percent) of the organizations used an explicit process.
Visibility. Respondents commented that they had a formal procedure to disseminate their products (75 percent). The most frequently used methods for dissemination included posting reports on the agency's Web site (92.3 percent), participating in and organizing academic, scientific, and training activities (84.6 percent), and producing electronic and printed versions of reports (79.5 percent).
Survey on Information Management in HTA Organizations
Thirty-one responses were received from fifteen countries. This represents 23 percent of the total sample (137 organizations). From the thirty-one responses, eight were excluded from the analysis because of exclusion criteria. Hence, the results were based on twenty-three questionnaires.
Structure and Organizational Data. All twenty-three respondents (100 percent) described the principal task of the information unit was to conduct bibliographic database searches. Other common activities were acquisitions or subscriptions to information resources (82.6 percent), answering specific information questions (78.3 percent), and managing project archives (69.6 percent). Webmaster tasks were reported by 39.1 percent of the respondents.
Managing Information Resources. Health bibliographic databases were reported to be the most used (100 percent) information source in assessing health technologies, followed by other sources.
Professional and Academic Background of the Staff. Most organizations (78.1 percent) employed between one and fourteen professionals with a background in information science/library sciences. With respect to the highest level of formal education of personnel working in these information units, the data revealed that the most frequent (56.5 percent) educational level was the Master's degree.
Survey on Educational HTA Programs and Courses. In total, forty-eight of eighty institutions answered the survey. Three MSc programs in HTA were identified. One MSc program was international in scope, whereas Canada and the United Kingdom offered national MSc programs in HTA. Other MSc programs in HTA-related areas were identified in Brazil, Canada, Israel, and Spain. Courses that were part of a postgraduate HTA-related course were identified in Argentina, Australia, Canada, Romania, and the United Kingdom.
The Handbook on HTA Capacity Building. The Handbook was the main deliverable of WP8. It addressed stakeholders who potentially had an interest in HTA capacity building, for example, healthcare administrators (local, regional, national, international), public and private healthcare providers, healthcare industry, healthcare payers, healthcare researchers, and other stakeholders.
Apart from CAHTA, which was responsible for coordinating development, twenty researchers from thirteen different countries contributed to the Handbook. It was subjected to an internal peer review process, and the document was also available for public review and comment on the EUnetHTA Web site.
The Handbook consists of nine chapters (Figure 1), each devoted to one specific field which also includes specific final remarks. Chapter 8 presents some of the main conclusions and recommendations of the Handbook (Figure 2).
Figure 1. Chapter contents and some final remarks of the Handbook on Health Technology Assessment (HTA) Capacity Building.
Figure 2. Some conclusions and recommendations of the Handbook on Health Technology Assessment (HTA) Capacity Building.
DISCUSSION
Establishing an effective HTA program that guides key policy decisions for a national healthcare system is a challenging task. The basis for this task is a solid commitment from politicians and key decision makers in the health system to integrate HTA findings and recommendations into key decision-making on the policy level. Governments and healthcare organizations took the lead in promoting the establishment of dedicated HTA agencies, which are largely not-for-profit institutions. Moreover, organizational characteristics regarding establishment, background, structure, and work processes do not appear to have changed significantly in the past 10 years.
However, it is recommended that an appropriate organizational structure and an efficient institutional approach to HTA work should be identified in advance. This does not necessarily signify the establishment of a dedicated HTA agency. Good examples can be found of network models with a coordination mechanism (e.g., an HTA office) that could be appropriate for many countries.
Common facilitating and impeding factors have been experienced in establishing HTA organizations regardless of their geographical setting or years of experience in conducting HTA activities, for example, securing funding and attracting trained staff. The former being especially relevant as many types of experts are needed in HTA, which could be partly overcome by intense networking and international collaboration. To ensure the continuous professional development necessary for their evolution, HTA organizations require multidisciplinary teams and adaptive human resources.
This highlights the need for educational and training programs in HTA and related areas to ensure that trained personnel, representing different disciplines but having a common language, can cooperate in performing HTAs. Therefore, sufficient investment funds should be available to train professionals in HTA. Funding for the recurrent operational costs of the established HTA structure should also be identified and secured on a long-term basis.
HTA is no longer conducted in national isolation. Collaboration either at the national or international levels could help overcome such difficulties. Networking among organizations plays an important role in the development and sharing of HTA activities, but better coordination and communication among HTA programs is needed.
Collaboration among HTA organizations is extremely relevant for countries without institutionalized HTA because it offers the opportunity to learn from others’ experiences and benefit from HTA studies that have already been done. However, one should proceed with caution, as the transferability of HTA evidence obtained from other organizations is not always straightforward. Both EU countries and non-EU countries have a high level of collaboration in the field of HTA. Often, however, only a small proportion of an HTA agency's activities are embedded in international projects. Hence, this collaboration should be promoted and supported by the European Union because it can help reduce duplication, thereby enabling resources to be used more efficiently and assuring the timeliness of HTA reports.
Ultimate success also depends on the quality and relevance of the HTA reports, an efficient information dissemination system, and willingness at the policy level to integrate HTA in decision making. All HTA organizations, particularly newly established ones, should secure high-quality products to establish themselves as scientific evidence reviewers in their context. It is important to take advantage of new technologies and to promote and be more active in planning different actions and strategies that include identifying HTA target audiences to avoid becoming fixed to the most common and known visions of distribution (Web site, participation and organizations of academic and training activities). Information specialists play an important role at this stage, but also throughout the development of the HTA and often beyond its completion, for example, in dissemination activities.
The Handbook on HTA Capacity Building adds new information and is specifically important because some of the contributors came from formal HTA agencies and were knowledgeable about the processes and structures of formal agencies. Other contributors were actively involved in the process of establishing formal HTA in their countries and for that reason contributed the handbook's sensitivity for obstacles these countries faced in institutionalizing the process. Moreover, the information may assist not only new HTA organizations, but it could also serve as a guide for improving processes in existing HTA agencies. The Handbook also tries to identify the challenges that HTA capacity building currently faces in its different stages. Although the Handbook was developed within a European context, it can be considered globally relevant and applicable because its recommendations are based on international data.
As a field, HTA is becoming established and institutionalized both in individual countries and internationally. Particular support for introducing formal HTA should be dedicated to Eastern and Central Europe to address the growing interest, for example, in Russia, Romania, Slovenia, and the Baltic States.
The surveys of HTA organizations and information management units had some limitations. For instance, the study sample might not be representative of all existing HTA institutions. In merging the information from various sources to prepare the list of HTA organizations, the sample underrepresented some countries and organizations, especially among Eastern European countries. However, this could also reflect that HTA is not as well institutionalized in this region as in other European regions. A general limitation in the surveys was the low total number of responses, which restricted the analysis. The nonrespondents among information management units might reflect a high percentage of HTA organizations without information specialists that did not respond because they did not consider themselves to be appropriate recipients. This could also apply to very small units, for example, those consisting of only one or two information specialists, or that dedicate only a part of their time to HTA searches.
This low response rate to surveys limited the sample size for bivariate and multivariate analyses, making it difficult to identify the factors or characteristics associated with more effective HTA organizations/programs. Other difficulties concerning the surveys were related to simplifying and classifying complex organizational models into key characteristics. Participation in the development of the Handbook was also limited.
Because WP8 included twenty-eight Partners, it was not always easy to coordinate such a large group. The first WP8 meeting was not scheduled at an early stage of the work, and this made it particularly difficult to coordinate and communicate among WP8 Partners. E-meetings were scheduled initially, but problems with Internet connections resulted in email becoming the most common communication channel.
A reduced budget caused some redistribution in the WP8 work plan. The work plan submitted at the outset was cut back in light of not being able to fulfill all objectives and functions that were described initially. Moreover, this suggested greater in-kind contributions to the Project than originally anticipated.
Future actions might include diverse strategies to help organizations implement HTA activities, such as producing an electronic format or a second edition of the Handbook (with active links to Web sites, documents, and practical information describing the experience of countries) or generating a database of HTA experts (people who could advise others on HTA institutionalization or more specific HTA activities). Other actions might involve building relevant indicators from results achieved by previous WP8 objectives to establish standards of quality for assessing the implementation and progress of new HTA agencies. The various WP8 products developed during the 3-year project could help define the relevant indicators for structure, process, and outcomes related to a reference framework (benchmarking) for strategies to develop new HTA units, programs, or agencies.
CONCLUSIONS
New technologies enter global markets swiftly and in large numbers. No single country can meet the need for HTA on its own. Therefore, an enhanced national and international coordination strategy is recommended, even though the debate on how to formalize this raises questions about its geographic scope and organizational structure (network or agency). Setting up organizational structures and establishing effective HTA programs that guide key policy decisions is a challenging task. There are no standard models or pathways. The “one size fits all” principle is not useful because of the wide systemic and cultural differences between countries. However, the Handbook on HTA Capacity Building presents some approaches for overall institutional development, especially in terms of formulating objectives, creating structures, and defining work processes by learning from the experience of already established HTA organizations. Comparisons with the experiences of other HTA organizations may provide some indications based on shared experiences. However, appropriate policies and laws need to be put in place to anchor and mandate the use of HTA for investment, reform, and reimbursement decisions.
The complexity of HTA has increased, while the theoretical foundation for the field remains underdeveloped. It is time for HTA to bring together more aspects of conceptual and theoretical works from other fields. Methodological research and development also needs to be undertaken to improve the relevance and validity of HTA findings.
The question remains open regarding how and where along the line from assessment (objective analysis of the impact of technology) to appraisal (interpretation of objective analysis) to decision making, the full breadth of perspectives on new technology should come into play—and this will vary among healthcare systems. In relation to scope, HTA can be viewed increasingly as a tool for promoting the introduction of cost-effective and promising technologies, not merely as an instrument for cost containment or gatekeeping.
In fact, there is a need to undertake, adapt, and interpret HTA on an international level for several reasons other than the increasing interest in networking and collaboration. Because most HTA organizations have a limited ability to undertake new primary data collection, many of them only use evidence from published sources. Assessments related to public health interventions, emerging technologies, and support systems remain less developed in comparison to pharmaceuticals, medical procedures, and medical devices.
Several challenges have been identified in relation to specific work processes. First, priority-setting mechanisms need to be developed that are transparent, flexible, and suit the needs of those who use them. Second, it is essential to foster cooperation and communication among HTA producers, users, and other stakeholders to ensure more comprehensive assessments of a wider set of technologies, reduce potential duplication, and ensure that assessments are in line with decision-makers’ priorities (Reference Marzo9).
Third, it is important to achieve balance between validity and opportunity in producing assessments in a way that secures timely, high-quality products that actually serve decision making in policy and practice. Another challenging strategy for some HTA organizations is to involve clinicians in the HTA process, specifically those who use or know the technology under evaluation. However, the most important future action may be to systematically and periodically evaluate the functioning of HTA organizations. Given HTA's focus on evaluation, it would be particularly misguided to exempt the HTA organization and HTA processes from evaluation.
Concerning visibility, HTA practitioners have recognized that dissemination activities are a key process that needs to be intensified. Hence, we need to develop a more comprehensive framework for analyzing the extent to which HTA has contributed to making rational choices, including its influence on decisions, the use and diffusion of technology and health outcomes, access, and efficiency (11).
Because the technologies that HTA must consider are evolving rapidly, the need to translate HTA results into policy, management, and practice decisions is an enduring theme from HTA's earliest days.
CONTACT INFORMATION
Montse Moharra, BA (mmoharra@aatrm.catsalut.net), Research Assistant, Health Technology Assessment, Mireia Espallargues, MD, PhD (mespallargues@aatrm.catsalut.net), Assistant Director for Health Care Quality Assessment, Health Care Quality, Nadine Kubesch, BSc (Hons), MPH (nadine.kubesch@web.de), Research Associate, Health Technology Assessment, Maria-Dolors Estrada, MD (destrada@aatrm.catsalut.cat), Senior Assessment Technician, Health Care Quality Assessment, Antoni Parada, BA (aparada@aatrm.catsalut.cat), Head, Communication and Knowledge Transfer Unit, Catalan Agency for Health Technology Assessment and Research, 885 Roc Boronat, 81–95 (2a planta), E-08005, Barcelona, Spain
Hindrik Vondeling, PhD (hvo@sam.sdu.dk), Associate Professor, Centre for Applied Health Services Research and Technology Assessment, University of Southern Denmark. University of Southern Denmark, 9, Winsloewsvej, Odense, 5000, Denmark
Alessandra Lo Scalzo, MPhil, MSc (loscalzo@agenas.it), Research Officer, Innovation and Development, Agenzia Nazionale per i Servizi Sanitari Regionali, Via Puglie 23, Rome, Italy, 00187
Stelios Christofides, PhD (cstelios@cytanet.com.cy), Head, Medical Physics, Nicosia General Hospital, 215 Old Nicosia Limassol Road, Nicosia, 2029, Cyprus
Eva Turk, MA, MBA (eva.turk@ivz-rs.si), Senior Researcher, Center for Health Economics and Organization in Health Care, Institute of Public Health of the Republic of Slovenia, Trubarjeva 2, Ljubljana, 1000, Slovenia
Martin Raab, MSc, MPH (martin.raab@unibas.ch), Health Technology Assessment Advisor, Swiss Network for Health Technology Assessment, Swiss Federal Office for Public Health, Medical Technology Unit, Seilerstrasse 8, CH-3003 Bern, Switzerland
