Skip to main content Accessibility help


  • Access
  • Open access


      • Send article to Kindle

        To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Available formats

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Available formats

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Available formats
Export citation



The aim of this study was to identify good practice principles for health technology assessment (HTA) that are the most relevant and of highest priority for application in Latin America and to identify potential barriers to their implementation in the region.


HTA good practice principles proposed at the international level were identified and then explored during a deliberative process in a forum of assessors, funders, and product manufacturers.


Forty-two representatives from ten Latin American countries participated. Good practice principles proposed at the international level were considered valid and potentially relevant to Latin America. Five principles were identified as priority and with the greatest potential to be strengthened at this time: transparency in the production of HTA, involvement of relevant stakeholders in the HTA process, mechanisms to appeal decisions, clear priority-setting processes in HTA, and a clear link between HTA and decision making. The main challenge identified was to find a balance between the application of these principles and the available resources in a way that would not detract from the production of reports and adaptation to the needs of decision makers.


The main recommendation was to progress gradually in strengthening HTA and its link to decision making by developing appropriate processes for each country, without trying to impose, in the short-term, standards taken from examples at the international level without adequate adaptation of these to local contexts.


To the event participants: Anmol Mullins (Alcon), Mariana Naranjo Muedano (AMGEN), Sean Nagle (Novartis), Adrian Griffin (Johnson & Johnson), Alarico Rodríguez (Fondo Nacional de Recursos - Uruguay), Albin Chaves Matamoros (Caja costarricense del Seguro Social - Costa Rica), Alexandre Lemgruber (OPS), Alicia Granados (Genzyme), Ana Eduviges Sancho Jiménez (Ministerio de Salud - Costa Rica), Ana Pérez Galán (Ministerio de Salud - Uruguay), Antoine Wolniewicz (Genzyme), Bernardo Tinajero (Novartis), Andrew Bruce (AMGEN), Carlo Crisostomo (Johnson & Johnson), Chris Henshall (HTAi), Clarice Alegre Petramale (CONITEC - Brasil), Diego Guarín (Merck), Elsa Koutsavakis. (Pfizer), Giovanni Francisco Guevara Vásquez: (Ministerio de Salud - El Salvador), Guillermo Sánchez Vanegas (IETS-Colombia), Homero Monsanto (Merck), Jens Grueger (Roche), Jocelyn Ramírez (Eli Lilly), Joice Valentim (Roche), José Luis Estrada Aguilar (IMSS - México), Joseph Cook (Pfizer), Lizbeth Acuña (Cuenta de Alto Costo - Colombia), Luis Gamero (Mnisterio de Salud - Perú), Manny Papadimitropoulos (Eli Lilly), María Luciana Aramijos Acurio (Ministerio de Salud - ECUADOR), Mariana Pineda (CENETEC - México), Marianela Castillo Riquelme (Ministerio de Salud - Chile), Mauricio Duarte Ruano (Caja costarricense del Seguro Social - Costa Rica), Rafael Zamora (PAMI - Argentina), Rubén Agustín Nieto (UCEETS - Argentina), Sean Tunis (HTAi), Sir Andrew Dillon (NICE-Reino Unido), Stefan Weber (Novartis International - Alcon), Virginia Baffigo Torre de Pinillos (ESSALUD - Perú). We also thank Tara Schuller for her work in translating the manuscript from Spanish to English. Financial support: The organization of and attendance at the Forum was financed by Health Technology Assessment International (HTAi). The preparation of this study was undertaken by the Institute for Clinical Effectiveness and Health Policy (IECS) with funding support from HTAi. Declaration: In any reproduction of this article, there should not be any suggestion that PAHO or this article endorse any specific organization or products. From “Evaluación de tecnologías sanitarias para la toma de decisiones en Latinoamérica: principios de buenas practices,” by Andrés Pichon-Riviere, Natalie Soto, Federico Augustovski, Sebastián García Martí, and Laura Sampietro-Colom, copyright 2017 PAHO, available at Translated with the permission of the Pan American Health Organization. (Non-official English translation from the original Spanish manuscript. In case of discrepancy, the original version [Spanish] shall prevail.)

Universal health coverage (UHC) is a health priority worldwide and many countries are committed to delivering UHC to their citizens. Given the limited resources available in all health systems, this requires having to prioritize and make decisions about which services will be provided, to whom, and at what cost (1). In 2012, the Pan-American Health Organization (PAHO) and then, in 2014, all member states of the World Health Organization (WHO), passed a resolution for health technology assessment (HTA) and UHC that urged member states to develop HTA capacity and to integrate the concepts and processes of HTA into strategies and work areas progressing toward universal health coverage (2;3). HTA is defined as a multidisciplinary field of policy analysis, which incorporates the medical, social, ethical, and economic implications of development, diffusion, and use of health technology (4).

The development of HTA has been remarkable over the past 3 decades worldwide, and particularly in the past 15 years in Latin America and the Caribbean (LAC). Today, it is an indispensable part of health systems in several countries in the region (5). Twenty-nine institutions across fourteen LAC countries have joined RedETSA, the HTA Network of the Americas coordinated by PAHO, and increasing numbers of countries in the region are using HTA reports to inform resource allocation decisions. These HTA reports are produced by several organizations, including HTA agencies, academic institutions, regulatory bodies, and also the producers of medicines and devices themselves.

HTA is being carried out and used by a wide variety of actors and it is increasingly being linked to the decision-making process. Therefore, there is reasonable concern to ensure that the production and application of HTA, and how it is used in decision making, is appropriate for each context. If HTA is not produced and used correctly, it runs the risk of generating inefficient resource allocation by granting coverage to interventions offering little or no benefit, preventing or delaying patient access to useful health technologies, and sending wrong messages to technology producers (4). For these reasons, various groups around the world have identified examples of good practices and established recommendations and guiding principles for HTA (614).

The publication of these different principles and recommendations has given rise to an important debate around the following questions (1520): (i) Are all good practice principles equally important?; (ii) Is it realistic, or reasonable, to expect all HTA agencies across the globe to adhere to these?; (iii) Should all be implemented in the same way in every health system, or are there regional or local characteristics that render certain principles inapplicable to some contexts?

To this debate it could be added that, in most cases, these good practice principles were developed by groups in high-income countries and intended for use in countries with established HTA processes and systems. Moreover, these recommendations typically come from academic groups and do not formally incorporate the perspectives of other stakeholders, such as government officials or payers, among others.

This study presents the results of the First Latin American Health Technology Assessment Policy Forum organized by Heath Technology Assessment International (HTAi) (21). The objectives of the Forum were to identify which of the HTA good practice principles proposed at the international level would be the most relevant and of highest priority for application in LAC; to identify barriers faced by the region in fostering their uptake; and, to formulate recommendations to health systems for improving HTA and its use in decision making in the region.


The Health Technology Assessment Policy Forum is a method used by HTAi to engage stakeholders and issue recommendations on specific topics. It has been operating for 13 years in Canada, the United States, and Europe; and in Asia for 4 years (22). The Forum was established to provide a neutral setting to engage in strategic discussions about the current state of HTA, its development, and implications for health systems, industry, patients, and other stakeholders. The Forum brings together representatives of three main institutional groups: (i) those who make decisions about the coverage and reimbursement/pricing of drugs and devices in health systems, (ii) those who produce HTA to support decision making, and (iii) biomedical companies that produce technologies.

At each Forum, a topic is discussed that was previously prioritized and selected by the participants. Forum discussions are protected by the Chatham House Rule (23), whereby participants are free to share information obtained at the meeting, but they may not reveal the identity or affiliation of the person who provided the information. Observing this rule creates an environment of trust and openness among the participants who attend the event year after year.

For this First HTA Policy Forum in LAC, the Institute for Clinical Effectiveness and Health Policy in Argentina (IECS, served as the scientific secretariat. The Organizing Committee, composed of representatives of participating institutions (both public and private), identified the discussion topic after consulting with Forum members about their preferences on a list of topics.

The topic selected for this first Forum was “Good practices in the application of health technology assessment to decision making in Latin America.”

The scientific secretariat prepared a background document to support discussions which summarized current knowledge on the topic. A literature search was conducted to identify documents that described principles and good practices in HTA in general terms. The search strategy included specific databases (MEDLINE, CRD, LILACS), internet search engines, relevant institutional websites, and expert consultation. Publications were selected that presented criteria for conducting HTA that could be interpreted as principles or good practices, or that made reference to their application in the world in general, or in LAC in particular. Reports and documents focusing on methodological issues were excluded. The background document was reviewed by participants before the event and their comments were incorporated into it.

The Forum was held in-person in San José, Costa Rica, on April 18–19, 2016. The keynote address was delivered by Sir Andrew Dillon, Executive Director of the National Institute for Health and Care Excellence (NICE) in the United Kingdom, and a series of presentations on the current state of the region were delivered by participants (HTA agencies, pharmaceutical and device industry, health system decision makers, and payers), which set the stage for discussions. Participants were divided into four breakout groups, each with a balanced distribution of the different stakeholders in attendance. On day 1, each group reviewed the international good practice principles and identified those of the highest priority upon which to focus efforts in LAC over the next few years. Their selections were based on the following criteria: (i) the potential for the principle to improve HTA processes in the region, and (ii) the presence of significant gaps between the ideal situation and the degree of current implementation of the principle. Participants then identified which of the prioritized principles would require further discussion/adaptation before being applied in the region.

On day 2, participants discussed the good practice principles that were prioritized the previous day. They analyzed experiences with the implementation of these principles and the challenges and opportunities encountered in promoting their uptake, discussed the requirements and conditions needed in the health system to support their implementation, and developed general recommendations to promote good practices in the region.

On both days, the breakout groups were followed by plenary sessions where their results were presented and discussed.

After the Forum, the scientific secretariat developed a summary report of the activities, results, and conclusions. Forum participants provided comments and suggestions on the report and these were incorporated into the final version.


The Forum comprised forty-two participants: eleven representatives of HTA agencies; seven representatives of funders from the public, social security, and private sectors; one representative of PAHO; seven academics and members of the scientific secretariat; and sixteen representatives from industry (pharmaceuticals, medical equipment, and diagnostic testing). In total, ten countries in the region were represented. Table 1 summarizes the main characteristics of the health systems of these countries.

Table 1. Summary of Characteristics of Participating Countries

Note. Source: Data from the World Bank, 2014, available at and a survey completed by attendees.

GNP, gross national product; HTA, health technology assessment; USD, United States dollars.


At the Forum, country representatives presented the drivers of HTA development in their countries and the main challenges currently faced. The heterogeneity of methods used in decision making and the inequalities and variability in access were the most frequently identified reasons for fostering HTA development and centralized decision making. Another driver mentioned was the need to increase the legitimacy of the decision-making process in health resource allocation. Among the most frequently encountered challenges were the lack of sufficient resources or technical capacity as well as limited awareness and knowledge of the subject among society in general.

From the pharmaceutical industry perspective, difficulties were described relating to the presence of many different HTA processes, decision rules, cost-effectiveness thresholds, and information requirements among the various countries. They viewed the main challenges in the incorporation of the patient voice in decision-making processes and the need to advocate for greater transparency in the decision-making process.

From the perspective of the device industry, it was highlighted that medical devices do not often undergo the HTA process and it is not unusual for much of the evidence to emerge after adoption, which is different from what happens with medicines. Challenges described for a greater leadership role of HTA, were the lack of communication between those who assess technologies and those who make purchasing decisions, and procurement strategies based on price and not the value of the technology. The emergence of hospital-based HTA initiatives was considered as an opportunity.


Through the search strategy described, eight publications were identified and selected (Table 2) that describe more than fifty good practice principles for the development and use of HTA and these served as a starting point for breakout group discussions and subsequent plenary discussions.

Table 2. Documents Selected That Contain Principles Considered Good Practices to Guide the Development and Use of HTA

HTA, health technology assessment; WHO, World Health Organization.

Even with the heterogeneity of HTA development among countries in the region and their different health system contexts, five principles were identified by the breakout groups as important and with the greatest potential to be applied or strengthened in the region in the short term (Table 3).

Table 3. Principles Identified as Priority for Application in Latin America and the Caribbean

HTA, health technology assessment.

The main points discussed relative to these five principles are described below:

1. Transparency in the Production of HTA and in Communicating HTA Results

All breakout groups agreed that the HTA process should be transparent and free from bias. There was general agreement to consider a public consultation process through which assessment and/or decision documents are published in different stages of development and to invite feedback from the general public or other stakeholders such as health professionals or technology producers. This would be a relatively simple and accessible measure for agencies to improve in this area. Brazil and Colombia were pointed to as examples of successful implementation of mechanisms to increase transparency.

2. Involvement of Relevant Stakeholders in the HTA Process

Some of the groups believed that this principle should be implemented immediately because it is seen as essential to legitimize the HTA and decision-making processes and thereby reduce the risk of conflicts and/or judicial appeals. Moreover, the involvement of stakeholders (including patients, users, health professionals, decision makers, and other interested groups such as industry) is important from the very beginning of the HTA process. However, other participants believed that this principle was not such a high priority because legitimacy can be achieved by other means, for example, through the involvement of scientific societies.

Furthermore, it was noted that stakeholder involvement can be complex, require fit-for-purpose methods, and can expose the HTA process to undesirable influences (for example, by patient groups financed by vested interests). It could also represent an excessive workload and cause delays to HTA processes thereby impeding a timely response to decision makers. Brazil and Colombia were once again mentioned as examples of successful implementation of mechanisms to involve different stakeholders.

3. Existence of Mechanisms for Appeal

These mechanisms allow different stakeholders to appeal the results of decisions made as part of the HTA process (for example, about reimbursement or inclusion of a specific intervention in a coverage package). Some participants agreed that this is not a complex matter and should be implemented without delay because in many countries it should already be part of every administrative act. However, other participants viewed this matter as complex and possibly a negative influence on the process by leaving it exposed to unfounded appeals and weakening the recommendations of the Ministry or agency.

4. Existence of Clear Mechanisms for the Prioritization of Topics for Evaluation by HTA

On this point as well, there were different views. The option to permit industry to solicit assessments, as in Brazil or Mexico, was mentioned; however, some participants felt that due to the limited resources available to conduct assessments this is not an option in their country and that requests for assessment must be limited to only those coming from public entities. Some described the importance for agencies to have clearly defined priorities for their HTA program according to what is important to their country, which would prevent the program agenda from becoming driven by the needs of industry and their requests for assessment, or by contingencies dictated by political demands. Conversely, other participants recommended that there be no explicit prioritization process, pointing out that the priorities would quickly become obsolete given that the agency must inevitably adapt their agenda to different pressures, for example, to update the benefits plan or to address issues that arise from political changes.

5. Existence of Clear Links between the Assessment and Decision Making

This was the principle where the greatest variation among countries was observed. The link between the HTA report and decision making is not often clear in the health system. Having an appropriate legal framework that defines the role of the HTA agency was recognized as a decisive factor in determining the degree of this linkage, although not all participants agreed that increasing this connection would always be desirable and/or possible in their health system. A limiting factor can be seen in cases where assessments are centralized in an HTA agency and where decision making occurs at different levels, such as in a fragmented health system. Brazil and Mexico were mentioned as examples where the role of HTA reports in the decision-making process is more clearly established due to the presence of a specific regulatory framework.

During the breakout group and plenary discussions, recurrent themes emerged around the lack of resources (particularly technical resources, time, and budget availability) for the proper application of these principles. These were especially relevant to the principles of transparency and stakeholder involvement as they would require the greatest investment of resources. The possibility was mentioned to obtain funding from industry sources, as is done in European countries where technology producers pay the agency for the assessment; however, several participants expressed concerns with the significant challenges in the region regarding the management of conflicts of interest. The pressures from different groups and the presence of conflicts of interest were said to be fundamental challenges to the application of some of the prioritized principles, for example, stakeholder involvement would add to the time and resources that the application of these principles would consume. One option proposed was to foster regional initiatives to coordinate the resources and efforts of different countries so as to share the workload required to implement these principles.

In discussion, several participants agreed that there are ranges of possible interpretations, foci, and degrees of depth to each of the principles, and, therefore, a gradual progression in their adoption is appropriate, without the expectation to apply a “NICE standard” from the outset.


This study presents the results of the first forum of this type to be held in the region that brought assessors, payers, and technology producers together in a common space to discuss themes of strategic interest in HTA.

The main conclusion of the Forum, as agreed by the majority of the participants, is that despite the significant advances in HTA in the region in recent years there remain many opportunities for improvement, particularly in relation to the use of HTA in the decision-making process.

There was agreement that the good practice principles proposed at the international level (614) are, in general, valid and potentially applicable in LAC and they could prove to be a useful to guide the establishment, expansion, or improvement of HTA processes in the region. Nevertheless, most principles would need to be adapted to the local context and the decision about which one(s) to implement, and to what extent, depends upon the state of HTA development in each country, the resources available, and the characteristics of the health system and decision-making process. In turn, important steps have been taken in the region (5) where some countries have already commenced the integration of many of these principles in their HTA process, and these cases can serve as guides for the other countries.

While significant effort was made to include a broad range of stakeholders in the Forum, the countries and health systems in the region are very diverse and not all were represented. This is one limitation to the generalizability of the findings. Nevertheless, issues discussed at the Forum about how to prioritize the principles and how they should be adapted to the local context aligns with similar issues identified at the global level. A recent publication by the WHO (24) describes the difficulty and uncertainty encountered in determining the best way to implement HTA in health systems and for the use of HTA in decision making.

The way in which HTA agencies apply these good practice principles also varies around the globe. Neumann et al. show significant variability in the degree to which a series of principles was accepted and implemented in fourteen HTA organizations, particularly in the areas of transparency and the link between HTA results and decision making (25). A study conducted in 2010 of health decision makers in LAC (26) found that the international principles were also considered valid and relevant in the region and, in line with the results of this Forum, showed that the most significant gaps existed in principles relating to the use of HTA in decision making.

The main challenges to HTA initiatives identified by country representatives at the Forum include the difficulty with involving different stakeholders in the process; the lack of resources to develop and sustain technical capacity; and, insufficient dissemination and links between HTA results and decision making. The key issue identified was how to be efficient in this process so as to strike an acceptable balance between achieving the HTA improvements desired and the available resources in the country (including staff, budget, and time), to ensure, above all, that the improvements introduced do not interfere with report development timelines or responsiveness to decision makers’ needs. These cautions about the dilemmas and potential problems that agencies face when aiming to apply or strengthen the international good practice principles matches up with similar criticisms and cautions conveyed by the global HTA community (1520).

Participants agreed that the next steps should be to promote the acceptance of the prioritized criteria. This will bring greater legitimacy to the difficult decisions that health systems must make on the road to universal health coverage. The main recommendation was to find appropriate HTA processes and/or methodologies, adapted to the context of each country that would allow for gradual improvements to the links between HTA and decision making, without trying to impose, in the short-term, excessively high standards taken from examples at the international level without appropriate adaptation of these to the local context.


None declared for the authors.


1.World Health Organization (WHO). The world health report. Health systems financing: The path to universal coverage. Geneva: World Health Organization; 2010.
2.Pan American Health Organization (PAHO). Resolution - CSP28.R9. Health technology assessment and incorporation into health systems 2012. (accessed May 2, 2018).
3.Sixty-seventh World Health Assembly. WHA67.23 - Health Intervention and Technology Assessment in Support of Universal Health Coverage. WHA Resolution. 2014.
4.Wilsdon, T, Serota, A. A comparative analysis of the role and impact of health technology assessment. London: Charles River Associates; 2011. (accessed May 2, 2018).
5.Augustovski, F, Alcaraz, A, Caporale, J, Garcia Marti, S, Pichon Riviere, A. Institutionalizing health technology assessment for priority setting and health policy in Latin America: From regional endeavors to national experiences. Expert Rev Pharmacoecon Outcomes Res. 2015;15:9-12. doi:10.1586/14737167.2014.963560.
6.Daniels, N. Accountability for reasonableness. BMJ. 2000;321:1300-1301.
7.Daniels, N. Justice, health, and healthcare. Am J Bioeth. 2001;1:2-16. doi:10.1162/152651601300168834.
8.Siebert, M, Clauss, LC, Carlisle, M, et al. Health technology assessment for medical devices in Europe. Int J Technol Assess Health Care. 2002;18:733-740. doi:doi:null.
9.EFPIA, editor. Position paper: The use of health technology assessments (HTA) to evaluate medicines - Key principles. Brussels: EFPIA; 2005.
10.Emanuel, EJ, Fuchs, VR, Garber, AM. Essential elements of a technology and outcomes assessment initiative. JAMA. 2007;298:1323-1325. doi:10.1001/jama.298.11.1323.
11.Drummond, MF, Schwartz, JS, Jonsson, B, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24:244-258; discussion 362-368. doi:10.1017/S0266462308080343.
12.Sorenson, C, Drummond, M, Borlum, F, Busse, R. How can the impact of health technology assessments be enhanced? Copenhagen, Dinamarca: World Health Organization; 2008.
13.HLPF. High Level Pharmaceutical Forum - 2005–2008 - Final Report: HLPF; 2008.
14.World Health Organization (WHO). Making fair choices on the path to universal health coverage 2014. (accessed May 2, 2018).
15.Sampietro-Colom, L. Consider context and stakeholders. Int J Technol Assess Health Care. 2012;28:166-167. doi:10.1017/S0266462312000153.
16.Henshall, C. Describe decision-making systems, assess health technology assessment reports. Int J Technol Assess Health Care. 2012;28:168. doi:10.1017/S0266462312000177.
17.Goodman, C. Toward international good practices in health technology assessment. Int J Technol Assess Health Care. 2012;28:169-170. doi:10.1017/S0266462312000189.
18.Banta, HD. Commentary on the article “Key principles for the improved conduct of health technology assessments for resource allocation decisions”. Int J Technol Assess Health Care. 2008;24:362-365.
19.Hailey, D. Commentary on the article “Key principles for the improved conduct of health technology assessments for resource allocation decisions”. Int J Technol Assess Health Care. 2008;24:365-366.
20.Neuhauser, D. Commentary on the article “Key principles for the improved conduct of health technology assessments for resource allocation decisions”. Int J Technol Assess Health Care. 2008;24:366-367.
21.HTAi - Health Technology Assessment International: HTAi; 2015 [Septiembre 2016]. (accessed May 2, 2018).
22.Office of Health Economics (OHE), Health Technology Assessment International (HTAi). HTA and Decision Making in Asia: How can the available resources be used most effectively to deliver high quality HTA that can be used by health system decision makers? HTA and decision making in Asia: Asia Policy Forum Style Meeting; Seoul, South Korea; 2013.
23.Chatham House Rule Londres, Reino Unido: Chatham House - The Royal Institute of International Affairs; 2016 [Septiembre 2016]. (accessed May 2, 2018).
24.World Health Organization (WHO). Using Health Technology Assessment for Universal Health Coverage and Reimbursement Systems. Geneva: WHO; 2015. (accessed May 2, 2018).
25.Neumann, PJ, Drummond, MF, Jonsson, B, et al. Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations? Int J Technol Assess Health Care. 2010;26:71-78. doi:10.1017/S0266462309990833.
26.Pichon-Riviere, A, Augustovski, F, Rubinstein, A, et al. Health technology assessment for resource allocation decisions: Are key principles relevant for Latin America? Int J Technol Assess Health Care. 2010;26:421-427. doi:10.1017/S0266462310001042.