Background
The European Network for Health Technology Assessment (EUnetHTA) is a collaboration between network members and external partners that promotes the health technology assessment (HTA) of approved drugs, with the goal of driving efficient HTA collaboration (Reference Kristensen, Lampe and Chase1). HTA is a multidisciplinary process that uses explicit methods to determine the value of a specific health technology at different points in its life cycle (Reference Orourke, Oortwijn and Schuller2). Since its inception in 2006, the EUnetHTA has carried out three multiannual joint actions (JAs) to develop HTA collaboration.
Among its many projects, the initiative members created the HTA Core Model® as a fundamental methodological framework for producing and sharing HTA information (Reference Lampe, Mäkelä and Garrido3). The model can be divided into clinical and nonclinical domains (Reference Lampe, Mäkelä and Garrido3). Clinical domains include relative clinical effectiveness, defined as the extent to which a drug does more good than harm compared to one or more alternative interventions in achieving the desired results when provided under typical healthcare practice (Reference Lampe, Mäkelä and Garrido3). Relative safety, the current technology, and the technical characteristics of the examined drug are also part of the model’s clinical elements (Reference Lampe, Mäkelä and Garrido3). The EUnetHTA’s collaboration in JA 3 (2016–2021) focused on these clinical aspects, which led to horizon scanning (HS) activities that identify, select, and prioritize drugs for rapid relative effectiveness assessments (REAs) (4). In addition, in a candidate drug’s pre-licensing phase, the initiative conducted early dialogues (EDs) to enable confidential exchange between the industry, multiple HTA bodies (HTAbs), and, where applicable, the European Medicines Agency (EMA) to allow for the integration of HTA requirements into study designs and the generation of evidence (4). Furthermore, the EUnetHTA made early assessments of drugs’ clinical evidence compared to the standard of care by performing rapid REAs as joint clinical assessments (JCAs) through an examination of all clinical domains in the HTA Core Model® (Reference Lampe, Mäkelä and Garrido3;4). Scattered pilot projects for specific products or registries for additional data and evidence collection were also carried out in the post-licensing phase to complement the evidence already generated (4). Furthermore, the EUnetHTA developed HTA-related templates and guidelines to strengthen the applicability of the initiative’s consolidated expertise (Reference Kristensen, Lampe and Chase1).
In contrast, the nonclinical domains of the HTA Core Model® include costs and economic evaluation, ethical analysis, organizational aspects, and social or legal affairs (Reference Lampe, Mäkelä and Garrido3). Although EUnetHTA-level cooperation among HTAbs has focused on the clinical domains of the HTA Core Model®, there are some other forms of collaboration on cost and economic evaluation between individual states. Participating countries in these initiatives are regionally close and use similar economic evaluation approaches. Specifically, the Northern Europe-based initiative between Finland, Norway, and Sweden, FINOSE, has started to perform several joint incremental cost-effectiveness ratio calculations to inform its collective price negotiations (5). The collaboration between Belgium, the Netherlands, Luxemburg, Austria, and Ireland, called BeNeLuxA, has likewise completed joint economic evaluations (6). However, recorded perceptions of collaboration in economic areas are rare.
Oncology faces a severe burden of disease and increasing challenges in the effective generation of evidence (Reference Gill, Fontrier, Miracolo and Kanavos7). Case-study-based policy recommendations demand collaboration among HTAbs, regulators, and patient organizations to make well-informed and faster decisions to assess and approve oncological innovations (Reference Gill, Fontrier, Miracolo and Kanavos7). The EUnetHTA has already included assessments for oncology in its activities. During JA 3, for instance, it conducted multiple EDs and five rapid REAs on this topic (8). It also performed two pilots for postlaunch evidence generation (PLEG) in oncology, one product- and one registry-specific (9).
In 2022, a new EU legal HTA framework (10) came into force, based on projects conducted during the EUnetHTA’s three JAs. The framework commits to a defined extent of cooperation in the clinical part of HTA, with its implementation set to follow a multi-stage procedure starting in 2025 (10). Oncology drugs will also play an important role from the beginning of the new framework’s implementation, as along with medicinal products for advanced therapy, they will be the first products with mandatory JCAs, as of 2025 (10).
This study aimed to provide expert insights into HTA cooperation in clinical and nonclinical domains and in oncology.
Methods
Semi-structured interviews with experts of HTA in the oncology field were conducted to garner professional views of the strengths and challenges in the EUnetHTA’s collaboration. Nine questions were asked that generally referred to the strengths and challenges in clinically oriented HTA collaboration in oncology across the technology life cycle, along with opportunities for future economic collaboration. Each researcher approved the questions, with their final forms depicted in Table 1.
Table 1. Interview questions
a On a scale of 1 (full refusal of the statement) to 5 (full agreement with the statement).
ED, early dialogue; EUnetHTA, European Network for Health Technology Assessment; HTA, health technology assessment; HS, horizon scanning; JA 3, Joint Action 3; PLEG, postlaunch evidence generation; rapid REA, rapid relative effectiveness assessment.
In total, 18 interviewees were selected. The experts were mainly identified through the lists of participants from EUnetHTA meetings or participants in the virtual EUnetHTA Forum in April 2021. Contacts provided by an internship organization and the researchers’ own networks led to further detection of participants. The experts first received an invitation letter via email, outlining the study’s background and scope. Before the interview, their written informed consent was obtained for collecting data including audio recordings, transcriptions, and pseudonymized data analysis. The interviews were conducted with Microsoft Teams in May and June 2021 and transcribed thereafter. Then, the resulting documents were sent back to the experts for editing and confirmation of their statements. Subsequently, the transcripts were evaluated using Mayring’s qualitative content analysis (Reference Mayring11). Main categories were derived from the questions’ topics, and subcategories emerged inductively from the answers.
Ethical approval was obtained from the ethics officers of Maastricht University, and the research was classified as a low-risk project (FHML/HPIM/2021.036). At the same time, the confidentiality of the interview data and personal anonymity of the respondents were ensured.
Results
The interviewees were categorized into four groups. The first group consisted of current or former senior board members of the EUnetHTA initiative (n = 2). The second group (n = 6) included representatives from HTAbs affiliated variously with the Netherlands, Germany, Finland, Sweden, Denmark, and Norway. The third group contained experts from the pharmaceutical industry (n = 2). Last, the mixed fourth group consisted of academic experts or consultants (n = 5), representatives from patient organizations (n = 2), and an expert from a regulatory agency (n = 1).
The following sections present the narrative of the main interview findings, while the number and the type of stakeholder behind it indicate how many and which participants in total have made a certain statement. Table 2 provides the full results.
Table 2. Full interview findings
a Alternatively, the mean or standard deviation of the evaluation of the Likert Scale question.
A, academia; AV, added value; BeNeLuxA, HTA collaboration between Belgium, the Netherlands, Luxemburg, Austria, and Ireland; CEA, cost-effectiveness analysis; CUA, cost-utility analysis; ED, early dialogue; EMA, European Medicines Agency; EU, European Union; EUnetHTA, European Network for Health Technology Assessment; EUn, EUnetHTA senior board; HS, horizon scanning; HTA, health technology assessment; HTAb, HTA body; JA 3, Joint Action 3; JCA, joint clinical assessment; M, mean; P, pharmaceutical industry; PAES, post-authorization efficacy study; PASS, post-authorization safety study; PICO, population, intervention, comparison, outcome; PLEG, postlaunch evidence generation; PO, patient organization; R, regulator; rapid REA, rapid relative effectiveness assessment; RCT, randomized controlled trial; SD, standard deviation.
The strengths and challenges of the EUnetHTA’s JA 3
On the one hand, the EUnetHTA’s JA 3 initiative consolidated the foundation of information exchange, consequently allowing for a common view of HTA, in particular with regard to the requirements for the clinical effectiveness of a drug (8: A, EUn, HTAb, P, PO). In addition, the guidelines and reports produced were of high quality (7: A, EUn, HTAb, PO, R). On the other hand, on average, stakeholders perceived the outcome of JA 3 as neutral. Specifically, the initiative’s challenges often featured multi-state navigating through decision-making to agree on the scope of collaboration (7: A, EUn HTAb, P, PO, R). Moreover, the legislative legitimacy of the EUnetHTA’s projects was not always provided (7: A, EUn, HTAb, P, PO, R), leading to uncertainty among stakeholders about the initiative’s future.
The clinically oriented HTA collaboration during JA 3
This section looks at collaboration during JA 3 more broadly. Specific areas of collaboration included in the interviews are HS, EDs, rapid REAs, and PLEG. HS should lead to a central overview of all forthcoming approved drugs and the joint prioritization of those drugs which are to be assessed during rapid REAs. Even though complete related lists or rigorous decision structures such as who should perform the joint prioritization were not in place (4: HTAb), the experts valued the EUnetHTA’s HS pilot project as an important first step toward identifying the critical elements for successful HS activities (5: A, HTAb, P, R). It was challenging to foresee all potentially available drugs, as the timing between HS activities and the provision of relevant information from the EMA were not aligned from the beginning (1: HTAb). The challenges in the EUnetHTA’s legislative legitimacy (7: R, PO, A, HTAb, P, EUn) might have led to EMA’s prohibition against providing relevant information to them. Based on the HS decisions, EUnetHTA stakeholders asked manufacturers if their drug could be subjected to rapid REA. In individual cases, some companies’ refusal made it necessary to adjust the process (1: HTAb).
EDs also allowed for the development of cross-stakeholder positions and opened the possibility to jointly foresee potential gaps in evidence that could occur in the national assessments of new modes of action (8: A, EUn, HTAb, P, R). However, the non-binding character and time gap between an ED and the actual assessment reduced their applicability to industry (2: P). In addition, constraints on resources and the resulting need to select which candidates could undergo EDs posed a challenge (2: HTAb, P). The consistent impact of patients’ involvement on the EDs’ outcomes was questionable (2: PO), as, for example, sharing ED-related information with patients and their inclusion in the EDs’ meetings were not consistently ensured (1: PO). Regarding the overall context of EDs, an interviewee from the EUnetHTA proposed that “EDs should be the starting point for an overall evidence generation plan, including the different needs of the HTAbs and thus going beyond the design of […] phase three clinical trial.”
The rapid REAs formed the core of the cooperation. There were only a limited number of rapid REAs, but they were of high quality and allowed for efficient contextualization to national requirements (7: A, EUn, HTAb). However, the authoring HTAbs were confronted with methodological challenges during the development process, including differences in standards of care, the handling of comparators and endpoints, and the inclusion of evidence from a non-randomized controlled trial (non-RCT) (10: A, HTAb, P, PO, R). Some participants also stated that the adoption of the reports varied between the drugs and HTAbs (3: A, EUn), and viewed the timing of the final report’s availability as a challenge, especially for countries that need to assess each drug within a specific timeframe after market approval (3: A, HTAb). Specifically, the pre-assessment’s scoping phase and review rounds were named as drivers of the duration of the process (2: HTAb). During the scoping phase, each stakeholder participating in the EUnetHTA was allowed to provide input (including population, intervention, comparison, and outcome) for a specific framework before the authoring HTAbs conducted their reports. Moreover, the constellations of HTA authors varied, leading to a diversity of views and exchange but also to process inefficiencies (1: HTAb). One stakeholder from the patient organizations missed an integrated framework that shows the specific impact of the input from patients on the reports.
Several participants indicated further that collaboration on PLEG should be expanded (5: A, EUn, HTAb). Specific hurdles included differences in resources, data infrastructure, and accessibility, as well as varying levels of experience with the synthesis of evidence (2: EUn, R). Data sharing from more experienced countries could support further development, while ways to manage confidentiality need to be initiated (1: HTAb). Despite these challenges, and in coordination with the existing legitimacy and experience of the regulator to impose the generation of new evidence, the PLEG requirements between the regulator and HTAbs should harmonize (1: A).
The future of HTA
The participants expect to see more single-arm trials with smaller subpopulations and surrogate endpoints in the future, leading increasingly to conditional or exceptional marketing approvals (8: A, EUn, HTAb, P, PO, R). Scientific progress manifested in personalized medicines targeting specific genetic expressions will provide potentially long-standing treatments or cures. It can be challenging, using current HTA methodologies, to assess these innovative medicines – for instance, to measure overall survival. As a result, for some stakeholders, there is a growing need for cooperation in clinically oriented HTA for developing new methodologies (7: EUn, HTAb, P, PO). Moreover, joint PLEG activities should be developed further. However, some of the experts’ expectations for future HTA collaboration, such as rapid REA reports and increasing permanent collaboration, will be addressed through the Regulation of the European Parliament and the Council on Health Technology Assessment (9: EUn, HTAb, P, PO) (10).
In the economic domains of HTA (Reference Lampe, Mäkelä and Garrido3), existing differences, including economic contexts and the costs or resources of the medical sector, are barriers to collaboration (11: A, EUn, HTAb, P, PO, R). Nevertheless, the stakeholders proposed some areas of voluntary collaboration. Within HTA, these include the publication of nonclinical reports (2: HTAb, PO), and joint activities related to instruments for measuring quality-adjusted life years or disability-adjusted life years (3: HTAb). The BeNeLuxA initiative members already share information regarding nonclinical elements (1: A). The context of voluntary yet clinically oriented EDs also has potential for systematizing nonclinical collaboration.
Discussion
This study provides insights into past and future HTA cooperation. It suggests that different stakeholders perceive the EUnetHTA’s various collaboration activities as valuable. However, on average, stakeholders rate the outcome of JA 3 as neutral. The participants cite challenges in HTA cooperation until 2021 and suggest possibilities for future collaboration. Joint work in HS systems and PLEG is still in the early pilot phase, but has the potential to mitigate uncertainty in the emergence of new technologies or gaps in evidence. EDs also help anticipate potential uncertainty, especially regarding a drug’s clinical effectiveness, and allow for international consensus on study design requirements. According to the stakeholders, until 2021 the main barriers to HTAbs’ adoption of rapid REAs included an insufficient number of relevant reports, different perspectives on comparators and endpoints, the inclusion of non-RCT evidence, and timing variances. A joint overall plan to generate evidence across the technology life cycle is proposed for clinically oriented HTA collaboration. The participants regard voluntary joint work in nonclinical HTA as challenging, but provide existing isolated approaches in countries with similar methods.
These findings are relevant for policymaking, especially as, to the authors’ knowledge, this is the first study that includes multi-stakeholder views on collaboration in clinical, nonclinical, and oncology domains. The coordination group and its subgroups can use these results when preparing to implement the HTA regulation (10). The coordination group consists of delegated members of the European member states. It provides the strategic direction for the work of its subgroups. The subgroups are composed of national or regional authorities and work on specific collaborative topics (10). First, under the new regulation, EUnetHTA EDs, referred to in the future as joint scientific consultations (JSCs) (10), are performed only for selected drugs. Applying JSCs to all potentially innovative drugs would strengthen HTA collaboration. The EUnetHTA also recommends a life cycle approach by sharing information from the JSC assessment teams, while remaining evidence gaps identified in assessments should be referred to PLEG activities (4). The interviewees valued PLEG’s opportunity to mitigate the uncertainties related to increasingly conditional or exceptional marketing approval, which were also highlighted by Moseley et al. (Reference Moseley, Vamvakas and Berntgen12). The HTA regulations contain a reference to voluntary cooperation in real-world evidence and to supporting the further development of related databases and registries (10). It will be of interest for all stakeholders to monitor the options for joint PLEG that will arise at the European level. Second, sufficient capacity must be made available to create the JCAs so that countries can meet their national timelines. These reports should include a description of the relative effectiveness and analysis of scientific uncertainties, but no binding conclusion (10). Thus, third, the reports’ content and structure will allow for practical contextualization to account for the different evidence requirements described in this and other studies (Reference Chassagnol, Marcelli and Wagle13;Reference Giuliani, Chassagnol and Traub14). Overall, the transparency and documentation of all HTA-related, non-confidential information at the EU level to countries and vice versa will be essential to further implement and develop collaboration.
This study also provides insights to plan optional collaboration in nonclinical domains. The participating experts perceived boundaries as strong, mainly due to differences in national methodologies. However, there are European methodological recommendations on economic evaluation (Reference Heintz, Gerber-Grote and Ghabri15;Reference Wang, Qiu, Zhou, Francois and Toumi16). The experts also mentioned jointly collecting data or developing instruments to measure quality-adjusted life years or disability-adjusted life years. The instruments required to measure health status, such as the preference-based EQ-5D questionnaire, are present across Europe (17).
According to this research, patient organizations do not yet feel that the patient’s perspective is systematically involved in HTA collaboration. At the same time, there are already frameworks outlining relevant criteria for their involvement (Reference Badia, Aguarón and Fernández18;19). A further study also claimed that patient involvement should be further strengthened (Reference Garrett, Imaz-Iglesia and Willemsen20). Ways toward the more impactful and systematic involvement of patient organizations in HTA could be a subject for further investigations.
There are some limitations to this study. To start, despite the initial focus on oncology, the results are mostly not oncology-specific. To obtain results that are truly oncology-focused, it may have been necessary to include examples of oncology drugs in the interviews. Still, the study’s general conclusions are of broad relevance and suggest that procedural and methodological challenges to HTA collaboration do not necessarily differ between therapeutic areas. Another study limitation is that the insights generated in the eighteen interviews do not create a representative picture, especially for stakeholders other than HTAbs. Moreover, the HTAbs came from Northern European countries and further research would be needed with other agencies participating in the EUnetHTA to mitigate potential selection bias. Given time constraints, one researcher did the data analysis, whereas several researchers could have better underlined and differentiated the findings.
Conclusion
This study delivers present and future-oriented insights into European HTA cooperation. The stakeholders involved support clinically oriented collaboration across the technology life cycle to cope with the uncertainty of relative effects between drugs. They also indicate that forthcoming collaboration on HTA must allow for practical content- and timing-related contextualization of the drug’s relative effectiveness, as differences remain in the requirements for evidence. There is potential for future collaboration in the PLEG field, but also severe challenges. Further, patients still need to be involved more systematically in European HTA. In the nonclinical area of HTA, countries using cost-effectiveness or cost-utility analysis could cooperate with each other more closely.
Funding statement
This research received no specific grant from any funding agency, nor from any person or organization in the commercial or non-profit sectors.
Competing interest
This research was conducted as part of an internship at UCB Pharma Ltd. This had no influence on its results.
Open access
