The Scottish Health Technologies Group (SHTG) provides evidence support and advice to the National Health Service in Scotland on the use of new and existing health technologies, which, although not medicines, are likely to have significant implications for people's care. The purpose of this paper is to highlight the developments that have taken place in the SHTG's patient involvement processes in the years 2017 to 2019, focusing primarily on specific engagement with patient organizations and considering how the new approaches have been received by stakeholders.

Feedback from patient organizations that participated in the SHTG submission process, alongside SHTG committee members’ views on patient organizations contributions, was gathered primarily via online questionnaires. The number of times that patient organizations were invited and accepted the opportunity to peer-review SHTG advice statements prior to and after the employment of a Public Involvement Advisor (PIA) was analyzed.

Completed questionnaires (n = 4) from three case study examples showed high patient organization satisfaction with their experience of the SHTG process. The feedback from SHTG committee members that was gathered indicated that patient organization participation was generally well received. The number of peer reviews from patient organizations for SHTG advice statements in 2018–2019 doubled to 86 percent of the total advice statements (n = 22), compared with 43 percent (n = 14) in 2016–2017.

Significant progress has been made toward improving the SHTG's patient involvement processes. A dedicated PIA post within the SHTG has allowed for a more tailored support to patient organizations and has encouraged their increased participation in SHTG processes.

In 2017, a Patient Involvement Interest Group (PIIG) was created in the Spanish Network for Health Technology Assessment of the National Health System (RedETS) to facilitate and promote Patient Involvement (PI) in Health Technology Assessment (HTA). The PIIG proposed a decisional flowchart to guide researchers’ in decisions regarding PI methods in HTA. The flowchart proposed a combination of direct involvement and incorporation of patient-based evidence depending on the scope and the aims of the assessment.

This work aims to present the flowchart and the results of the evaluation of the latest experiences in PI in HTA in RedETS (2018–2020), including direct-involvement and patient-based evidence.

A survey was sent to the HTA researchers who implemented PI initiatives in RedETS assessments. The survey asked to describe their experiences, lessons learned, challenges and added value regarding the use of direct-involvement, systematic reviews (SR) and primary studies. A descriptive analysis was performed and the results were discussed in an online PIIG workshop.

Thirty-two assessments included direct PI, twenty-one SR synthesized qualitative and quantitative studies about patient experiences, values and preferences and eight included primary studies, mainly of qualitative design. Recruitment and the lack of methodological resources were the main barriers both for direct PI and primary studies. Relevance of the included studies was the main barrier for SR. Added value was found in all PI methods. Direct-involvement had an impact on the project plan and PICO definition, outcomes relevance, information about the health condition and treatments. SR contributed with relevant patient-based evidence, deeper assessment of patient experiences, values and preferences and implementation factors. Primary studies developed new or contextualized knowledge directly applicable to decision-making.

The PI flowchart has served to facilitate the incorporation of patient input in HTA reports. The different approaches implemented have allowed to provide relevant and well-grounded data in each report to inform decision-making in patient-centered healthcare provision, but it is necessary that specific training and resources are provided to enable adequate and timely implementation.

The European Health Technology Assessment Network (EUnetHTA) Work Package 5B1, is focused on testing the levels of cross-border collaboration on real world data for supporting reimbursement/pricing decision-making. Within this Work Package, we are conducting a pilot on Left Ventricular Assist Device on destination therapy in collaboration with the National Institute for Health and Care Excellence (NICE, UK), the Belgian Health Care Knowledge Centre (KCE, Belgium) and the Italian National Agency for Regional Health Services (AGENAS, Italy). This pilot aims to define the minimum data set for gathering and sharing high quality registry data on key uncertainties found at the time of the health technology assessment (HTA). Furthermore, the pilot will assess the feasibility of carrying out a common analysis or reusing this data for National or Joint Reassessments.

Evidence gaps were based on the four national assessments. Collaborating partners were responsible for agreeing on the key outcomes and proposing the minimum dataset to be registered. European clinical experts and patients rated and prioritized the dataset using a two round Delphi technique (not relevant, important but not critical; critical). The dataset will confirm the basis for the Spanish LVAD registry, implemented at the national health service level to inform inclusion into the healthcare portfolio.

The key outcomes agreed upon by agencies relate to safety, effectiveness, satisfaction and acceptability of the patient and cost-effectiveness, budget impact and organizational impact. Expert cardiologists and cardiac surgeons representing the European and Spanish Society, among others, participated in the prioritization of basic data. The final dataset is expected by December 2020.

The variation in the quality and definition of outcome measures for measuring key evidence gaps reduces the utility of registries for HTA, making it difficult to compare, link, and aggregate data across countries. The EUnetHTA pilot is intended to offer a model for cross-border collaboration on real world data for supporting the decision-making process for pricing and reimbursement.

The COVID-19 pandemic shows that the impact of effective vaccines extends well beyond vaccinated individuals and healthcare systems. Yet, these externalities are not typically considered in health technology assessments (HTA) which may underestimate vaccines’ broader value. We explored to what extent future vaccines relevant to England might exhibit such broader value.

We compared the ten value elements of an existing vaccine evaluation framework to the value elements considered in England according to the Joint Committee on Vaccine and Immunisation (JCVI) and the National Institute for Health and Care Excellence's (NICE) guidelines. Using literature and expert opinion we then explored, for a selection of ten vaccines with an expected UK-launch within five years, on which value elements each vaccine might potentially show added value.

Up to five of ten value elements are unlikely to be considered by JCVI or NICE, including patient and carer productivity, enablement value, impact on antimicrobial resistance and transmission value. Of vaccines studied, 100 percent will potentially generate value on at least one broader value element that is currently ignored; 60 percent to 80 percent may increase vaccinee/patient or carer productivity respectively.

There is a substantial gap between value generation and value recognition of vaccines in HTA in England. This might lead to undervaluation and underutilization of vaccines, leaving societies more vulnerable than needed when faced with infectious diseases.

Healthcare services, such as cochlear implants and subsequent rehabilitation, aim to increase valuable activities and opportunities of those affected. Their impact may be inferred from the extent that they protect or restore capability, which reflects the real freedoms that people have to be or do things they have reason to value. Capability emerges from the dynamic interaction between available resources, individual, social, and environmental conversion factors, and functionings. This model sets the informational requirements of the capability approach.

On the basis of interviews with thirty-three hearing impaired children and thirty hearing peers, information on capability elements (values, resources, conversion factors, and functionings) was collected. Qualitative results were triangulated with standardized clinical audiological and psycholinguistic quantitative measures.

Hearing impaired children and their hearing peers concurred in terms of the doings and beings they valued, but differed in terms of conversion factors to realize capability. Parents of hearing impaired children played a more upfront role, hearing impairment predominated many areas of life, and communicating through hearing aids required more energy than was usually acknowledged by the people around them.

The capability approach offers opportunities not only to assess impact of technology on dimensions that are important to patients, but also to better understand the mechanisms that are involved in value generation.

Health Technology Wales (HTW) is committed to evaluating the impact of our work. In March 2020, HTW directed efforts to support Welsh Government and health and social care providers in response to the COVID-19 pandemic. We adapted the HTW evaluation framework to specifically capture the impact of our additional COVID-19 work. Here we analyze data collected since the framework was implemented.

Both formal and informal feedback was analyzed. Formal feedback was obtained through the HTW Impact Questionnaire, which was developed to support more formalized data capture for all HTW workstreams and to facilitate feedback from all stakeholder groups. It was piloted with a targeted list of individuals and responses were received for COVID-19 work. Informal feedback included feedback received via email or through word of mouth.

HTW COVID-19 products to date include Topic Exploration Reports, rapid evidence summaries and an Evidence Appraisal Report (EAR) on COVID-19 diagnostic tests (molecular and antibody tests). Stakeholders were positive about these outputs, describing them as valuable and informative. Reported impacts included informing policy and decision making, reducing duplication of efforts and helping to target development. The EAR received national and international focus, leading to HTW involvement in the European Network for Health Technology Assessment (EUnetHTA) COVID-19 reviews. Survey participants who gave feedback on COVID-19 activities included two members of Health Technology Assessment organizations, a health board representative and an industry representative; all agreed that HTW's COVID-19 work was useful, that the methods were reliable and robust and that HTW is responsive. All participants also felt that HTW's COVID-19 work had a positive impact in the wider health and social care context.

HTW was able to respond rapidly to the COVID-19 pandemic and adapt current evaluation practices to capture the impact of COVID-19 work. We will continue to evaluate our COVID-19 activities. Future work will involve following up on the developing impact of our COVID-19 work and expanding our methods for data capture, for example conducting stakeholder interviews.

The Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS) defined a patient involvement (PI) framework for health technology assessment (HTA) activities in 2016. The aim of this study is to evaluate the process and impact of those PI initiatives that were implemented in the first year following the publication of this new framework.

A survey was sent to those HTA researchers who implemented PI in RedETS projects. Responses were reviewed by two authors. An adapted thematic analysis was performed and the results were later discussed by all authors.

Six responses from six agencies/units were analyzed. The objectives of PI initiatives were the following: inclusion of patient perspectives, preferences and values; elicitation of important health outcomes measures; and barriers, facilitators, or suggestions for implementation. Different methods were used for PI: surveys, focus groups, in depth interviews, and participation in an expert panel. Five main themes emerged: (i) challenges with the recruitment process, (ii) needs identified, (iii) impact of PI, (iv) lessons learned, and (v) suggestions for the future.

PI initiatives within the RedETS framework were tailored to each HTA project, its specific goals and the individual needs and resources of each HTA agency. The results also pointed out how PI has a relevant impact that has enriched RedETS products providing key information on experiences, values, and preferences of patients, contributions that benefit the HTA and the process of drawing up recommendations. The main challenges were related to recruitment processes and capacity building.

The expansion of health data offers exciting opportunities to support better and more efficient drug discovery, development and implementation. Data protection and governance provide the legal framework to balance safeguarding patients’ privacy with the benefits to society of medical research. Our aim is to highlight current legal barriers to the better use of health data and propose ways to address them.

Analysis of the relevant legislative texts was supplemented by interviews with external experts in data protection, health research, informatics and cyber security and a workshop with pharmaceutical industry members. We investigated the legal issues arising for six key activities along the pharmaceutical lifecycle, from identifying unmet need through to health technology assessment and pharmacovigilance.

The General Data Protection Regulation (GDPR) was introduced in May 2018 to Harmonise data protection across Europe. However, considerable ambiguity remains, particularly around the appropriate legal bases for data processing in the absence of consent: scientific research, public interest, or provision of health or social care. Other key themes included data subject rights, anonymization, compatibility of primary and secondary (re-)use of data, heterogeneity arising from divergent interpretation, the need for guidance on digital health, and the importance of trust.

We speculate which legal bases are most appropriate for the six pharmaceutical activities studied, but clear guidance and consensus is required. The GDPR was not designed to hamper scientific research, and the issues identified arose from uncertainties rather than barriers per se. Industry and academic researchers should therefore deal proactively with the prevailing uncertainties, share good practice, and engender trust by co-creating a code of conduct and outlining principles of responsible use. Engagement with patients will be critical in encouraging a shared understanding of the value to society of health data for research.

Often health technology assessment (HTA) products developed by the Scottish Health Technologies Group (SHTG) did not reach clear directive conclusions because the evidence base for a technology was weak. Despite being methodologically robust, these products did not meet the needs of decision-makers and may have had negligible impact.

SHTG set out to equip and empower the recommendation-making council (that is, appraisal committee) to reach clear conclusions. SHTG broadened the HTA components and types of evidence that could be considered. The increased breadth of evidence included: clinicians attending council meetings to respond to questions; patient groups making submissions and presenting at council meetings; Scotland-specific economic modelling; and consultation on draft recommendations. SHTG also restructured the council for improved deliberative decision-making.

Clear directive conclusions were reached in a substantially higher proportion of HTA products (eighty-eight percent in 2019 compared with eight percent in 2017). It became possible for decision-makers to implement findings. It also became feasible to assess the impact and implementation of recommendations.

Broadening SHTG's consideration of HTA components has led to a clearer conclusion being reached and stronger messaging for decision makers. This positions SHTG to increase its influence in the use of health technologies in Scotland.

The frameworks used by Health Technology Assessment (HTA) agencies for value assessment of medicines aim to optimize healthcare resource allocation. However, they may not be effective at capturing the value of antimicrobial drugs.

To analyze stakeholder perceptions regarding how antimicrobials are assessed for value for reimbursement purposes and how the Australian HTA framework accommodates the unique attributes of antimicrobials in cost-effectiveness evaluation.

Eighteen individuals representing the pharmaceutical industry or policy-makers were interviewed. Interviews were transcribed verbatim, coded, and thematically analyzed.

Key emergent themes were that reimbursement decision-making should consider the antibiotic spectrum when assessing value, risk of shortages, the impact of procurement processes on low-priced comparators, and the need for methodological transparency when antimicrobials are incorporated into the economic evaluation of other treatments.

Participants agreed that the current HTA framework for antimicrobial value assessment is inadequate to properly inform funding decisions, as the contemporary definition of cost-effectiveness fails to explicitly incorporate the risk of future resistance. Policy-makers were uncertain about how to incorporate future resistance into economic evaluations without a systematic method to capture costs avoided due to good stewardship. Lacking financial reward for the benefits of narrower-spectrum antimicrobials, companies will likely focus on developing broad-spectrum agents with wider potential use. The perceived risks of shortages have influenced the funding of generic antimicrobials in Australia, with policy-makers suggesting a willingness to pay more for assured supply. Although antibiotics often underpin the effectiveness of other medicines, it is unclear how this is incorporated into economic models.

The Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU) is commissioned to assess ethical aspects in their health technology assessment (HTA) reports, in addition to effects and health economic aspects of the examined interventions. For this purpose, a framework for systematic evaluation of ethical aspects of healthcare technologies has been developed and used at SBU since 2014. With seven years of practice, we decided it was time to evaluate experiences from using the ethical framework and consider possible adjustments to improve future use.

SBU reports in the time period 2014–2020 were systematically screened for ethical content. Focus group meetings with users of the framework (mainly HTA project managers) were held where opinions regarding usability and possible obstacles were collected. A revised version of the document was sent for consultation to relevant stakeholders (possible users, reviewers and recipients) in order to collect additional views.

Of fifty-eight HTA reports produced in the time frame, ethical aspects were evaluated in fifty-five reports (ninety-five percent), and in most cases, the framework had been used as support. In twenty-one cases (thirty-six percent), a professional ethicist had been engaged in the work. In twelve cases (twenty-one percent), ethical aspects were presented in the main conclusions of the report. Opinions from users and reviewers revealed that the framework was generally regarded as a helpful tool, but problems regarding interpretation of specific questions were highlighted and subjected to revision.

The ethical framework is a valuable tool for systematic and transparent identification and discussion of ethical aspects in the HTA context, and it has been well implemented at SBU. A systematic approach to assess ethical aspects can facilitate the communication and dissemination of ethical aspects as principal results from the HTA project.

Depending on the health system context and the demands of relevant stakeholders in countries, the need, organizational structure, and prerequisites for enabling capacity building and development in health technology assessment (HTA) will vary. Core competencies are instrumental in this and include essential knowledge, skills, and attitudes (KSAs). They provide building blocks for delivering high-quality and effective practices of HTA. We aimed to systematically explore and develop an overview of the core competencies necessary for HTA.

This study was conducted during 2016–19 using different methods in a structured manner. We drew concepts of KSAs from various literature sources, surveyed universities and HTA professionals, and conducted expert workshops to arrive at a common understanding of the required competencies.

The terminology for KSAs defining competencies in HTA programs has been clarified. In addition, a list of competencies offered through different educational and training programs has been created. The surveys provided clarity on a common understanding of KSAs among HTA stakeholders. Thereafter, a set of competencies was described and classified according to the HTA domains.

Our study shows that there is diversity in HTA programs offered by educational institutions. The content of the programs varies due to differences between countries regarding the level of HTA development and the need for HTA, including the understanding of what HTA is. The preparation of a competency checklist or a “menu” of options mirroring the diversity of HTA will ensure that the specific needs of the HTA community will be covered.

Since 2013, the coverage of innovative and expensive drugs by the French National Health Insurance considers cost-effectiveness and budget impact, as assessed by the National Authority for Health (HAS) on the basis of an evaluation submitted by the firm. First CAR-T cell therapies were subject to economic evaluation in 2019 in France. We aim at describing the process and results of the economic evaluation of tisagenlecleucel and axicabtagene ciloleucel and the challenges these evaluations raised.

Primary evaluations were submitted by the firms to be reviewed by HAS. The final analyses were submitted to the Committee of Economic Evaluation and Public Health (CEESP), composed of independent economists, clinicians and patients’ representatives. The CEESP issued Opinions related to i) the methodological quality of economic evidence and ii) the cost-effectiveness and budget impact of the drugs under review.

The estimated incremental cost-utility ratio (ICUR) of tisagenlecleucel were rejected, being based on insufficient clinical evidence to estimate and extrapolate the long-term progression and to compare tisagenlecleucel with alternatives. Thus, the CEESP concluded that tisagenlecleucel was not proved cost-effective. The estimated ICUR of axicabtagene ciloleucel at 114,509EUR/QALY vs. chemotherapies was associated with an acceptable level of evidence despite being based on a frail indirect comparison and limited data on quality of life. In a context where France has no official cost-effectiveness threshold, the CEESP considered axicabtagene ciloleucel ICUR to be “very high” and questioned the collective acceptability of the claimed price.

The CEESP stressed that the main source of uncertainty surrounding the ICUR estimates of both drugs was related to the lack of hindsight on effectiveness, especially in terms of overall survival and safety.

The economic evaluation of CAR-T cell therapies highlights the sources of uncertainty underlying the decision and the risk of inefficient resource allocation driven by limited clinical data. It calls for payment schemes accounting for the uncertainty, and effective collection of relevant post-marketing data.

The aim of this study was to determine if magnetic resonance-guided focused ultrasound (MRgFUS) is cost-effective compared with medication, for refractory pain from bone metastases in the United States.

We constructed a Markov state transition model using TreeAge Pro software (TreeAge Software, Inc., Williamstown, MA, USA) to model costs, outcomes, and the cost-effectiveness of a treatment strategy using MRgFUS for palliative treatment of painful bone metastases compared with a Medication Only strategy (Figure 1). Model transition state probabilities, costs (in 2018 US$), and effectiveness data (quality-adjusted life-years [QALYs]) were derived from available literature, local expert opinion, and reimbursement patterns at two U.S. tertiary academic medical centers actively performing MRgFUS. Costs and QALYs, discounted at three percent per year, were accumulated each month over a 24-month time horizon. One-way and probabilistic sensitivity analyses were performed.

In the base-case analysis, the MRgFUS treatment strategy costs an additional $11,863 over the 2-year time horizon to accumulate additional 0.22 QALYs, equal to a $54,160/QALY ICER, thus making MRgFUS the preferred strategy. One-way sensitivity analyses demonstrate that for the base-case analysis, the crossover point at which Medication Only would instead become the preferred strategy is $23,341 per treatment. Probabilistic sensitivity analyses demonstrate that 67 percent of model iterations supported the conclusion of the base case.

Our model demonstrates that MRgFUS is cost-effective compared with Medication Only for palliation of painful bone metastases for patients with medically refractory metastatic bone pain across a range of sensitivity analyses.

Before the coronavirus pandemic, children who were on the Early Years Neurodevelopment (EYND) assessment pathway and suspected to have possible Autism Spectrum Disorder (ASD), received clinic based appointments. This process included a parental interview by a doctor, a specialist speech and language therapy assessment, autism diagnostic observation schedule (ADOS), which were all carried out on hospital sites. These were postponed in March following national guidance. Our aim was to continue providing accurate evidence-based service for ASD diagnosis.

We utilised evidence-based telehealth methods to perform a specialist speech and language assessment in a child's home via video call. Parents were also invited to share videos of everyday activities via a secure portal. We could observe the child in a meaningful setting and witness functional impact of their needs. Each case is discussed by a multiagency panel based on DSM-V criteria.

Online training was undertaken by professionals to deliver the Brief Observation of Autism Symptoms (BOSA) based on the ADOS for COVID times. Parents were coached by the therapist to enable them to become the administrator, rather than a professional.

Telephonic feedback from the first ten parents whose children underwent a telehealth assessment has been positive; the home was deemed more natural and for some less distressing than clinic. Formal patient surveys have been devised for both the telehealth and BOSA clinic assessments. Analysis is expected by the end of March.

To date we have been able to reach an outcome for thirty children, the diagnosis of ASD for twenty-four children and the other six received a diagnosis of global developmental delay or language disorder.

We expect that telehealth will reduce the number of assessments before an ASD diagnosis is made resulting in more prudent healthcare. The new methods have demonstrated clear increased parental participation.