Bariatric surgery has become one of the fastest growing operative procedures due to its sustained results and the increasing prevalence of obesity worldwide. Despite this fact, bariatric surgery carries the usual risks and threats of surgical interventions and therefore its benefits might be undermined by its mid and long-term complications.

This retrospective study included obese patients requiring bariatric surgery from January 2004 to December 2017 provided by a private healthcare organization in Belo Horizonte, Brazil. Data regarding healthcare utilization were extracted from an administrative database (software Oracle Business Intelligence). Continuous variables were expressed as mean and standard deviation. Log-Rank test was used to adjust the survival curve (software STATA 13.1, Stata Corp, USA). This historical cohort resulted in no interventions, neither during the instituted treatment nor after the observed outcome. Privacy of subjects and the confidentiality of their personal information were handled in accordance with the ethical principles of the Declaration of Helsinki.

In total, 16,786 patients were included in the study (mean age 37.2 ± 10.2 years; female 79.2 percent; mean body mass index 42.4 ± 5.5 kg/m2). Patients were followed for up to seven years before and after surgery (total of 78,113 patients/year). For this group, the hospitalization rate was 0.099 / patients-year before versus 0.151 / patients-year after the bariatric surgery (p < 0.001). There were 224 deaths (1.33 percent) identified during the follow-up period, 0.4 percent in the first 30 postoperative days. The average costs for hospitalization were USD 3,339.36 and USD 4,305.04 for open and laparoscopic surgery, respectively.

Bariatric surgery has been an increasingly popular choice in the management of obesity. In our sample, it did not reduce the overall mid-term healthcare utilization rate.

The Department of Sciences and Technology (Decit) of the Brazilian Government has played a vital role in drafting of the National Policy for Biologic Medicines. Decit has provided methodological support to the working group, conducting a rapid review and a rapid evidence synthesis to subsidize decisions and recommendations.

We used the Methodological Guidelines for the Elaboration of Evidence Synthesis for Health Policies, which is a product of our own team, based on the SUPporting POlicy relevant Reviews and Trials (SUPPORT) Tools for evidence-informed health Policymaking.

The Decit team participated in the key steps to develop an evidence-informed policy. Our product, “Barriers to Access to Biologic Products: a Rapid Review” was used for the prioritization of health problems and the description of the problem. We then proceeded to the evidence synthesis planning and definition of the research question from an acronym. Together with the coordination of the working group, we decided to tackle the problem of interchangeability of biologic products motivated solely by economic factors in a synthesis of policy evidence. Our evidence synthesis went so far as to describe policy options. The working group used this product to inform a Policy Dialog.

This was the first time that the Decit team provided hands-on methodological assistance the development of a health policy. Not all steps recommended in the SUPPORT Tools were feasible due to time restraints. We observed that rapid evidence synthesis products were helpful to inform decision-making.

Due to the high judicialization rates which pressure the financing of biologic medicines by the Brazilian Unified Health System (Sistema Único de Saúde - SUS), it has been decided to formulate the National Policy for Biologic Medicines. After identification of problems and prioritization, interchangeability based only on economic criteria was the main problem to be confronted. The primary objective of this study was to identify political options to approach the problem of interchangeability in systematic reviews.

We conducted a rapid evidence synthesis for policy based on an adaptation of the SUPPORT tools, and searched in six literature databases. The selection of studies was performed in a systematic, transparent and independent manner. The International Network of Agencies in Health Technology Assessment (INAHTA) members were consulted to learn how this practice occurs worldwide.

We included seven systematic reviews and one policy brief, whose options to approach the problem were: production of robust scientific evidence on interchangeability; implementation of a pharmacovigilance system; appreciation of the clinical efficacy in the practice of interchangeability; and educational strategies for healthcare professionals in Brazil. Nine countries responded to our query.

Evidence-informed policy has a central role for the Brazilian Ministry of Health. The present rapid evidence synthesis for policy will subsidize decision making regarding the interchangeability of biologic medicines within the Brazilian Unified Health System.

The French health technology assessment (HTA) body, Haute Autorité de la Santé (HAS), started to provide early advice on evidence generation plans to pharmaceutical manufacturers in 2010. It became an official mission in 2016. Requests are eligible when the product has a new mechanism of action, if there is an unmet or partially met medical need in the claimed indication and when the pivotal study has not yet started. This analysis aims to provide a first overview of clinical developments for which pharmaceutical companies sought an early dialogue with HAS.

For each product that went through an early dialogue procedure with HAS, information regarding the clinical development was collected on pharmaceuticals companies’ pipelines, clinicaltrials.gov, the website of the European Medicine Agency (EMA) and HAS's internal database.

By the end of 2018, HAS has performed 84 early dialogues of which 53 were conducted in collaboration with the EMA and/or others European HTA bodies. They were mainly focused on phase III trials. Following early dialogue, the clinical study for which the company sought advice was not yet implemented in 25 cases. When the clinical trial was effectively launched, results were negative in 10 cases, positive in 11 cases and the study was still ongoing for 29 products. In nine cases, the clinical development was officially withdrawn or suspended before the initiation of the trial. Overall, only eight medicinal products were appraised by HAS, they all obtained a clinical added value score.

The success rate of clinical development for products that underwent an early dialogue procedure tends to be higher than data from literature, although it is likely to decrease in follow-up analysis. This could be partially explained by HAS's eligibility criteria that restrict early dialogues to promising products and by the scientific recommendations provided to pharmaceuticals companies.

Chimeric antigen receptor (CAR) T-cell therapy is offered as a once-only treatment for patients with certain cancers that are not responsive to standard treatment. While clinicians, patients and their families increasingly seek access to CAR T-cell therapy, there is no revenue stream to support access through public or private health systems.

The New South Wales (NSW) Ministry of Health and Victorian Department of Health and Human Services oversighted a health technology assessment (HTA) to explore the status and geography of regulatory frameworks supporting delivery of CAR T-cell therapy, evidence for the safety, efficacy and cost, clinical trials conducted or underway and manufacturing aspects.

CAR T-cell therapies are approved in the European Union and United States of America, and being considered in Australia, Canada, China and Japan. Efficacy, safety and cost-effectiveness is limited by the size and single-arm design of early stage trials and variation between them. While overall response ranges from 36–93 percent, early results for some cancers are less favorable. Durability of treatment effect is unknown, adverse events are common and can be life-threatening and risk of delayed onset toxicity remains unknown. Treatment requires access to approved manufacturing facilities (none in Australia) and specialist clinical staff.

CAR T-cell therapy is promising and demand is increasing, but the limited safety profile and evidence base should mitigate policy and investment decisions. Broader consideration should be given to developing, or identifying access to, manufacturing and clinical workforce capability and capacity to meet national demand. Australia is likely to encounter similar issues in other jurisdictions, such as limited evidence base and complex safety issues. Factors to be considered on a local and national basis for assessment and implementation include: (i) Regulatory support for industry; (ii) Strategies to manage uncertainties in long-term risks, benefits and costs; (iii) Access to accredited manufacturing facilities; (iv) Developing clinical and manufacturing workforce capability and capacity.

The recent European Medicines Agency (EMA) approval of chimeric antigen receptor (CAR) T-cell therapies, axicabtagene ciloleucel and tisagenlecleucel, means the imminent arrival of health technology assessment (HTA) submissions to HTA agencies. HTA requires identification of all resources and organizational impacts pertaining to an intervention. Rapid review is a form of knowledge synthesis that abbreviates certain methodological aspects of systematic reviews to produce information in a timelier manner. Considering the time-sensitive nature of CAR T-cell HTAs, the aim of this research was to conduct a rapid review to identify the institutional requirements for the provision of a CAR T-cell program.

A Rapid Review protocol was developed and registered in PROSPERO. Electronic databases, EMBASE and MEDLINE, and grey literature were searched. All study designs published in English after the year 2000 were included. Studies pertained to the use of CAR T-cells in adult and pediatric patients with solid and hematological malignancies. No restrictions were placed on the comparators or study setting. Primary outcomes were organized into two categories: (i) resource use, (ii) processes relating to implementation of CAR T-cell programs. Secondary outcomes included associated costs of implementation and barriers to successful implementation. Screening, review, and extraction of relevant data was conducted by a single reviewer. Extracted data included publication details, population and setting, study characteristics, outcomes and outcome measures, and strengths and limitations of research. Data was synthesized by means of thematic analysis.

Results indicate that the provision of a CAR T-cell program in Ireland will require the establishment of bespoke infrastructural support. This includes additional outpatient facilities, ICU resources, and nursing capacity. Close relationships will need to be formed between hematology, ICU and neurology.

The findings of this Rapid Review will inform the assessment of organizational impacts associated with the introduction of a CAR T-cell program, ensuring a robust HTA assessment.

Despite widespread use of oxygen (O2) therapy, there is relatively little available information on routine O2 administration and monitoring; this is an issue particularly when considering the potential risks associated with inappropriate O2 utilization. A rapid health technology assessment (HTA) was conducted to inform the Respiratory Health Strategic Clinical Network Oxygen Summit in Alberta on aspects related to current practice in the use of O2 therapy in acute care, including administration, safety and quality, and inappropriate practice. Clinical audit is a tool used to determine deviations in practice and to identify opportunities for improvement. The objective of this presentation is to describe the experience and lessons learned from including clinical audit studies in the rapid HTA.

A standardized rapid review approach was used to identify, select, and synthesize evidence from studies published in English from 2005 to 2016. A supplementary literature search conducted in 2018 provided additional background information on the value, applicability, and limitation of using results from clinical audit studies to inform questions of good practice.

Twenty-four clinical audit studies on O2 therapy were identified; the majority were conducted in the United Kingdom. The studies varied in design, methodology, and data and outcomes reporting. Ten studies investigated the appropriateness of O2 therapy prescription pre- and post-implementation of local initiatives and interventions, which helped pinpoint major gaps in current practice, and identified general recommendations for improvement of practice. A list of reporting criteria is proposed for improving the reporting of clinical audit studies results.

Conducting clinical audit studies is resource-intensive. In the absence of other research evidence and local practice data, translating results from clinical audit studies conducted in other jurisdictions, while challenging, can help address appropriateness questions. However, inferences from these studies may be suitable only for certain topics or an operating context.

Due to the specific characteristics and challenges of mobile health (mHealth) technologies there is a need to have assessment tools based on their particularities to be used by health technology assessment (HTA) agencies and evaluation experts. In the development of a comprehensive and practical evaluation tool for the evaluation of mHealth solutions we aimed to include the views and opinions of key stakeholders: health professionals, developers, hospital managers, HTA agencies, patients and general public.

Focus groups and an online modification of the Delphi technique are being used to discuss and agree on domains and criteria to be included in the mHealth assessment tool. Domains and criteria used for health apps evaluation were drawn from a literature review on the topic. The initial list includes 95 criteria grouped into the following domains: purpose of the app, privacy and security, clinical effectiveness, content of the intervention, user experience and usability, interoperability, expenses, impact on the organization, and legal and ethical aspects. Data coming from focus groups is currently being analyzed from a thematic and content analysis perspective.

Focus groups with professionals have showed that the most important domains to be considered when evaluating health apps are those related with security, user experience, and clinical effectiveness. Some criteria were considered to be mandatory (mainly regarding safety issues), on which a first step assessment should indicate whether the app ‘pass or fails’ for the subsequent throughout assessment. Focus groups with patients will provide insight on critical aspects related to the choice, use and adherence to a health app.

Insights from main stakeholders on the design of the tool for mHealth assessment are relevant and complementary between them. Next steps include (i) the agreement of criteria by using an online modification of the Delphi Technique and (ii) piloting of the tool.

In order to facilitate patient information, patient involvement, and to support patient-centered care, healthcare organizations are increasingly offering access to patient data that are stored in the institution-specific electronic health record (EHR). Patients can access these data, read, and print them, or download and integrate them into any type of patient-held record. This EHR access is typically web-based and called “patient portal” allowing the independent access via the Internet from everywhere. A patient portal may also offer additional features such as prescription requests, appointment booking, messaging, personal health-related reminders, individual therapeutic recommendations, personal diaries, and social networking with other patients. In a Cochrane review, we assessed the effects of providing access to EHR for adult patients on patient empowerment and health-related outcomes compared to usual care.

According to the methods of evidence-based medicine, we developed a protocol for a Cochrane review, which is published in the Cochrane database.

We identified ten randomized controlled trials (RCTs) including 6,668 randomized participants. Seven RCTs took place in the USA, two in Canada, and one in Japan. Additional functionalities of interventions and disease conditions were heterogeneous. Three studies (n = 601) reported on patient empowerment. The risk differences reported were neither statistically significant nor clinically relevant. Eight studies (n = 2,070) reported on nine different risk factors (blood pressure, blood glucose, poor asthma control, 10-year Framingham risk score, cholesterol, body mass index, composite score of eight variables, intraocular pressure, composite score of three variables). The results were heterogeneous. Mostly there were no statistically significant risk differences between study groups.

Overall, there is no evidence for a clear positive effect of patient portals on patient empowerment and health related outcomes (mainly risk factors). However, we identified only a small number of studies. The usage of portals was often low and several studies were older.

The project DigiHelse aims to support the municipality health in Norway by offering a digital communication platform to users of the home care service nationally. In a concept stage of innovation, an early assessment of the potential socioeconomic value of the project was carried out by means of stakeholder insight and scenario drafting. As the assessment showed favorable potential in providing decision support and reducing risk, the project received funding to move into the pilot phase. The objective of this study is to reassess the effect of stakeholder insight and scenario drafting by validating the results using empirical data from the first pilot of DigiHelse.

Through collecting empirical data on resource consumption and inquiries to the service from four intervention districts and one control district in Oslo, the socioeconomic value of DigiHelse was reassessed. In addition to survey and register data collected before and after the pilot, behavioral data was introduced as a new data source.

The effect of early assessment by means of stakeholder insight and scenario drafting was successfully studied adding empirical data from the projects first pilot. The real-time data on user behavior registered in the DigiHelse server contributed to verify the assumptions from the first assessment of the project. Although the results from the analysis were less optimistic than the first assessment, the study revealed important improvement measures necessary to improve the innovation process.

The usefulness of early assessment is questioned, due to lack of precision of estimates caused by scarce available data. The present study presents a first step in evaluating the precision of employing stakeholder insight and scenario drafting as additional information in early assessment of innovation. The studied approach to early assessment showed potential in enhancing decision support and reducing risk from a concept stage of innovation.

Australians are adjusting to mobile health (mHealth) applications (apps) being used in clinical care. The nature of apps presents unique challenges (e.g. rapid lifecycle) to mHealth regulation. The risks they pose are mainly through the information they provide and how it is used in clinical decision-making. This study explores the international regulation of mHealth apps. It assesses whether the approach used in Australia to regulate apps is consistent with international standards and suitable to address the unique challenges presented by the technology.

A policy analysis was conducted of all nine member jurisdictions of the International Medical Device Regulator's Forum (IMDRF), to determine if their regulatory agencies addressed the IMDRF recommendations relevant to the clinical evaluation of mHealth apps. Case-studies (submission to regulatory agencies) were also selected on varying types of regulated apps (standalone, active implantable, etc.) and assessed relative to the principles in the IMDRF's software as a medical device (SaMD): Clinical evaluation (2017) guidance document.

All included jurisdictions evaluated the effectiveness of mHealth apps, assessing the majority of the key sub-categories recommended by SaMD: Clinical evaluation. The submissions and jurisdictional regulatory bodies did not address the IMDRF safety principles in terms of the apps’ information security (cybersecurity). Furthermore, by failing to use the method recommended by the IMDRF (risk-classification), none of the submissions or jurisdictions recognized the potential dangers of misinformation on patient safety.

None of the approaches used by global regulatory bodies adequately address the unique challenges posed by apps. Australia's approach is consistent with app regulatory procedures used internationally. We recommend that mHealth apps are evaluated for cybersecurity and are also classified using the IMDRF risk-categories so as to fully protect the public.

Mobile health (mHealth) applications (app) are being integrated into healthcare by patients and practitioners in Australia. However, there are currently no policies or frameworks available that can be used to conduct a health technology assessment (HTA) on mHealth apps for reimbursement purposes. The aim of the study was to determine what policy changes and assessment criteria are needed to facilitate the development of a system that evaluates mobile medical apps for regulatory and reimbursement purposes in Australia.

To obtain the information to determine what policy changes are needed and create an evidence-based framework that can evaluate mHealth apps for reimbursement decision-making, four studies were conducted. This research included (i) a policy analysis on international mHealth app regulation; (ii) a case study on American and Australian app regulation; (iii) a methodological systematic review on the suitability of current mHealth evaluation frameworks for reimbursement purposes; and (iv) the identification of HTA pathways and impediments to app reimbursement through stakeholder interviews. An evaluation framework for apps was created by combining and synthesizing the results.

Software changes, connectivity, and cybersecurity need to be considered when evaluating mHealth apps for reimbursement purposes. Additionally, the potential dangers of apps providing misinformation, and poor software reliability in current regulation must be considered. Stakeholders indicated that they trust how traditional medical devices are currently appraised for reimbursement in Australia. They expressed caution around the lack of clarity regarding who is responsible for app quality as well as concerns about the digital literacy of medical practitioners and their patients.

Since stakeholder trust in the current HTA process for medical devices in Australia is high, the process was adapted to create an evaluation framework for mHealth apps. The adaptations included making provisions for cybersecurity, software updates, and compatibility issues. Provisions to address concerns around practitioner responsibility and misinformation were incorporated into the framework.

The burden of mental health disorders has a wide societal impact affecting primarily individuals and their significant others. Mental health interventions produce costs and benefits in the health care sector but can also lead to costs and benefits in non-healthcare sectors, also known as inter-sectoral costs and benefits (ICBs). The aim of this study was to develop an internationally applicable list of ICBs in the educational sector resulting from mental health interventions and to facilitate the inclusion of ICBs in economic evaluations across the European Union (EU) by prioritizing important ICBs.

Some ICBs of mental health interventions were identified in earlier research, which were used as a basis for this study. Additional data was collected via a systematic literature search of PubMed and a grey literature search carried out in six EU countries. In order to validate the international applicability of the list and prioritize the ICBs, a survey was conducted with the international group of experts from the educational sector. The outcomes of the expert survey were used to create the condensed list containing the most important ICBs.

The literature search allowed identifying additional ICBs and creating a comprehensive list of items. In order to improve its usability, a multi-dimensional list was constructed distinguishing between tangible (i.e. special education) and intangible items (i.e. cognitive deficits). Based on the expert survey, the international applicability of the list was validated and the most important ICBs from the economic perspective were determined.

Mental health interventions can affect a large number of educational facilities. The list of ICBs developed in this study could be used to select relevant educational facilities for economic evaluations of specific mental health disorders. Further research is needed to define, measure, and valuate the identified ICBs in order to facilitate the practical application of the list in economic evaluations.

A number of economic evaluations of gender-neutral human papillomavirus (HPV) vaccination have been published, generally finding that the cost-effectiveness is sensitive to the uptake rate in girls. In Ireland there is a girls-only program in place, but the initial high uptake rate (>85 percent) was substantially impacted by high profile negative publicity concerning perceived vaccine safety issues. Efforts to address perceived safety concerns have recently yielded a partial recovery in uptake rates. The aim of this study was to estimate the cost-utility of extending the program to include boys and explore the impact of fluctuating uptake rates.

A previously published cost-utility model used in the United States of America and Norway was adapted to the Irish setting and populated with Irish epidemiological and cost data. Comparators included no vaccination, and girls-only and gender-neutral vaccination, both with either a 4-valent or 9-valent vaccine. Vaccination is at age 12 years and oropharyngeal and penile cancers were excluded in the base case analysis. Additional analyses were used to incorporate fluctuating uptake rates into the model.

A 9-valent girls-only program dominated the existing girls-only 4-valent program. The incremental cost-effectiveness ratio (ICER) for a gender-neutral 9-valent program was EUR 50,823/quality-adjusted life year (QALY). Gender-neutral vaccination would be cost-effective at a willingness-to-pay threshold of EUR 45,000/QALY when the uptake rate is below 78 percent. The ICER decreased to between EUR 41,000 and EUR 42,000/QALY when the uptake rate was allowed to fluctuate across six to 12 yearly cycles.

The cost-effectiveness of gender-neutral HPV vaccination is highly sensitive to the assumed uptake rate in girls. Large fluctuations in HPV vaccine uptake rates have been observed in a number of countries in the last decade. Incorporating fluctuating uptake rates in the model shows that a gender-neutral program may be more cost-effective than when a stable uptake is assumed.

All new products to be reimbursed from the Irish health budget are subject to a rigorous assessment by the National Centre for Pharmacoeconomics (NCPE). Following assessment, a recommendation is made regarding its cost-effectiveness at the submitted price. This may lead a reduction in the drug price. This study aimed to determine the reduction in expenditure due to the pharmacoeconomic assessment process in Ireland.

Product details, submitted price and gross budget impact were recorded for each NCPE submission from 2012 to 2015. The latter was chosen as reimbursement data are currently available until 2016. A product was included if its assessment suggested price reduction was required and the product was reimbursed under the High-Tech Drug Scheme (HTDS), a scheme for high cost drugs in a primary care setting. The utilization and actual expenditure of each product was extracted from national reimbursement data for the year after approval. The expected expenditure, calculated using the submitted price, was then compared to the actual expenditure.

A total of 162 products were assessed during the study period. There was a potential price reduction for 65 products based on the assessment outcome. Of these, 15 were reimbursed under the HTDS. A reduction in expenditure was evident for eight of the 15 products (53 percent). The average reduction was eight percent of the expected expenditure. All products showed an actual expenditure greater the predicted budget impact submitted by the applicant.

To the authors’ knowledge, this is the first report of expenditure reduction due to a pharmaco-economic assessment process. With the ever-increasing utilization of high cost drugs, the study demonstrates the importance of a process to assess and negotiate cost-effective drug prices. However, the study underestimates reductions, as it is yet to include commercial rebates returned to a central budget. Future research will aim to capture these reductions.

Due to the promising physical dose distribution of carbon ion radiation therapy (CIRT), CIRT can be regarded as a novel tumor irradiation technique and is sometimes considered as a breakthrough therapy for various tumor types. However, it is unclear whether superiority or inferiority can be claimed when compared to standard irradiation. This study aimed to assess the scientific evidence regarding the effectiveness and safety of CIRT.

A systematic literature review was conducted using the European Network for Health Technology Assessment (EUnetHTA) Core Model® for rapid relative effectiveness assessment. The literature search for clinical outcome studies on CIRT was performed in four databases [Cochrane (Central), Centre for Research and Dissemination (CRD), Embase and OVID MEDLINE]. The risk of bias was assessed using the Cochrane Risk of Bias Tool (for randomized controlled trials) and the Institute of Health Economics (IHE) Checklist (for observational studies). The evidence synthesis was restricted to 12 tumor regions (and 54 indications) and studies with a low or moderate risk of bias, published between 2005 and 2017.

In total 27 studies were eligible for the qualitative synthesis of the evidence regarding the effectiveness and safety of CIRT; one randomized controlled trial that primarily focused on the feasibility of CIRT, three case-control studies, three before-after studies focusing on quality of life, and 20 further case series studies. Overall, insufficient scientific evidence was found for 13 (out of 54) indications in seven tumor regions and no scientific evidence was found for 41 (out of 54) indications.

Theoretically, CIRT is undoubtedly a promising cancer treatment. To date, however, 54 oncologic indications in 12 tumor regions under investigation lack randomized controlled trials assessing the long-term effectiveness and harms associated with its use. CIRT must be considered as an experimental treatment due to the lack of high-quality clinical research.

Appropriate involvement of stakeholders is one of the founding principles of the European Cooperation on Health Technology Assessment. The European Network for Health Technology Assessment (EUnetHTA) produces Rapid Relative Effectiveness Assessments (REAs) to assess pharmaceutical (PT) or other technologies (OT). Stakeholders essentially participate in the scoping, the draft assessment phase, or both.

All REAs published since 2013 were reviewed. Stakeholder participation in scoping (project plan) and draft assessment was evaluated. We aggregated categories of stakeholders in four groups (Health Care Providers and Academia, Patients and Consumers, Manufacturers, and Regulators and Payers). Means of collaboration (meetings, comments to project plan and draft assessment, questionnaires, focus groups) are also analyzed. Data is continuously updated with new REAs.

More than 20 REAs have been published at the moment, with a higher number of OT. Health Care Providers and Academia acted as experts in both phases, participating in all REA of OT, and less of PT. Manufacturers participated in all REA in the scoping phase. Regulators and Payers, less involved, participated mainly in the scoping phase. The main methods are providing comments in a standardized form and meetings. Patients' contribution, similar in OT and PT, has increased over the years. Questionnaires or interviews were the main method of involvement, followed by participation in meetings and focus groups. Visibility and transparency have also improved, with a clearer reporting of the stakeholder contribution in the last assessments.

The stakeholder involvement in EUnetHTA REAs is steadily growing, with the different nature of stakeholders’ categories reflected in their contribution to the assessments. EUnetHTA is standardizing stakeholder involvement procedures taking into account the particularities of each group when generating guidance for stakeholder involvement.

A core outcome set (COS) is a minimum standardized set of agreed-upon outcomes for clinical trials of a specific condition. COS development can improve research by aligning stakeholder priorities for the outcomes most important in decision-making across the life-cycle of a product. It is important to include health technology assessment (HTA) representatives in COS development to ensure that outcomes useful to HTA are consistently included in clinical trials. Here we describe the role of HTA representatives in two COS projects: coreHEM, for gene therapy for hemophilia, a genetic blood clotting disease; and coreNASH, for nonalcoholic steatohepatitis (NASH), a progressive form of fatty liver disease that can lead to cirrhosis. We will describe the voting patterns of HTA representatives and consider aspects of their role in shaping the final COS.

For each multi-stakeholder COS, a modified Delphi process was utilized (three online surveys plus an in-person consensus meeting). Candidate outcome lists were compiled via a literature review complemented by participant interviews. Voters condensed and prioritized the lists by rating each outcome on a scale of 1-9 (not important-essential). Votes on each outcome were stratified by stakeholder group; HTA votes were compared with those of other stakeholders.

HTA representatives made up 12.2 percent and 13.5 percent of the voters in coreHEM and coreNASH, respectively. They tended to give the highest votes to mortality outcomes, outcomes measuring the severity of disease, and outcomes related to a patient's quality of life, general well-being and general health perspective. HTA votes helped certain outcomes meet the inclusion criteria in the final voting rounds; without HTA voters, the “mental health status” outcome in coreHEM and the “hepatic-related mortality” and “liver transplantation” outcomes in coreNASH would have been eliminated.

HTA participation in COS projects provides HTA representatives an opportunity to help shape COS in clinical research for better decision-making.

Since 2009, China has initiated a national program on free provision of essential public health services. The national program has expanded both in terms of service categories and funding, showing China's great commitment to universal health coverage. However, with slowdown of public input in the health sector, the government decided to prioritize interventions and optimize reimbursement packages. Researchers in the China National Health Development Research Center (CNHDRC)—the Chinese national health technology assessment (HTA) agency were asked to design the tools to facilitate the decision process.

With multi-criteria decision analysis (MCDA) method, the researchers analyzed value dimensions in public health issues, and built an evidence matrix for the priority-setting decisions. Supported by HTA tools, they appraised interventions and services through literature review and field studies, and projected budget impact of potential adjustment decisions based on cost analysis results. A deliberative process of key stakeholder groups was taken, and their views were counted in making the final recommendations.

Based on evidence review and scores of stakeholders’ judgment, two public health service interventions were recommended for removal, and another two for adjustment (one for merger, one for optimizing care pathway). Cost estimation and potential budgetary impact were also analyzed to support financial decisions.

HTA and MCDA are key tools for defining the value criteria, evidence framework, and deliberative process for the essential public health program. However, lack of cost-effectiveness evidence hinders fine-tuned decisions on resource allocation. Continual health economic evaluation needs to be conducted in the near future.

One objective of the European Network for Health Technology Assessment (EUnetHTA) Joint Action 3 (JA3) is to set up a quality management system (QMS) for joint work that serves as a standalone infrastructure for a sustainable European HTA-collaboration. Structures of the QMS (quality policy, processes and procedures, and organizational structures) combined with the measures of QM (quality planning, assurance, control and improvement) both ensure achieving the objective of producing high-quality HTAs.

Based on a thorough concept with involvement of a spectrum of EUnetHTA partners, re-evaluation, internal workshops and national expertise, the existing inventory consisting of procedures, templates, methodological guidelines and tools is being refined, complemented and revised. Procedures are gradually being transferred into Standard Operating Procedures (SOPs), seamlessly and chronologically covering all assessment phases. Supplemented by quality-control-checklists and templates these SOPs are linked to relevant guidelines and tools. The so-established QMS is subject to continuous improvement by recurrently applying the Plan-Do-Check-Act (PDCA)-cycle. Members of the assessment teams are surveyed after the publication of each EUnetHTA-report (check-phase). Processed results lead to modification, maintenance and improvement of the inventory (act-phase).

The majority of content has already been developed or revised and was already made available to the assessment teams to put to practical test. The survey results are systematically collected, processed and compiled. Derived from these results, a comprehensive report and a thorough list of improvement measures have been developed and will serve as a basis for further adaptions.

The establishment of structures of QMS and measures of QM both are supposed to assure high-quality HTA-reports for EUnetHTA and a possible permanent European collaboration on HTA after JA3 as proposed by the EU-commission's regulation proposal 2018/0018 (COD). The installed systematic data collection, processing and compilation mechanisms are a solid basis for the identification of future needs for developments.