The aim of this study was to provide an overview of the methodological characteristics and compare the assessment methods applied in health technology assessments (HTAs) of public health interventions (PHIs).

We defined a PHI as a population-based intervention on health promotion or for primary prevention of chronic or nonchronic diseases. HTAs on PHIs were identified by systematically searching the Web pages of members of international HTA networks. We included only full HTA reports published between 2012 and 2016. Two reviewers extracted data on the methods used to assess effectiveness/safety, as well as on economic, social, cultural, ethical, and legal aspects using a-priori standardized tables.

We included ten HTAs provided by four different organizations. Of these, all reports assessed the effectiveness of the interventions and conducted economic evaluations, seven investigated social/cultural aspects, and four each considered legal and ethical aspects, respectively. Some reports addressed applicability, context/setting, and intervention fidelity issues in different ways. We found that most HTAs adapted their methods to some extent, for example, by including nonrandomized studies, expanding the search strategy, involving stakeholders, or applying a framework to guide the HTA process.

Our analysis provides a comprehensive overview of methods applied in HTAs on public health interventions. We found that a heterogeneous set of approaches is used to deal with the challenges of evaluating complex public health interventions.

Traditional decision rules have limitations when a new technology is less effective and less costly than a comparator. We propose a new probabilistic decision framework to examine non-inferiority in effectiveness and net monetary benefit (NMB) simultaneously. We illustrate this framework using the example of repetitive transcranial magnetic stimulation (rTMS) and electroconvulsive therapy (ECT) for treatment-resistant depression.

We modeled the quality-adjusted life-years (QALYs) associated with the new intervention (rTMS), an active control (ECT), and a placebo control, and we estimated the fraction of effectiveness preserved by the new intervention through probabilistic sensitivity analysis (PSA). We then assessed the probability of cost-effectiveness using a traditional cost-effectiveness acceptability curve (CEAC) and our new decision-making framework. In our new framework, we considered the new intervention cost-effective in each simulation of the PSA if it preserved at least 75 percent of the effectiveness of the active control (thus demonstrating non-inferiority) and had a positive NMB at a given willingness-to-pay threshold (WTP).

rTMS was less effective (i.e., associated with fewer QALYs) and less costly than ECT. The traditional CEAC approach showed that the probabilities of rTMS being cost-effective were 100 percent, 39 percent, and 14 percent at WTPs of $0, $50,000, and $100,000 per QALY gained, respectively. In the new decision framework, the probabilities of rTMS being cost-effective were reduced to 23 percent, 21 percent, and 13 percent at WTPs of $0, $50,000, and $100,000 per QALY, respectively.

This new framework provides a different perspective for decision making with considerations of both non-inferiority and WTP thresholds.

Ethics has been considered among the core domains of health technology assessment (HTA), but there are still disputes regarding ethical analysis. This study aimed to examine full final reports of the European Network for Health Technology Assessment (EUnetHTA) in terms of their compliance with the ethical methodology and ethical perspective of the HTA Core Model®.

The study examines seven full final HTA reports of EUnetHTA written based on the methodology proposed in the HTA Core Model®. The reports were analyzed using the following parameters: competency of the person/group who conducted ethical analysis, assessment elements, and the methodology of ethical analysis.

The results show that, although the HTA Core Model® helped to standardize the final reports of the assessment, there are still concerns regarding the competency of the ethical analysis team, the perspectives on the purpose of ethical analysis, data sources and viewpoints of various stakeholders, use of ethical analysis methodology, and the evaluation of the ethical appropriateness of the entire HTA process.

The HTA Core Model® helped to standardize the final reports on the HTA; however, not all issues with the content and outcomes were solved. The lack of expertise in ethics and insufficiency of the teams regarding ethical analysis are other existing problems. This study also demonstrated that stakeholder viewpoints in general and patient perspectives, in particular, have been overlooked in the HTA process.

Timely access to innovative medical technologies driven by accelerated patient access pathways can substantially improve the health outcomes of patients who often have few therapeutic alternatives. We analyzed lead-times for the medical procedure reimbursement coverage process undertaken in South Korea from 2014 to 2017, which is considered one of the most important factors contributing to delays in patient access to new medical technologies.

This analysis was performed using the open datasets source of “Medical Procedure Expert Evaluation Committee (MPEEC)” meeting results and medical procedure coverage application information published on the Health Insurance Review and Assessment Service Web site.

From 2014 to 2017, 90 percent of all new coverage determinations took on average >250 days with almost 20 percent taking more than 2 years (>750 days), The average lead-time from the medical procedure coverage application to MPEEC meeting in 2015 was 435.0 ± 214.7 days (n = 26), which was significantly shorter than the average lead-time in 2014 (624.9 ± 290.3 days, n = 16) (p < .05). The average lead-time from application to official enforcement in 2015 was significantly shorter than that of 2014 (540.8 ± 217.4; n = 16 versus 734.1 ± 299.7 days; n = 26, respectively) (p < .05).

While this analysis showed a general trend of a reduction in the time taken to receive a positive coverage determination for a new medical technology, the average lead-time remains well over the government mandated 100 days. To continue this trend and further enhance the patient access pathway for medical procedure coverage determinations, some measures can be applied. In particular, the extended “One-Stop Service” program encompassing coverage determinations is one such recommendation that could be considered.

Indirect comparisons via a common comparator (anchored comparisons) are commonly used in health technology assessment. However, common comparators may not be available, or the comparison may be biased due to differences in effect modifiers between the included studies. Recently proposed population adjustment methods aim to adjust for differences between study populations in the situation where individual patient data are available from at least one study, but not all studies. They can also be used when there is no common comparator or for single-arm studies (unanchored comparisons). We aim to characterise the use of population adjustment methods in technology appraisals (TAs) submitted to the United Kingdom National Institute for Health and Care Excellence (NICE).

We reviewed NICE TAs published between 01/01/2010 and 20/04/2018.

Population adjustment methods were used in 7 percent (18/268) of TAs. Most applications used unanchored comparisons (89 percent, 16/18), and were in oncology (83 percent, 15/18). Methods used included matching-adjusted indirect comparisons (89 percent, 16/18) and simulated treatment comparisons (17 percent, 3/18). Covariates were included based on: availability, expert opinion, effective sample size, statistical significance, or cross-validation. Larger treatment networks were commonplace (56 percent, 10/18), but current methods cannot account for this. Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk.

Population adjustment methods are becoming increasingly common in NICE TAs, although their impact on decisions has been limited to date. Further research is needed to improve upon current methods, and to investigate their properties in simulation studies.

It is important to capture all health effects of interventions in cost-utility analyses conducted under a societal or healthcare perspective. However, this is rarely done. Household spillovers (health effects on patients’ other household members) may be particularly likely in the context of technologies and interventions to change behaviors that are interdependent in the household. Our objective was to prospectively collect outcome data from household members and illustrate how these can be included in a cost-utility analysis of a behavior change intervention in chronic obstructive pulmonary disease (COPD).

Data were collected from patients’ household members (n = 153) alongside a randomized controlled trial of a COPD self-management intervention. The impact of the intervention on household members’ EQ-5D-5L scores (primary outcome), was evaluated. Analyses were then carried out to estimate household members’ quality-adjusted life-years (QALYs) and assess the impact of including these QALYs on cost-effectiveness.

The intervention had a negligible spillover on household members’ EQ-5D-5L scores (−0.007; p = .75). There were also no statistically significant spillovers at the 5 percent level in household member secondary outcomes. In the base-case model, the inclusion of household member QALYs in the incremental cost-effectiveness ratio (ICER) denominator marginally increased the ICER from GBP 10,271 (EUR 13,146) to GBP 10,991 (EUR 14,068) per QALY gained.

This study demonstrates it is feasible to prospectively collect and include household members’ outcome data in cost utility analysis, although the study highlighted several methodological issues. In this case, the intervention did not impact significantly on household members’ health or health behaviors, but inclusion of household spillovers may make a difference in other contexts.

Healthcare organizations have invested efforts on hospital-based health technology assessment (HB-HTA) and enterprise risk management (ERM) processes for novel systems to obtain more accurate data on which to base strategic decisions. This study proposes to analyze how HB-HTA and ERM processes can share personal resources and skills to achieve principles with value-oriented results.

Literature on ERM and HB-HTA and data from interviews with healthcare managers compose the research data sources, which were submitted to a qualitative data analysis. It was oriented to identify the association between ERM and HB-HTA application in hospitals and the common principles between both processes, in addition to proposing the capability to share personal resources between both teams in a matrix.

The common principles and personal background suggested for HB-HTA and ERM teams allowed the build of a matrix identifying how both teams can work in an integrated manner being more effective and value-oriented. The shared resource matrix reports how each professional (with a specific background) may interact with each activity associated to HB-HTA or ERM implementation guidelines.

The identification of common principles and capabilities between ERM and HB-HTA suggested advances with the literature from both research areas. The opportunity to share personal resources also contributes to the implementation of those processes in hospitals with less financial resources, approaching its own management to be more efficient with the care chain.

Very few practical frameworks exist to guide the formulation of recommendations at hospital-based health technology assessment (HTA) units. The objectives of our study were: (i) to identify decision criteria specific to the context of hospital-based health technologies and interventions, (ii) to estimate the extent to which the expert community agrees on the importance of the identified criteria, (iii) to incorporate the identified criteria into a decision-aid tool, and (iv) to illustrate the application of a prototype decision-aid tool.

Relevant decision criteria were identified using existing frameworks for HTA recommendations, our past experience, a literature search, and feedback from a survey of diverse stakeholders.

Based on the survey results, twenty-three decision criteria were incorporated into the final framework. We defined an approach that eschewed a scoring system, but instead relied on a visual means for arriving at a final recommendation, by juxtaposing the importance rating for each criterion against the results of the health technology assessment. For a technology to be approved, a majority of criteria considered important should also have received favorable findings.

We created a simple and practical decision-aid tool that incorporates all decision criteria relevant to a hospital-based HTA unit. With its ease of use and accessibility, our tool renders the subjective decision-making process more structured and transparent.

As healthcare decision makers continue to face challenges in health services delivery to their patients, disinvestment programs are being established for a sustainable healthcare system. This study aimed to collect data and information by means of a survey of disinvestment candidates and ongoing disinvestment projects in the health technology assessment (HTA) community.

An online survey was conducted to collect information on disinvestment candidates and activities from members of the Health Technology Assessment International Disinvestment & Early Awareness Interest Group, the EuroScan International Network and International Network of Agencies for Health Technology Assessment.

Among the 362 invitees, twenty-four unique responses were received, and almost 70 percent were involved in disinvestment initiatives. The disinvestment candidates identified represented a range of health technologies. Evidence or signaling of clinical ineffectiveness or inappropriate use typically led to the nomination of disinvestment candidates. Health technology assessments and reassessments were usually conducted to evaluate the technology in question, and decisions usually led to the limited use of the technology. Barriers to disinvestment decisions included the strength of interest and advocacy groups, insufficient data for assessments, a systematic decision process and political challenges, while obstacles to their implementation were clinicians’ reluctance and insufficient funding and incentives.

The survey results suggested that disinvestment activities are occurring in the HTA community, especially in the public sector. Future research can further investigate the processes and methods used to reach and implement disinvestment decisions from our survey respondents and explore to form closer ties between the HTA and clinical communities.

There is no established methodology to assess the feasibility of medicine price data sources. Against this backdrop, a framework to guide the selection of most appropriate price data sources for pharmacoeconomic research has been developed.

A targeted literature review was carried out. Dimensions discussed in literature as relevant for medicine price comparisons and practical experience of the authors in medicine price studies informed the conceptional work of the framework development. A draft version of the framework was reviewed by peer pricing experts. The feasibility of the framework was tested in case studies.

According to the developed framework (called Re-ADAPT), a medicine price data source should meet the following criteria: reliability and sustainability; accessibility at a cost that users can afford; provision of medicine price information at the date(s) required; information for the defined geographic area, or at least in a representative way; coverage of the pharmaceuticals and at the price type(s) required. Easy handling and provision of additional information were defined as supportive assets of candidate data sources (secondary criteria). The case studies confirmed the feasibility of the Re-ADAPT framework. In some cases, however, it can be difficult to disentangle assessment criteria (particularly geographic area, scope of pharmaceuticals and price types) for separate consideration, given their interlinkage.

While selection of the most appropriate data sources will remain a challenge, the Re-ADAPT framework aims to provide practical guidance and thus contribute to a more careful, balanced, and evidence-based selection of data sources for medicine price studies.

Social and cultural aspects are rarely assessed in health technology assessments (HTA), despite being part of most HTA definitions. One hypothesis for the reason why they are hardly considered in HTA is that we lack relevant assessment methods. Accordingly, this review aims at providing an overview of methodological approaches to address social and cultural aspects related to health technologies in HTA.

We conducted a comprehensive literature search by searching fourteen databases and a hand-search of two pertinent journals. Additionally, we sent a query to all member agencies of the International Network of Agencies for Health Technology Assessment (INAHTA) asking them for methods they use to assess social and cultural aspects.

A total of 125 publications met our inclusion criteria. We grouped the methodological approaches into checklists for experts, literature reviews, stakeholder participatory approaches, primary data collection methods, and combinations of methodological approaches.

There is a wide variety of methods available for assessing social and cultural aspects of health technologies, some of which have been applied in HTA. The presented overview of the different approaches and their merits can facilitate the assessment of these aspects, and improve the knowledge regarding (potential) success and failure of the implementation of a health technology.

The aim of this study was to develop a feasible and effective strategy to involve patients in the Spanish Network of Agencies of Health Technology Assessment (RedETS).

The framework for patient involvement (PI) in the assessment activities and processes of RedETS were developed through a research project that included: (i) a systematic search of the international literature describing a strategy and/or a methodology linking health technology assessment (HTA) and PI; (ii) a qualitative study through interviews with RedETS members to analyze the perceptions of PI among HTA managers in the Spanish context; (iii) a Delphi consultation with three large platforms of patients, carers and consumer organizations in Spain about their perspectives of PI; (iv) a consensus process with the members of the RedETS Governing Council to define the final strategy.

Three main themes were identified in the literature and Web site review: (i) PI methods for the different HTA phases; (ii) Participant definition and selection; (iii) Resources needed. A three-step implementation strategy was proposed: (i) short-term actions: piloting and testing patient participation in HTA and building patients' capacity; (ii) medium-term actions: broadening the participation of patients, and building internal capacity; (iii) long-term actions: consolidating and mainstreaming patient involvement

Patient participation can be incorporated into almost all the HTA phases and products with greater or lesser degrees of difficulty. However, a progressive implementation strategy is suggested for a feasible PI process.

Population-based colorectal cancer (CRC) screening programs that use a fecal immunochemical test (FIT) are often faced with a noncompliance issue and its subsequent waiting time (WT) for those FIT positives complying with confirmatory diagnosis. We aimed to identify factors associated with both of the correlated problems in the same model.

A total of 294,469 subjects, either with positive FIT test results or having a family history, collected from 2004 to 2013 were enrolled for analysis. We applied a hurdle Poisson regression model to accommodate the hurdle of compliance and also its related WT for undergoing colonoscopy while assessing factors responsible for the mixture of the two outcomes.

The effect on compliance and WT varied with contextual factors, such as geographic areas, type of screening units, and level of urbanization. The hurdle score, representing the risk score in association with noncompliance, and the WT score, reflecting the rate of taking colonoscopy, were used to classify subjects into each of three groups representing the degree of compliance and the level of health awareness.

Our model was not only successfully applied to evaluating factors associated with the compliance and the WT distribution, but also developed into a useful assessment model for stratifying the risk and predicting whether and when screenees comply with the procedure of receiving confirmatory diagnosis given contextual factors and individual characteristics.

Evidence requirements and assessment methods access differ between health technology assessment (HTA) agencies. The HTA Core Model® provides a standardized approach to HTA, targeting evidence sharing and collaboration between participating HTA bodies. It is fit for purpose from an industry perspective and was used by pharmaceutical company Roche to develop a framework for internal assessment of evidence required for market access and coverage/reimbursement (“access evidence”).

Tools were developed to systematically scope, assess, plan, and summarize access evidence generation. The tools were based mainly on the first four HTA Core Model® domains and rolled-out in selected development teams in 2017. Five months after full implementation, the impact of tools was assessed in an internal survey.

Systematic access evidence generation started with the Access Evidence Questionnaire, to scope evidence requirements and identify evidence gaps. Findings were summarized in the Access Evidence Metric, which assessed the alignment of available/planned evidence against HTA bodies’ requirements and developed scope mitigation strategies. The Access Evidence Plan was then used to plan and document (additional) evidence generation. Once generated, evidence was summarized in the Access Evidence Dossier. A survey of twenty-seven Roche employees involved in evidence generation showed that the tools made discussions around access strategies and evidence more efficient and transparent.

The HTA Core Model® provided a useful framework around which to optimize internal evidence generation and assessment. The benefits of using a standardized HTA approach in industry mirror those expected from implementing the HTA Core Model® in HTA agencies.

Early assessment can assist in allocating resources for innovation effectively and produce the most beneficial technology for an institution. The aim of the present study was to identify methods and discuss the analytical approaches applied for the early assessment of innovation in a healthcare setting.

Knowledge synthesis based on a structured search (using the MEDLINE, Embase, and Cochrane databases) and thematic analysis was conducted. An analytical framework based on the stage of innovation (developmental, introduction, or early diffusion) was applied to assess whether methods vary according to stage. Themes (type of innovation, study, analysis, study design, method, and main target audience) were then decided among the authors. Identified methods and analysis were discussed according to the innovation stage.

A total of 1,064 articles matched the search strategy. Overall, thirty-nine articles matched the inclusion criteria. The use of methods has a tendency to change according to the stage of innovation. Stakeholder analysis was a prominent method in the innovation stages and particularly in the developmental stage, as the introduction and early diffusion stage has more availability of data and may apply more complex methods. Barriers to the identified methods were also discussed as all of the innovation stages suffered from lack of data and substantial uncertainty.

Although this review has identified applicable approaches for early assessment in different innovation stages, research is required regarding the value of the available data and methods and tools to enhance interactions between different parties at different stages of innovation.

The aim of this study was to describe patient level costing methods and develop a database of healthcare resource use and cost in patients with AHF receiving ventricular assist device (VAD) therapy.

Patient level micro-costing was used to identify documented activity in the years preceding and following VAD implantation, and preceding heart transplant for a cohort of seventy-seven consecutive patients listed for heart transplantation (2009–12). Clinician interviews verified activity, established time resource required for each activity, and added additional undocumented activities. Costs were sourced from the general ledger, salary, stock price, pharmacy formulary data, and from national medical benefits and prostheses lists. Linked administrative data analyses of activity external to the implanting institution, used National Weighted Activity Units (NWAU), 2014 efficient price, and admission complexity cost weights and were compared with micro-costed data for the implanting admission.

The database produced includes patient level activity and costs associated with the seventy-seven patients across thirteen resource areas including hospital activity external to the implanting center. The median cost of the implanting admission using linked administrative data was $246,839 (interquartile range [IQR] $246,839–$271,743), versus $270,716 (IQR $211,740–$378,482) for the institutional micro-costing (p = .08).

Linked administrative data provides a useful alternative for imputing costs external to the implanting center, and combined with institutional data can illuminate both the pathways to transplant referral and the hospital activity generated by patients experiencing the terminal phases of heart failure in the year before transplant, cf-VAD implant, or death.

This study investigated which databases and which combinations of databases should be used to identify economic evaluations (EEs) to inform systematic reviews. It also investigated the characteristics of studies not identified in database searches and evaluated the success of MEDLINE search strategies used within typical reviews in retrieving EEs in MEDLINE.

A quasi-gold standard (QGS) set of EEs was collected from reviews of EEs. The number of QGS records found in nine databases was calculated and the most efficient combination of databases was determined. The number and characteristics of QGS records not retrieved from the databases were collected. Reproducible MEDLINE strategies from the reviews were rerun to calculate the sensitivity and precision for each strategy in finding QGS records.

The QGS comprised 351 records. Across all databases, 337/351 (96 percent) QGS records were identified. Embase yielded the most records (314; 89 percent). Four databases were needed to retrieve all 337 references: Embase + Health Technology Assessment database + (MEDLINE or PubMed) + Scopus. Four percent (14/351) of records could not be found in any database. Twenty-nine of forty-one (71 percent) reviews reported a reproducible MEDLINE strategy. Ten of twenty-nine (34.5 percent) of the strategies missed at least one QGS record in MEDLINE. Across all twenty-nine MEDLINE searches, 25/143 records were missed (17.5 percent). Mean sensitivity was 89 percent and mean precision was 1.6 percent.

Searching beyond key databases for published EEs may be inefficient, providing the search strategies in those key databases are adequately sensitive. Additional search approaches should be used to identify unpublished evidence (grey literature).

When making decisions in health care, it is essential to consider economic evidence about an intervention. The objective of this study was to analyze the methods applied for systematic reviews of health economic evaluations (SR-HEs) in HTA and to identify common challenges.

We manually searched the Web pages of HTA organizations and included HTA-reports published since 2015. Prerequisites for inclusion were the conduct of an SR-HE in at least one electronic database and the use of the English, German, French, or Spanish language. Methodological features were extracted in standardized tables. We prepared descriptive statistical (e.g., median, range) measures to describe the applied methods. Data were synthesized in a structured narrative way.

Eighty-three reports were included in the analysis. We identified inexplicable heterogeneity, particularly concerning literature search strategy, data extraction, assessment of quality, and applicability. Furthermore, process steps were often missing or reported in a nontransparent way. The use of a standardized data extraction form was indicated in one-third of reports (32 percent). Fifty-four percent of authors systematically appraised included studies. In 10 percent of reports, the applicability of included studies was assessed. Involvement of two reviewers was rarely reported for the study selection (43 percent), data extraction (28 percent), and quality assessment (39 percent).

The methods applied for SR-HEs in HTA and their reporting quality are very heterogeneous. Efforts toward a detailed, standardized guidance for the preparation of SR-HEs definitely seem necessary. A general harmonization and improvement of the applied methodology would increase the value of SR-HE for decision makers.

The HTA Core Model® was developed to improve the transferability of health technology assessment (HTA) between settings. The model has been used by HTA agencies but is also of interest to manufacturers, for improving internal evidence generation and communicating with other HTA stakeholders. To establish if the model is fit for purpose from an industry perspective, the pharmaceutical company Roche, collaborating with the European Network for HTA (EUnetHTA), conducted an assessment of the model.

A questionnaire was developed to evaluate all assessment elements in the HTA Core Model v2.0 for their usefulness in meeting payers’ evidence needs and demonstrating value. The questionnaire was completed by country affiliate teams working in evidence generation and reimbursement submissions for pharmaceuticals. Survey results were discussed in workshops to ensure consistency and alignment between teams.

The questionnaire was completed by six teams. An additional team from global pricing and market access participated in workshops. Model domains pertaining to the health problem and current technology use, technology description, clinical effectiveness, and economic value were considered most important because they meet payers’ evidence needs. Overall, the model was considered useful to improve the efficiency of HTA evidence generation, share evidence internally, and communicate value to payers and HTA agencies.

From an industry perspective, the HTA Core Model provides a useful framework and common terminology for efficient generation of transferable HTA evidence. The timeliness, efficiency, and transparency of HTA processes could be improved by a more standardized approach to HTA across settings.

Integration of ethics into health technology assessment (HTA) remains challenging for HTA practitioners. We conducted a systematic review on social and methodological issues related to ethical analysis in HTA. We examined: (1) reasons for integrating ethics (social needs); (2) obstacles to ethical integration; (3) concepts and processes deployed in ethical evaluation (more specifically value judgments) and critical analyses of formal experimentations of ethical evaluation in HTA.

Search criteria included “ethic,” “technology assessment,” and “HTA”. The literature search was done in Medline/Ovid, SCOPUS, CINAHL, PsycINFO, and the international HTA Database. Screening of citations, full-text screening, and data extraction were performed by two subgroups of two independent reviewers. Data extracted from articles were grouped into categories using a general inductive method.

A list of 1,646 citations remained after the removal of duplicates. Of these, 132 were fully reviewed, yielding 67 eligible articles for analysis. The social need most often reported was to inform policy decision making. The absence of shared standard models for ethical analysis was the obstacle to integration most often mentioned. Fairness and Equity and values embedded in Principlism were the values most often mentioned in relation to ethical evaluation.

Compared with the scientific experimental paradigm, there are no settled proceedings for ethics in HTA nor consensus on the role of ethical theory and ethical expertise hindering its integration. Our findings enable us to hypothesize that there exists interdependence between the three issues studied in this work and that value judgments could be their linking concept.