Objectives: The aim of this study was to determine the aspects of expert advice that decision makers find most useful in the development of evidence-based guidance and to identify the characteristics of experts providing the most useful advice.

Methods: First, semi-structured interviews were conducted with seventeen members of the Interventional Procedures Advisory Committee of the UK's National Institute of Health and Care Excellence. Interviews examined the usefulness of expert advice during guidance development. Transcripts were analyzed inductively to identify themes. Second, data were extracted from 211 experts’ questionnaires for forty-one consecutive procedures. Usefulness of advice was scored using an index developed through the qualitative work. Associations between usefulness score and characteristics of the expert advisor were investigated using univariate and multivariate analyses.

Results: Expert opinion was seen as a valued complement to empirical evidence, providing context and tacit knowledge unavailable in published literature, but helpful for interpreting it. Interviewees also valued advice on the training and experience required to perform a procedure, on patient selection criteria and the place of a procedure within a clinical management pathway. Limitations of bias in expert opinion were widely acknowledged and skepticism expressed regarding the anecdotal nature of advice on safety or efficacy outcomes. Quantitative analysis demonstrated that the most useful advice was given by clinical experts with direct personal experience of the procedure, particularly research experience.

Conclusions: Evidence-based guidance production is often characterized as a rational, pipeline process. This ignores the valuable role that expert opinion plays in guidance development, complementing and supporting the interpretation of empirical data.

Objective: As health technology assessment (HTA) organizations in Canada and around the world seek to involve the public and patients in their activities, frameworks to guide decisions about whom to involve, through which mechanisms, and at what stages of the HTA process have been lacking. The aim of this study was to describe the development and outputs of a comprehensive framework for involving the public and patients in a government agency's HTA process.

Methods: The framework was informed by a synthesis of international practice and published literature, a dialogue with local, national and international stakeholders, and the deliberations of a government agency's public engagement subcommittee in Ontario, Canada.

Results: The practice and literature synthesis failed to identify a single, optimal approach to involving the public and patients in HTA. Choice of methods should be considered in the context of each HTA stage, goals for incorporating societal and/or patient perspectives into the process, and relevant societal and/or patient values at stake. The resulting framework is structured around four actionable elements: (i) guiding principles and goals for public and patient involvement (PPI) in HTA, (ii) the establishment of a common language to support PPI efforts, (iii) a flexible array of PPI approaches, and (iv) on-going evaluation of PPI to inform adjustments over time.

Conclusions: A public and patient involvement framework has been developed for implementation in a government agency’s HTA process. Core elements of this framework may apply to other organizations responsible for HTA and health system quality improvement.

Objectives: Rapid reviews are characterized as an accelerated evidence synthesis approach with no universally accepted methodology or definition. This modified Delphi consensus study aimed to develop a comprehensive set of defining characteristics for rapid reviews that may be used as a functional definition.

Methods: Expert panelists with knowledge in rapid reviews and evidence synthesis were identified. In the first round, panelists were asked to answer a seventeen-item survey addressing a variety of rapid review topics. Results led to the development of statements describing the characteristics of rapid reviews that were circulated to experts for agreement in a second survey round and further revised in a third round. Consensus was reached if ≥70 percent of experts agreed and there was stability in free-text comments.

Results: A panel of sixty-six experts participated. Consensus was reached on ten of eleven statements describing the characteristics of rapid reviews. According to the panel, rapid reviews aim to meet the requirements and timelines of a decision maker and should be conducted in less time than a systematic review. They use a variety of approaches to accelerate the evidence synthesis process, tailor the methods conventionally used to carry out systematic reviews, and use the most rigorous methods that the delivery time frame will allow.

Conclusions: This study achieved consensus on ten statements describing the defining characteristics of rapid reviews based on the opinion of a panel of knowledgeable experts. Areas of disagreement were also highlighted. Findings emphasize the role of the decision maker and stress the importance of transparent reporting.

Objectives: This study assesses the use of routinely collected claims data for managed entry agreements (MEA) in the illustrative context of German statutory health insurance (SHI) funds.

Methods: Based on a nonsystematic literature review, the data needs of different MEA were identified. A value-based typology to classify MEA on the basis of these data needs was developed. The typology is oriented toward health outcomes and utilization and costs, key components of a new technology's value. For each MEA type, the suitability of claims data in establishing evidence of the novel technology's value in routine care was systematically assessed. Assessment criteria were data availability, completeness, timeliness, confidentiality, reliability, and validity.

Results: Claims data are better suited to MEA addressing uncertainty regarding the utilization and costs of a novel technology in routine care. In schemes where safety aspects or clinical effectiveness are assessed, the role of claims data is limited because clinical information is not included in sufficient detail.

Conclusions: The suitability of claims data depends on the source of uncertainty and, in consequence, the outcome measures chosen in the agreements. In all schemes, the validity of claims data should be judged with caution as data are collected for billing purposes. This framework may support manufacturers and payers in selecting the most suitable contract type and agreeing on contract conditions. More research is necessary to validate these results and to address remaining medical, economic, legal, and ethical questions of using claims data for MEA.

Objectives: The aim of this project was to identify the ten most important research questions for attention deficit/hyperactivity disorder (ADHD) treatment as identified by people with ADHD together with personnel involved in the treatment of ADHD in school, health, and correction services.

Methods: A working group consisting of consumers and personnel was established. The method for prioritization was primarily based on James Lind Alliance's guidebook, consisting of an interim priority setting exercise and a workshop.

Results: The top ten list includes the risk of drug dependency later in life when treated with methylphenidate as a child, teacher support, multimodal therapy, comparisons between atomoxetine and methylphenidate, methylphenidate treatment in substance abusers, parental support programmes, supported conversation, computer-aided working memory training, psychoeducative treatment, and melatonin.

Conclusions: We have shown that consumers and personnel can reach consensus on research priorities for treatments for ADHD. We encourage researchers and funders to consider the list for future studies.

Objectives: Treatment switching refers to the situation in a randomized controlled trial where patients switch from their randomly assigned treatment onto an alternative. Often, switching is from the control group onto the experimental treatment. In this instance, a standard intention-to-treat analysis does not identify the true comparative effectiveness of the treatments under investigation. We aim to describe statistical methods for adjusting for treatment switching in a comprehensible way for nonstatisticians, and to summarize views on these methods expressed by stakeholders at the 2014 Adelaide International Workshop on Treatment Switching in Clinical Trials.

Methods: We describe three statistical methods used to adjust for treatment switching: marginal structural models, two-stage adjustment, and rank preserving structural failure time models. We draw upon discussion heard at the Adelaide International Workshop to explore the views of stakeholders on the acceptability of these methods.

Results: Stakeholders noted that adjustment methods are based on assumptions, the validity of which may often be questionable. There was disagreement on the acceptability of adjustment methods, but consensus that when these are used, they should be justified rigorously. The utility of adjustment methods depends upon the decision being made and the processes used by the decision-maker.

Conclusions: Treatment switching makes estimating the true comparative effect of a new treatment challenging. However, many decision-makers have reservations with adjustment methods. These, and how they affect the utility of adjustment methods, require further exploration. Further technical work is required to develop adjustment methods to meet real world needs, to enhance their acceptability to decision-makers.

Objectives: The aim of this study was to implement health technology assessment (HA) in the First General City Hospital in Astana, Kazakhstan.

Methods: We organized trainings to familiarize hospital staff with the purpose and details of HTA. An HTA committee was established, with representation from hospital physicians and managers, and criteria for prioritization of health technologies determined. Clinical departments of the hospital were asked to prepare applications for new technologies for their services.

Results: The HTA committee reviewed five applications and selected a technology from one of these, on single incision laparoscopic surgery (SILS), for assessment. A short HTA report on SILS was prepared, covering its safety, clinical effectiveness, and cost effectiveness. The report was used to support a request to the Department of Health for additional funding to implement this technology within the hospital. This funding was approved and SILS was established in several hospital departments.

Conclusions: This successful initial experience with HTA has paved the way for its routine use by the hospital for informing decisions on the procurement and use of new health technologies.

Objectives: Rapid reviews can be conducted in a narrower time frame, as compared to systematic reviews, by featuring restrictions. To estimate the validity of the results, assessment of methodological quality is required. Our aim was to analyze the methodological restrictions of rapid reviews compared with systematic reviews using the AMSTAR checklist and assess its feasibility for rapid reviews.

Methods: A systematic search for literature on rapid reviews of surgical interventions was conducted in three databases: Medline, Embase, and the Cochrane library. Additionally, health technology assessment (HTA) databases were searched. We analyzed reviews using AMSTAR and additionally compared the results with those of an overview of reviews on the same topic.

Results: Items found more frequently in rapid reviews were search for gray literature (65 percent versus 33 percent), listing of excluded studies (59 percent versus 37 percent), and provision of study characteristics (77 percent versus 44 percent), whereas consideration of study quality in formulating conclusions, conduct of meta-analysis, and statement of conflicts of interest were less frequent. Median time between search and publication was 8 months, with a range between 1 and 27.

Conclusions: With some adjustments, AMSTAR can be used as a checklist for rapid reviews to describe methodological restrictions in comparison to systematic reviews and to roughly estimate the validity of the results. Strikingly, only 14.3 percent of rapid reviews were published within 3 months.

Objectives: Effective formulary management in healthcare institutions safeguards rational drug use and optimizes health outcomes. We implemented a formulary management program integrating the principles of health technology assessment (HTA) to improve the safe, appropriate, and cost-effective use of medicine in Singapore.

Methods: A 3-year formulary management program was initiated in 2011 in five public healthcare institutions. This program was managed by a project team comprising HTA researchers. The project team worked with institutional pharmacy and therapeutics (P&T) committees to: (i) develop tools for formulary drug review and decision making; (ii) enhance the HTA knowledge and skills of formulary pharmacists and members of P&T committees; (iii) devise a prioritization framework to overcome resource constraints and time pressure; and (iv) conceptualize and implement a framework to review existing formulary.

Results: Tools that facilitate drug request submission, drug review, and decision making were developed for formulary drug inclusion. A systematic framework to review existing formulary was also developed and tested in selected institutions. A competency development plan was rolled out over 2 years to enhance formulary pharmacists’ proficiency in systematic literature search and review, meta-analysis, and pharmacoeconomic evaluation. The plan comprised training workshops and on-the-job knowledge transfer between the project team and institutional formulary pharmacists through collaborating on selected drug reviews. A resource guide that consolidated the tools and templates was published to encourage the adoption of best practices in formulary management.

Conclusions: Based on the concepts of HTA, we implemented an evidence-based approach to optimize formulary management.

Background: Cardiovascular disease is one of the most common causes of death worldwide, with many individuals having experienced acute coronary syndrome (ACS). How patients with a history of ACS value aspects of their medical treatment have been evaluated rarely. The aim of this study was to determine patient priorities for long-term drug therapy after experiencing ACS.

Methods: To identify patient-relevant treatment characteristics, a systematic literature review and qualitative patient interviews were conducted. A questionnaire was developed to elicit patient's priorities for different characteristics of ACS treatment using Analytic Hierarchy Process (AHP). To evaluate the patient-relevant outcomes, the eigenvector method was applied.

Results: Six-hundred twenty-three patients participated in the computer-assisted personal interviews and were included in the final analysis. Patients showed a clear priority for the attribute “reduction of mortality risk” (weight: 0.402). The second most preferred attribute was the “prevention of a new myocardial infarction” (weight: 0.272), followed by “side effect: dyspnea” (weight: 0.165) and “side effect: bleeding” (weight: 0.117). The “frequency of intake” was the least important attribute (weight: 0.044).

Conclusion: In conclusion, this study shows that patients strongly value a reduction of the mortality risk in post-ACS treatment. Formal consideration of patient preferences and priorities can help to inform a patient-centered approach, clinical practice, development of future effective therapies, and health policy for decision makers that best represents the needs and goals of the patient.

Objectives: Treatment switching occurs when patients in a randomized clinical trial switch from the treatment initially assigned to them to another treatment, typically from the control to experimental treatment. This study discusses the issues this raises and possible approaches to addressing them in trials of cancer drugs.

Methods: Stakeholders from around the world were invited to a 1.5-day Workshop in Adelaide, Australia. This study attempts to capture the key points from the discussion and the perspectives of the various stakeholder groups, but is not a formal consensus statement.

Results: Treatment switching raises challenging ethical issues with arguments for and against allowing it. It is increasingly common in cancer drug trials and presents challenges for the interpretation of results by regulators, clinicians, patients, and payers. Proposals are offered for good practice in the design, management, and analysis of trials and wider development programs for cancer drugs in which treatment switching has occurred or is likely to. Recommendations are also offered for further action to improve understanding of the importance and challenges of treatment switching and to promote agreement between key stakeholders on guidelines and other steps to address these challenges.

Conclusions: The handling of treatment switching in trials is of concern to all stakeholders. On the basis of the discussions at the Adelaide International Workshop, there would appear to be common ground on approaches to addressing treatment switching in cancer trials and scope for the development of formal guidelines to inform the work of regulators, payers, industry, trial designers and other stakeholders.

Objectives: Canada has witnessed expansion of the health technology assessment (HTA) infrastructure in the last 25 years. Local HTA entities at the hospital or regional level are emerging to assist decision makers in the acquisition, implementation, maintenance, and disinvestment of healthcare technologies. There is a need to facilitate collaboration and exchange of expertise and knowledge between these entities regarding the role of local HTA in Canada.

Methods: In November 2013, the pan-Canadian Collaborative hosted a symposium, Hospital/Regional HTA: Local Evidence-based Decisions for Health Care Sustainability, bringing together over 60 HTA producers, researchers, stakeholders, and manufacturers involved in local HTA across Canada. The objective was to showcase the diversity of local HTA in Canada, while highlighting common gaps to be addressed.

Results: The Symposium focused on current practices in local HTA in Canada to support informed decision making, and opportunities for information sharing and provide equal access to timely evidence-based information to decision makers. The main themes included assessment of evidence for local HTA, contextualization, stakeholder engagement in local HTA, knowledge translation and impact of recommendations, and challenges and opportunities for local HTA.

Conclusions: Local HTA in Canada complements HTAs conducted at the provincial and federal levels to improve the efficient and effective health service delivery in institutions or regions faced with limited resources. Some challenges faced by local HTA producers to influence hospital policies and clinical practice involve the engagement of healthcare professionals and potential lack of training and support necessary for the introduction of a new technology.

Objectives: Health technology assessment (HTA) yields information that can be ideally used to address deficiencies in health systems and to create a wider understanding of the impact of different policy considerations around technology reimbursement and use. The structure of HTA programs varies across different jurisdictions according to decision-maker needs. Moreover, conducting HTA requires specialized skills. Effective decision making should include multiple criteria (medical, economic, technical, ethical, social, legal, and cultural) and requires multi-disciplinary teams of experts working together to produce these assessments. A workshop explored the multi-disciplinary skills and competencies required to build an effective and efficient HTA team, with a focus on low- and middle-income settings.

Methods: This proceeding summarizes main points from a workshop on capacity building, drawing on presentations and group discussions among attendees including different points of view.

Results and Conclusions: The workshop and thus this study would have benefited from a larger variety of stakeholders. Therefore, the conclusions arising from the workshop are not the opinion of a representative sample of HTA professionals. Nonetheless, organizations and speakers were carefully selected to provide a valuable approach to this theme. Thus, these proceedings highlight some of the gaps and needs in the education and training programs offered worldwide and calls for further investigation.

Owing to a copyediting error, Marjukka Mäkelä was incorrectly listed as an author in the editorial by Oortwijn & van der Wilt (1) in February/April 2016 issue of International Journal of Technology Assessment in Health Care. The publisher apologizes to the author and readers for this error. The original article has been corrected online to rectify this error.

Background: Health technology assessment (HTA) examines the consequences of the application of health technologies and is aimed at better informing decision-makers. Over the past 30 years, different countries have implemented HTA organizations. Colombia established by law its own HTA agency (IETS) in 2011 which started operations in November 2012. The aim of this study was to assess the feasibility of conducting and using HTA to inform decision-making in this context. Through a qualitative approach, ten “drivers” emerged with the ability to help or hinder HTA development in this context: availability and quality of data, implementation strategy, cultural aspects, local capacity, financial support, policy/political support, globalization, stakeholder pressure, health system context, and usefulness perception.

Methods: Semi-structured interviews were conducted with key HTA researchers, after following rigorous transcription, and thematic content analysis, those aspects that may be barriers or facilitator for HTA development and use in Colombia were identified.

Results: Although HTA has become a tool to inform decision-making around the world, its use may vary according to setting. Determining those aspects which may enable or interfere with HTA development and use in Colombia may be useful for other countries when considering the establishment of HTA systems.

Conclusions: The conceptual transferability of concepts like “drivers” with caveats may be of interest for similar settings trying to incorporate HTA processes and institutions into systematic decision-making.

Objectives: This study was carried out to evaluate the opinions of stakeholders on their roles in health technology assessment (HTA) in Iran and to determine the barriers and facilitators existing in the organizations to help increase their involvement in the HTA program.

Methods: The study was conducted in two stages, semi-structured interviews, and “policy dialogue” with stakeholders. The data were analyzed through the framework approach.

Results: The interviews were held with ten stakeholder representatives from various organizations. In addition, Twenty-one representatives participated in the policy dialogue. Based on the findings, all the stakeholder organizations considered themselves as interest groups in all the stages of the HTA process; however, their tendencies and methods of involvement differed from one another. According to the participants, the most important issue to be considered in the context of HTA was that the structures, stages, and procedures of the HTA process must be made transparent.

Conclusions: Stakeholder involvement in the HTA program cannot readily take place. Various stakeholders have different interests, responsibilities, infrastructures, and barriers. If a program does not meet these considerations, its chances of succeeding will substantially decrease. Therefore, to prevent overlooking the needs and expectations of stakeholders from the HTA process, it is essential to create opportunities in which their thoughts and ideas are taken into account.

Objectives: In 2012, Colombia experienced an important institutional transformation after the establishment of the Health Technology Assessment Institute (IETS), the disbandment of the Regulatory Commission for Health and the reassignment of reimbursement decision-making powers to the Ministry of Health and Social Protection (MoHSP). These dynamic changes provided the opportunity to test Multi-Criteria Decision Analysis (MCDA) for systematic and more transparent resource-allocation decision-making.

Methods: During 2012 and 2013, the MCDA framework Evidence and Value: Impact on Decision Making (EVIDEM) was tested in Colombia. This consisted of a preparatory stage in which the investigators conducted literature searches and produced HTA reports for four interventions of interest, followed by a panel session with decision makers. This method was contrasted with a current approach used in Colombia for updating the publicly financed benefits package (POS), where narrative health technology assessment (HTA) reports are presented alongside comprehensive budget impact analyses (BIAs).

Results: Disease severity, size of population, and efficacy ranked at the top among fifteen preselected relevant criteria. MCDA estimates of technologies of interest ranged between 71 to 90 percent of maximum value. The ranking of technologies was sensitive to the methods used. Participants considered that a two-step approach including an MCDA template, complemented by a detailed BIA would be the best approach to assist decision-making in this context. Participants agreed that systematic priority setting should take place in Colombia.

Conclusions: This work may serve as the basis to the MoHSP on its interest of setting up a systematic and more transparent process for resource-allocation decision-making.