Objectives: The aim of this project was to identify the ten most important research questions for attention deficit/hyperactivity disorder (ADHD) treatment as identified by people with ADHD together with personnel involved in the treatment of ADHD in school, health, and correction services.

Methods: A working group consisting of consumers and personnel was established. The method for prioritization was primarily based on James Lind Alliance's guidebook, consisting of an interim priority setting exercise and a workshop.

Results: The top ten list includes the risk of drug dependency later in life when treated with methylphenidate as a child, teacher support, multimodal therapy, comparisons between atomoxetine and methylphenidate, methylphenidate treatment in substance abusers, parental support programmes, supported conversation, computer-aided working memory training, psychoeducative treatment, and melatonin.

Conclusions: We have shown that consumers and personnel can reach consensus on research priorities for treatments for ADHD. We encourage researchers and funders to consider the list for future studies.

Objectives: The aim of this study was to develop criteria for the prioritization of topics for health technology assessment (HTA) in the healthcare system of Kazakhstan.

Methods: Initial proposals for criteria were suggested through consultation with Ministry of Health (MoH) policy areas. These were refined through a workshop attended by HTA department staff, persons from medical universities and research institutes, and MoH policy makers. The workshop included discussion on methods used in international HTA practice. Opinions of participants on selection of criteria from those specified in a review of prioritization processes were used to define a list for inclusion in an instrument for routine use. A scoring system was established in later discussion.

Results: Selected criteria for HTA prioritization were burden of disease, availability of alternative technology, clinical effectiveness, economic efficiency, budget impact, and ethical, legal, and/or psychosocial aspects. For each criterion, a health technology under consideration is given a score from 3 (High) to 1 (Low). The total score determines whether the technology is of high to medium priority or of low priority. Determination of priorities for assessment, using the instrument, should be carried out by an expert group appointed by the MoH. The process was applied in 2014 to a selection of topics, and three health technologies were chosen for full assessments.

Conclusions: Criteria for prioritization have evolved with development of the HTA program in Kazakhstan. A method for HTA prioritization has been developed that is easy to apply, requires comparatively few resources, and is compatible with processes required by the MoH.

Objectives: External experts can be consulted at different stages of an HTA. When using vague information sources, it is particularly important to plan, analyze, and report the information processing in a standardized and transparent way. Our objective was to search and analyze recommendations regarding where and how to include expert data in HTA.

Methods: We performed a systematic database search and screened the Internet pages of seventy-seven HTA organizations for guidelines, recommendations, and methods papers that address the inclusion of experts in HTA. Relevant documents were downloaded, and information was extracted in a standard form. Results were merged in tables and narrative evidence synthesis.

Results: From twenty-two HTA organizations, we included forty-two documents that consider the use of expert opinion in HTA. Nearly all documents mention experts in the step of preparation of the evidence report. Six documents address their role for priority setting of topics, fifteen for scoping, twelve for the appraisal of evidence and results, another twelve documents mention experts when considering the dissemination of HTA results.

During the assessment step, experts are most often asked to amend the literature search or to provide expertise for special data analyses. Another issue for external experts is to appraise the HTA results and refer them back to a clinical and social context. Little is reported on methods of expert elicitation when their input substitutes study data.

Conclusions: Despite existing recommendations on the use of expert opinion in HTA, common standards for elicitation are scarce in HTA guidelines.

Objectives: We describe a new evidence-based method for screening and evaluating emerging medical technologies. Washington State agencies, under legislative direction, have granted authority to its agency Medical Directors and policy leaders to make coverage decisions on medical technologies using a “dossier” process. The dossier process is employed when technology advocates or manufacturers request Washington State healthcare purchasers to pay for new and emerging technologies. This offers the advocate an opportunity to submit scientific evidence and information classically associated with a more formal health technology assessment.

Methods: The submitted information is independently reviewed and summarized for Washington State's public healthcare purchasers allowing a more standardized coverage decision for all public purchasers in Washington State.

Results: This process has allowed Washington State to make twelve evidence-based coverage decisions at a fraction of the cost of classic technology assessment. To date, of twelve reviews over 6 years, one health technology was approved for coverage, ten were not covered and one did not require a coverage decision.

Conclusions: This evidence-based dossier process has yielded high-value coverage decisions of new and emerging medical technologies for public healthcare purchasers in Washington State.

Objectives: There is little specific guidance on performing an early cost-effectiveness analysis (CEA) of medical tests. We developed a framework with general steps and applied it to two cases.

Methods: Step 1 is to narrow down the scope of analysis by defining the test's application, target population, outcome measures, and investigating current test strategies and test strategies if the new test were available. Step 2 is to collect evidence on the current test strategy. Step 3 is to develop a conceptual model of the current and new test strategies. Step 4 is to conduct the early-CEA by evaluating the potential (cost-)effectiveness of the new test in clinical practice. Step 5 involves a decision about the further development of the test.

Results: The first case illustrated the impact of varying the test performance on the headroom (maximum possible price) of an add-on test for patients with an intermediate-risk of having rheumatoid arthritis. Analyses showed that the headroom is particularly dependent on test performance. The second case estimated the minimum performance of a confirmatory imaging test to predict individual stroke risk. Different combinations of sensitivity and specificity were found to be cost-effective; if these combinations are attainable, the medical test developer can feel more confident about the value of further development of the test.

Conclusions: A well-designed early-CEA methodology can improve the ability to develop (cost-)effective medical tests in an efficient manner. Early-CEAs should continuously integrate insights and evidence that arise through feedback, which may convince developers to return to earlier steps.

Objectives: The aim of this study was to determine the aspects of expert advice that decision makers find most useful in the development of evidence-based guidance and to identify the characteristics of experts providing the most useful advice.

Methods: First, semi-structured interviews were conducted with seventeen members of the Interventional Procedures Advisory Committee of the UK's National Institute of Health and Care Excellence. Interviews examined the usefulness of expert advice during guidance development. Transcripts were analyzed inductively to identify themes. Second, data were extracted from 211 experts’ questionnaires for forty-one consecutive procedures. Usefulness of advice was scored using an index developed through the qualitative work. Associations between usefulness score and characteristics of the expert advisor were investigated using univariate and multivariate analyses.

Results: Expert opinion was seen as a valued complement to empirical evidence, providing context and tacit knowledge unavailable in published literature, but helpful for interpreting it. Interviewees also valued advice on the training and experience required to perform a procedure, on patient selection criteria and the place of a procedure within a clinical management pathway. Limitations of bias in expert opinion were widely acknowledged and skepticism expressed regarding the anecdotal nature of advice on safety or efficacy outcomes. Quantitative analysis demonstrated that the most useful advice was given by clinical experts with direct personal experience of the procedure, particularly research experience.

Conclusions: Evidence-based guidance production is often characterized as a rational, pipeline process. This ignores the valuable role that expert opinion plays in guidance development, complementing and supporting the interpretation of empirical data.

Objectives: This study assesses the use of routinely collected claims data for managed entry agreements (MEA) in the illustrative context of German statutory health insurance (SHI) funds.

Methods: Based on a nonsystematic literature review, the data needs of different MEA were identified. A value-based typology to classify MEA on the basis of these data needs was developed. The typology is oriented toward health outcomes and utilization and costs, key components of a new technology's value. For each MEA type, the suitability of claims data in establishing evidence of the novel technology's value in routine care was systematically assessed. Assessment criteria were data availability, completeness, timeliness, confidentiality, reliability, and validity.

Results: Claims data are better suited to MEA addressing uncertainty regarding the utilization and costs of a novel technology in routine care. In schemes where safety aspects or clinical effectiveness are assessed, the role of claims data is limited because clinical information is not included in sufficient detail.

Conclusions: The suitability of claims data depends on the source of uncertainty and, in consequence, the outcome measures chosen in the agreements. In all schemes, the validity of claims data should be judged with caution as data are collected for billing purposes. This framework may support manufacturers and payers in selecting the most suitable contract type and agreeing on contract conditions. More research is necessary to validate these results and to address remaining medical, economic, legal, and ethical questions of using claims data for MEA.

Objectives: The regularly structured adaptation of health technology assessment (HTA) programs is of utmost importance to sustain the relevance of the products for stakeholders and to justify investment of scarce financial resources. This study describes internal adjustments and external measures taken to ensure the Horizon Scanning Programme in Oncology (HSO) is current.

Methods: Formal evaluation methods comprising a survey, a download, an environmental analysis, and a Web site questionnaire were used to evaluate user satisfaction.

Results: The evaluation showed that users were satisfied with HSO outputs in terms of timeliness, topics selected, and depth of information provided. Discussion of these findings with an expert panel led to changes such as an improved dissemination strategy and the introduction of an additional output, that is, the publication of a league table of emerging oncology drugs. The rather high level of international usage and the environmental analysis highlighted a considerable overlap in topics assessed and, thus, the potential for international collaboration. As a consequence, thirteen reports were jointly published based on eleven “calls for collaboration.” To further facilitate collaboration and the usability of reports for other agencies, HSO reports will be adjusted according to tools developed at a European level.

Conclusions: Evaluation of the impact of HTA programs allows the tailoring of outputs to fit the needs of the target population. However, within a fast developing HTA community, estimates of impact will increasingly be determined by international collaborative efforts. Refined methods and a broader definition of impact are needed to ultimately capture the efficiency of national HTA programs.

Background: Relative effectiveness has become a key concern of health policy. In Europe, this is because of the need for early information to guide reimbursement and funding decisions about new medical technologies. However, ways that effectiveness (does it work?) and efficacy (can it work?) might differ across health systems are poorly understood.

Methods: This study proposes an analytical framework, drawing on production function theory, to systematically identify and quantify the determinants of relative effectiveness and sources of variation between populations and healthcare systems. We consider how methods such as stochastic frontier analysis and data envelopment analysis using a Malmquist productivity index could in principle be used to generate evidence on, and improve understanding about, the sources of variation in relative effectiveness between countries and over time.

Results: Better evidence on factors driving relative effectiveness could: inform decisions on how to best use a new technology to maximum effectiveness; establish the need if any for follow-up post-launch studies, and provide evidence of the impact of new health technologies on outcomes in different healthcare systems.

Conclusions: The health production function approach for assessment of relative effectiveness is complementary to traditional experimental and observational studies, focusing on identifying, collecting, and analyzing data at the national level, enabling comparisons to take place. There is a strong case for exploring the use of this approach to better understand the impact of new medicines and devices for improvements in health outcomes.

Background: Pharmaceuticals’ relative effectiveness has come to the fore in the policy arena, reflecting the need to understand how relative efficacy (what can work) translates into added benefit in routine clinical use (what does work). European payers and licensing authorities assess value for money and post-launch benefit–risk profiles, and efforts to standardize assessments of relative effectiveness across the European Union (EU) are under way. However, the ways that relative effectiveness differs across EU healthcare settings are poorly understood.

Methods: To understand which factors influence differences in relative effectiveness, we developed an analytical framework that treats the healthcare system as a health production function. Using evidence on breast cancer from England, Spain, and Sweden as a case study, we investigated the reasons why the relative effectiveness of a new drug might vary across healthcare systems. Evidence was identified from a literature review and national clinical guidance.

Results: The review included thirteen international studies and thirty country-specific studies. Cross-country differences in population age structure, deprivation, and educational attainment were consistently associated with variation in outcomes. Screening intensity appeared to drive differences in survival, although the impact on mortality was unclear.

Conclusions: The way efficacy translates into relative effectiveness across health systems is likely to be influenced by a range of complex and interrelated factors. These factors could inform government and payer policy decisions on ways to optimize relative effectiveness, and help increase understanding of the potential transferability of data on relative effectiveness from one health system to another.

Objectives: Equity is one of the founding principles in most healthcare systems. Financial constraints entail an increased risk of exacerbating inequities and a greater need for evidence-based decisions. It is, therefore, both important and timely to enquire how equity can be addressed in health technology assessment (HTA) practice. We aimed to explore related practices from a broad range of HTA agencies, identify exemplary approaches and common concerns, and offer insights for future considerations.

Methods: HTA agencies for which both methodological guides and HTA reports were publicly available were selected from an initial comprehensive pool. Information was extracted on issues ranging from a general commitment to fairness to specific measures targeting both methodological and process-related elements.

Results: Methodological documents and ninety-eight reports from nineteen agencies were analyzed. Our findings indicate that equity was not a standard consideration in HTA report production. The nature of specific approaches and the amount of resources invested into including an equity perspective varied considerably. Specific measures (e.g., appropriate information sources, analytical tools, and schemes) were mentioned by almost half of the agencies analyzed. Albeit sporadic, both horizontal and vertical equity considerations were identified in included HTA reports.

Conclusions: While varying legal contexts and institutional principles can lead to different interpretations of equity at the decision point, a combination of methodological and process-related practices could contribute to more equity-sensitive evaluations, especially in conjunction with enhanced dissemination of existing methodological tools. Networking initiatives on behalf of existing collaborating platforms could play an important role in this direction.

Objectives: The suitability of general HTA methodology for medical devices is gaining interest as a topic of scientific discourse. Given the broad range of medical devices, there might be differences between groups of devices that impact both the necessity and the methods of their assessment. Our aim is to develop a taxonomy that provides researchers and policy makers with an orientation tool on how to approach the assessment of different types of medical devices.

Methods: Several classifications for medical devices based on varying rationales for different regulatory and reporting purposes were analyzed in detail to develop a comprehensive taxonomic model.

Results: The taxonomy is based on relevant aspects of existing classification schemes incorporating elements of risk and functionality. Its 9 × 6 matrix distinguishes between the diagnostic or therapeutic nature of devices and considers whether the medical device is directly used by patients, constitutes part of a specific procedure, or can be used for a variety of procedures. We considered the relevance of different device categories in regard to HTA to be considerably variable, ranging from high to low.

Conclusions: Existing medical device classifications cannot be used for HTA as they are based on different underlying logics. The developed taxonomy combines different device classification schemes used for different purposes. It aims at providing decision makers with a tool enabling them to consider device characteristics in detail across more than one dimension. The placement of device groups in the matrix can provide decision support on the necessity of conducting a full HTA.

Objectives: Many of the currently used health technologies have never been systematically assessed or are misused, overused or superseded. Therefore, they may be ineffective. Active identification of ineffectiveness in health care is gaining importance to facilitate best care for patients and optimal use of limited resources. The present research analyzed processes and experiences of programs for identifying ineffective health technologies. The goal of this study was to elucidate factors that facilitate implementation.

Methods: Based on an overview article, a systematic literature search and unsystematic hand-search were conducted. Further information was gained from international experts.

Results: Seven programs were identified that include identification, prioritization and assessment of ineffective health technologies and dissemination of recommendations. The programs are quite similar regarding their goals, target groups and criteria for identification and prioritization. Outputs, mainly HTA reports or lists, are mostly disseminated by means of the internet. Top–down and bottom–up programs both have benefits in terms of implementation of recommendations, either as binding guidelines and decisions or as nonbinding information for physicians and other stakeholders. Crucial facilitators of implementation are political will, transparent processes and broad stakeholder involvement focusing on physicians.

Conclusions: All programs can improve the quality of health care and enable cost reduction in supportive surrounding conditions. Physicians and patients must be continuously involved in the process of evaluating health technologies. Additionally, decision makers must support programs and translate recommendations into concrete actions.

Objectives: The headroom approach to medical device development relies on the estimation of a value-based price ceiling at different stages of the development cycle. Such price-ceilings delineate the commercial opportunities for new products in many healthcare systems. We apply a simple model to obtain critical business information as the product proceeds along a development pathway, and indicate some future directions for the development of the approach.

Methods: Health economic modelling in the supply-side development cycle for new products.

Results: The headroom can be used: initially as a ‘reality check’ on the viability of the device in the healthcare market; to support product development decisions using a real options approach; and to contribute to a pricing policy which respects uncertainties in the reimbursement outlook.

Conclusions: The headroom provides a unifying thread for business decisions along the development cycle for a new product. Over the course of the cycle attitudes to uncertainty will evolve, based on the timing and manner in which new information accrues. Within this framework the developmental value of new information can justify the costs of clinical trials and other evidence-gathering activities. Headroom can function as a simple shared tool to parties in commercial negotiations around individual products or groups of products. The development of similar approaches in other contexts holds promise for more rational planning of service provision.

Objectives: Model for ASsessment of Telemedicine Applications (MAST) is a health technology assessment (HTA) inspired framework for assessing the effectiveness and contribution to quality of telemedicine applications based on rigorous, scientific data. This study reports from a study of how it was used and perceived in twenty-one pilots of the European project RENEWING HEALTH (RH). The objectives of RH were to implement large-scale, real-life test beds for the validation and subsequent evaluation of innovative patient-centered telemedicine services. The study is a contribution to the appraisal of HTA methods.

Methods: A questionnaire was administered for project leaders of the pilots. It included questions about use and usefulness of MAST for (i) preceding considerations, (ii) evaluation of outcomes within seven domains, and (iii) considerations of transferability. Free text spaces allowed for proposals of improvement. The responses covered all pilots. A quantitative summary of use and a qualitative analysis of usefulness were performed.

Results: MAST was used and considered useful for pilot evaluations. Challenges included problems to scientifically determine alternative service options and outcome within the seven domains. Proposals for improvement included process studies and adding domains of technological usability, responsible innovation, health literacy, behavior change, caregiver perspectives and motivational issues of professionals.

Conclusions: MAST was used according to its structure. Its usefulness in patient centered pilots can be improved by adding new stakeholder groups. Interdependencies between scientific rigor, resources and timeliness should be addressed. Operational options for improvements include process studies, literature reviews and sequential mini-HTAs for identification of areas for more elaborate investigations.

Background: “Mapping” onto generic preference-based outcome measures is increasingly being used as a means of generating health utilities for use within health economic evaluations. Despite publication of technical guides for the conduct of mapping research, guidance for the reporting of mapping studies is currently lacking. The MAPS (MApping onto Preference-based measures reporting Standards) statement is a new checklist, which aims to promote complete and transparent reporting of mapping studies.

Methods: In the absence of previously published reporting checklists or reporting guidance documents, a de novo list of reporting items was created by a working group comprised of six health economists and one Delphi methodologist. A two-round, modified Delphi survey with representatives from academia, consultancy, health technology assessment agencies, and the biomedical journal editorial community was used to identify a list of essential reporting items from this larger list.

Results: From the initial de novo list of twenty-nine candidate items, a set of twenty-three essential reporting items was developed. The items are presented numerically and categorized within six sections, namely: (i) title and abstract, (ii) introduction, (iii) methods, (iv) results, (v) discussion, and (vi) other. The MAPS statement is best applied in conjunction with the accompanying MAPS explanation and elaboration document.

Conclusions: It is anticipated that the MAPS statement will improve the clarity, transparency. and completeness of reporting of mapping studies. To facilitate dissemination and uptake, the MAPS statement is being co-published by seven health economics and quality of life journals, and broader endorsement is encouraged. The MAPS working group plans to assess the need for an update of the reporting checklist in five years’ time.

Objectives: The objective of this study was to explore barriers and facilitators influencing the integration of ethical considerations in health technology assessment (HTA).

Methods: The study consisted of two complementary approaches: (a) a systematic review of the literature; and (b) an eighteen-item online survey that was distributed to fifty-six HTA agencies affiliated with the International Network of Agencies for Health Technology Assessment.

Results: The review identified twenty-six relevant articles. The most often cited barriers in the literature were: scarcity, heterogeneity and complexity of ethical analysis methods; challenges in translating ethical analysis results into knowledge that is useful for decision makers; and lack of organizational support in terms of required expertise, time and financial resources. The most frequently cited facilitators included: usage of value-based appraisal methods, stakeholder and public engagement, enhancement of practice guidelines, ethical expertise, and educational interventions.

Representatives of twenty-six (46.5 percent) agencies from nineteen countries completed the survey. A median of 10 percent (interquartile range, 5 percent to 50 percent) of the HTA products produced by the agencies was reported to include an assessment of ethical aspects. The most commonly perceived barriers were: limited ethical knowledge and expertise, insufficient time and resources, and difficulties in finding ethical evidence or using ethical guidelines. Educational interventions, demand by policy makers, and involvement of ethicists in HTA were the most commonly perceived facilitators.

Conclusions: Our results emphasize the importance of simplification of ethics methodology and development of good practice guidelines in HTA, as well as capacity building for engaging HTA practitioners in ethical analyses.

Objectives: The objective of this study was to compare evidence requirements for health technology assessment of pharmaceuticals by national agencies across Europe responsible for reimbursement decisions focusing specifically on relative effectiveness assessment.

Methods: Evidence requirements from thirty-three European countries were requested and twenty-nine national agencies provided documents to review. Data were extracted from national documents (manufacturer's submission templates and associated guidance) into a purpose-made framework with categories covering information about the health condition, the technology, clinical effectiveness and safety.

Results: The level of detail in the required evidence varies considerably across countries. Some countries include specific questions while others request information under general headings. Some countries include all information in a single document, which may or may not include guidance on how to complete the template. Others have specific guidance documents or methods and process manuals that help with the completion of the submission templates. Despite differences in quantity and detail, the content of the evidence requirements is broadly similar. All countries ask for information on the health technology, target disease, and clinical effectiveness and safety. However, one country only requests clinical effectiveness information as part of cost-effectiveness analyses. We found twenty-six evidence requirements for which generic answers may apply across borders and nineteen in which countries requested nationally specific information.

Conclusions: This work suggests that it would be possible to put together a minimum set of evidence requirements for HTA to support reimbursement decisions across Europe which could facilitate collaboration between jurisdictions.

Objectives: Our objective was to gather perspectives from payers on how comparative effectiveness research (CER) in the United States and relative effectiveness (RE) research in Europe will impact evidentiary standards for access decisions of new drugs by 2020.

Methods: We conducted semi-structured interviews with fourteen senior officials representing public and private payers, health technology assessment groups, and pricing and reimbursement bodies in the United States and Europe. An online survey assessed current use of CER/RE evidence and potential trends that might influence its use for decision making by 2020. A semi-structured interview elicited payers' definitions of CER/RE and was structured around four hypothetical cases resembling drugs expected to be more common or poised to create policy challenges by 2020. Topics included acceptance of study designs and analytic methods associated with CER/RE. A systematic content review was done to extract relevant information.

Results: According to key informants, randomization will remain an essential component for assessing comparative or relative effectiveness. They anticipate greater use of policy levers such as conditional reimbursement or prior authorization to manage diffusion of new drugs. Case studies provided important insights into situations when certain types of CER evidence may be acceptable (e.g., observational data when differences between drugs are largely convenience).

Conclusions: Industry perceptions that CER/RE will change payers' evidentiary requirements in the future are consistent with our findings. Growing investment in payers' own data and increased reliance on policy tools to control diffusion of new drugs may also influence the type of evidence industry will be required to produce by 2020.

Objectives: Assessment of ethical aspects of a technology is an important component of health technology assessment (HTA). Nevertheless, how the implementation of ethical assessment in HTA is to be organized and adapted to specific regulatory and organizational settings remains unclear. The objective of this study is to present a framework for systematic identification of ethical aspects of health technologies. Furthermore, the process of developing and adapting the framework to a specific setting is described.

Methods: The framework was developed based on an inventory of existing approaches to identification and assessment of ethical aspects in HTA. In addition, the framework was adapted to the Swedish legal and organizational healthcare context, to the role of the HTA agency and to the use of non-ethicists. The framework was reviewed by a group of ethicists working in the field as well as by a wider set of interested parties including industry, interest groups, and other potential users.

Results: The framework consists of twelve items with sub-questions, short explanations, and a concluding overall summary. The items are organized into four different themes: the effects of the intervention on health, its compatibility with ethical norms, structural factors with ethical implications, and long term ethical consequences of using the intervention.

Conclusions: In this study, a framework for identifying ethical aspects of health technologies is proposed. The general considerations and methodological approach to this venture will hopefully inspire and present important insights to organizations in other national contexts interested in making similar adaptations.