Objectives: Gender as a social construct is a recognized health determinant. Because best practice in reporting health technology assessment (HTA) clearly specifies the need to appraise a technology's social impact within the target population, the extent to which gender issues are taken into account in HTA production is of interest, not only in light of equitable practices but also for reasons of effectiveness. The aim of this study is to provide a first assessment of the degree of gender sensitivity shown by HTA agencies around the world today.

Methods: The Web sites of sixty HTA agencies were analyzed. The consideration of gender aspects was specifically looked for in each agency's general mission statement, its priority setting process, and its methodological approach. Additionally, specific gender-oriented initiatives not belonging to any of the aforementioned categories were identified.

Results: Of the sixty agencies, less than half mention a commitment to addressing the social implication of health technologies. Only fifteen institutions make information on their priority setting principles available on their Web sites and gender was an issue in two of those cases. Data on methodology were obtainable online from18 agencies, two of which mentioned gender issues explicitly. Finally, gender-oriented initiatives were identified by thirteen agencies.

Conclusions: A gender-sensitive approach is apparently rarely adopted in current HTA production. Exceptional practices and relevant tools do exist and could serve as examples to be promoted by international collaborative networks.

Objectives: This study explores the factors that enhance or reduce the prospects for public involvement in the activities of health technology assessment (HTA) agencies.

Methods: The analytical framework for this study is based on the work of John W. Kingdon, which provides a comprehensive synthesis of the factors influencing governments and public organizations' agenda. The study draws insights from forty-two semistructured telephone interviews with informants involved in international HTA networks and/or in HTA agencies in Canada, Denmark, and the United Kingdom.

Results: This exploratory study suggests that the HTA community is moving toward greater public involvement. However, the HTA community remains cautious and ambivalent about the technical feasibility of public involvement, its acceptability to policy makers and practitioners, and its impacts on HTA agencies' resources and procedures.

Conclusions: The study stresses the importance of conducting rigorous and compelling evaluations to inform HTA agencies' decision to adopt, or reject, public involvement practices.

Objectives: Ethical analysis can highlight important ethical issues related to implementing a technology, values inherent in the technology itself, and value-decisions underlying the health technology assessment (HTA) process. Ethical analysis is a well-acknowledged part of HTA, yet seldom included in practice. One reason for this is lack of knowledge about the properties and differences between the methods available. This study compares different methods for ethical analysis within HTA.

Methods: Ethical issues related to bariatric (obesity) surgery were independently evaluated using axiological, casuist, principlist, and EUnetHTA models for ethical analysis within HTA. The methods and results are presented and compared.

Results: Despite varying theoretical underpinnings and practical approaches, the four methods identified similar themes: personal responsibility, self-infliction, discrimination, justice, public funding, and stakeholder involvement. The axiological and EUnetHTA models identified a wider range of arguments, whereas casuistry and principlism concentrated more on analyzing a narrower set of arguments deemed more important.

Conclusions: Different methods can be successfully used for conducting ethical analysis within HTA. Although our study does not show that different methods in ethics always produce similar results, it supports the view that different methods of ethics can yield relevantly similar results. This suggests that the key conclusions of ethical analyses within HTA can be transferable between methods and countries. The systematic and transparent use of some method of ethics appears more important than the choice of the exact method.

Objectives: The aim of this study was to explore whether Bayesian reasoning can be applied to therapeutic questions in a way that is similar to its application in diagnostics.

Methods: A clinically relevant, therapeutic question was formulated in accordance with Bayesian reasoning for the clinical management of patients with newly diagnosed rheumatoid arthritis (RA). Prior probability estimates of response to drug treatment (methotrexate, MTX) were obtained from the literature. As a marker of treatment response, changes in the Health Assessment Questionnaire (HAQ) scores were assessed after three months of treatment. Likelihood ratios for this marker were calculated on the basis of data from a clinical registry, using changes in the Disease Activity Score (DAS) as gold standard. Using Bayes’ theorem, prior probability and likelihood ratios were combined to estimate posterior probabilities of treatment response in individual patients.

Results: On the basis of the literature, the prior probability of response of RA patients to MTX was estimated 45 percent. At 3 months follow-up, this probability increased to 80 percent or decreased to 23 percent, depending on the changes that were observed in Health Assessment Questionnaire scores.

Conclusions: Bayesian reasoning can be applied to therapeutic issues in a way that is conceptually fully compatible with its use in diagnostics. As such, it can be used to bridge the gap between aggregate data and individual patient management.

Objectives: The role and impact of health technology assessment (HTA) in health policy has been widely discussed. Researchers have started to analyze how decisions on coverage of new technologies are made. Although the involvement of HTA may be an indicator of a well established decision process, this hypothesis requires validation. Also, it is not known whether HTA involvement is associated with other characteristics of decision making like participation or transparency. The primary objective of this study was to develop and test statements on the association between the publication of an HTA and coverage decision making for newborn screening tests in European Union countries.

Methods: Five statements were defined on the relative role of HTA during the steps of decision processes: trigger, participation, publication, assessment, and appraisal. For this purpose, data on twenty-two decision processes in the area of newborn screening across Europe were analyzed, defined as a coverage decision for a given disorder in a specific country. Decision processes were compared by whether the decision was accompanied by the publication of an HTA report. To test differences, nonparametric statistical tests were used.

Results: The decision steps of trigger, participation and publication differed between the HTA and the non-HTA groups. No clear association between HTA and assessment methods in coverage decision making was identified.

Conclusions: It appeared that there is an association between HTA and coverage decision processes that are more explicit, inclusive, and transparent. It is unclear whether HTA is associated with formal evidence reviews and economic evaluations.

Objectives: Values are intrinsic to the use of health technology assessments (HTAs) in health policy, but neglecting value assumptions in HTA makes their results appear more robust or normatively neutral than may be the case. Results of a 2003 survey by the International Network of Agencies for Health Technology Assessment (INAHTA) revealed the existence of disparate methods for making values and ethical issues explicit when conducting HTA.

Methods: An Ethics Working Group, with representation from sixteen agencies, was established to develop a framework for addressing ethical issues in HTA. Using an iterative approach, with email exchanges and face-to-face workshops, a report on Handling Ethical Issues was produced.

Results: This study describes the development process and the agreed upon framework for reflexive ethical analysis that aims to uncover and explore the ethical implications of technologies through an integrated, context-sensitive approach and situates the proposed framework within previous work in the development of ethics analysis in HTA.

Conclusions: It is important that methodological approaches to address ethical reflection in HTA be integrative and context sensitive. The question-based approach described and recommended here is meant to elicit this type of reflection in a way that can be used by HTA agencies. The questions proposed are considered only as a starting point for handling ethics issues, but their use would represent a significant improvement over much of the existing practice.

Background: Most developing countries and resource-limited settings lack robust health technology assessment (HTA) systems. Because the development of locally relevant HTA is not immediately viable, and the extrapolation of external HTA is inappropriate, a new model for evaluating health technologies is required.

Objectives: The aim of this study was to describe the development and application of KNOW ESSENTIALS, a tool facilitating evidence-based decisions on health technologies by stakeholders in settings lacking formal HTA systems.

Methods: Current HTA methodology was examined through literature search. Additional issues relevant to resource-limited settings, but not adequately addressed in current methodology, were identified through further literature search, appraisal of contextually relevant issues, discussion with healthcare professionals familiar with the local context, and personal experience. A set of thirteen elements important for evidence-based decisions was identified, selected and combined into a tool with the mnemonic KNOW ESSENTIALS. Detailed definitions for each element, coding for the elements, and a system to evaluate a given health technology using the tool were developed.

Results and Conclusions: Developing countries and resource-limited settings face several challenges to informed decision making. Models that are relevant and applicable in high-income countries are unlikely in such settings. KNOW ESSENTIALS is an alternative that facilitates evidence-based decision making by stakeholders without formal expertise in HTA. The tool could be particularly useful, as an interim measure, in healthcare systems that are developing HTA capacity. It could also be useful anywhere when rapid evidence-based decisions on health technologies are required.

Objectives: The objective of this exploratory study was to survey international health technology assessment (HTA) professionals to determine attitudes toward ethics in HTA.

Methods: An exploratory, quantitative, cross-sectional study design was developed. The sample population (n = 636) was selected from authors of the 206 articles published in the International Journal of Technology Assessment in Health Care between 2005 and 2007. A survey instrument was piloted and e-mailed.

Results: The response rate was low (n = 104; 16.4 percent). Respondents were primarily middle-aged (46 ± 11 years) male (62 percent) health professionals from Western countries (n = 92; 88.5 percent), with a mean of 10 (± 6 years) years of HTA experience. Although at least 90 percent of respondents agreed that healthcare decisions involved value judgments and that ethical analysis was important to HTA, respondents were divided as to whether normative (n = 45; 44.6 percent) or descriptive (n = 38; 37.6 percent) ethical recommendations were necessary. Most respondents (n = 83; 81.4 percent) believed that HTA should include citizen participation, but two thirds (n = 67; 67.0 percent) agreed that the final decision should be restricted to decision makers. A majority of respondents thought that ethical analysis could be discussed anywhere within the HTA process, either by an expert trained in ethics (n = 62; 60.8 percent) or by an external consultant (n = 80; 78.4 percent).

Conclusions: This study showed that ethical discourse in HTA is constrained by practical considerations, which serves to limit moral inquiry. To increase ethical analysis, a positive attitude toward ethics needs to be fostered within the HTA community.

Objectives: The UK's National Institute for Health and Clinical Excellence (NICE) and the Jordan office of the Medicines Transparency Alliance embarked on a pilot project to design an evidence-based guideline for cost-effective pharmacological treatment of essential hypertension in Jordan. The project's objectives were to directly address a major health problem for Jordan by producing a guideline; and to delineate the strengths and weaknesses of Jordan's healthcare process to allow similar future efforts to be planned more efficiently.

Methods: The pilot spanned a period of approximately 8 months. Activities were overseen by local technical and guideline development teams, as well as experts from NICE. NICE's hypertension guidelines and economic model were used as a starting point. Parameters in the economic model were adjusted according to input and feedback from local experts with regards to Jordanian physician and patient practices, resource costs, and quality of life estimates. The results of the economic model were integrated with the updated available clinical trial literature.

Results: The outputs of the economic model were used to inform recommendations, in the form of a clinical algorithm. A report of the process and the strengths and weaknesses observed was developed, and recommendations for improvements were made.

Conclusions: The pilot represented the start of what is intended to be a healthcare process change for the country of Jordan. Issues emerged which can inform strategies to ensure a more cohesive and comprehensive approach to the cost-effective use of appropriate drugs in managing chronic disease in Jordan and countries operating in a similarly resource-constrained environment. Furthermore, our pilot highlights how richer countries with relevant experience in evidence-informed healthcare policy making can assist others in strengthening their decision-making methods and processes.

Objectives: To propose a new method for comparing and integrating original qualitative data with systematic reviews of quantitative and qualitative studies, demonstrated by a study of the psychosocial needs of chronic fatigue syndrome (CFS) sufferers in Québec.

Methods: A systematic literature review was performed across various databases for English and French language studies, on the psychosocial aspects of CFS. Qualitative, quantitative, and mixed method studies published between January 1994 and July 2008 were included. Unpublished literature and reference lists of included studies were also searched. Themes identified in the literature were used to guide semi-structured interviews with seventeen CFS-sufferers, mostly recruited from a large specialist practice in Montreal. Interviews were transcribed verbatim and validated by a research assistant. Transcripts were coded using the identified themes. New codes were created when new issues arose. All themes were subsequently synthesized into overall categories using a constant comparative method.

Results: The literature search yielded thirty-one papers: twenty-eight primary studies and three systematic reviews. Twelve themes were identified and synthesized into four overall problem categories, such as “Lack of professional recognition.” Interviews confirmed findings from the literature, but also revealed unidentified needs specific to CFS-sufferers in Québec. Policy recommendations were provided to address these needs.

Conclusions: Multi-Source Synthesis provides a systematic method for synthesizing data from original studies with literature findings, thereby broadening the knowledge base and the local relevance of decisions concerning specific patient populations.

Objectives: The aim of this study was to analyze the motives, enablers, and barriers to promote or initiate health technology assessment (HTA) in different contexts.

Methods: An observational study design was used to address the above question that included a survey questionnaire and a two-phase study. The respondents for the questionnaire and first round of the study were from HTA agencies of high income countries and those low and middle income countries that have managed to establish HTA agencies (n = 50), that are members of International Network of Agencies for Health Technology Assessment (INAHTA), EuroScan, or European network for Health Technology Assessment (EUnetHTA). The second round of the study was exclusively with respondents from low and middle income countries that were manly affiliated to Health Technology Assessment International (HTAi) interest subgroup for low and middle income countries and aimed to initiate HTA activities (n = 34).

Results: Forty-one of fifty HTA agencies answered the survey questionnaire. Thirty-three of fifty individuals belonging to HTA agencies from high income countries and sixteen of thirty-four individuals from low and middle income countries answered in the first and second phases of the study, respectively. In the promotion and/or initiation of HTA, the top three motives were the same for both high income and low and middle income countries. The top three enablers were also similar but the prioritization varies. The top three barriers were more context specific.

Conclusions: HTA promotion or initiation is influenced by the following: (i) key players that affect the time taken to establish HTA agencies; (ii) three models for HTA promotion and initiation: top-down (political interest), bottom-up (academic/research interest), and converging (political and academic/research interests); and (iii) motives, enablers, and barriers at the local context.

Health systems are placing more and more emphasis on designing and delivering services that are focused on the patient, and there is a growing interest in patient aspects of health policy research and health technology assessment (HTA). Only a few HTA agencies use and invest in scientific methods to generate knowledge and evidence about the patient aspects of a given technology. This raises questions about how knowledge is produced in HTA reports and what kind of knowledge is considered relevant. This article uses a Danish HTA on patient education from 2009 as empirical material for a critical examination and discussion of knowledge and knowledge production about the patient aspects of HTA.

Objectives: Evidence-based guideline committees are multidisciplinary and explicitly consider the existing evidence. They are thus in an ideal position to identify research gaps. However, gaps have not been systematically identified through guidelines. We pilot tested a method to systematically identify and classify gaps from evidence-based guidelines.

Methods: We reviewed all evidence-based guidelines published by the Cystic Fibrosis Foundation. We identified research gaps as topics for which there was insufficient evidence (recommendations were not made or consensus recommendations were made) and topics specified as needing further research. We characterized gaps using a standard framework and classified them by type of management issue, specificity of target population, and age of target population.

Results: We identified sixty-two research gaps in five guidelines (mean = 12.4/guidelines document). While thirteen gaps were topics specified as needing further research, most (n = 49) were topics with insufficient evidence. Of these forty-nine, recommendations were not made for twenty-two topics while consensus recommendations were made for twenty-seven topics. Most gaps were issues of comparative effectiveness (44/62), addressed the general cystic fibrosis population (40/62), and were specific to infants (33/62). Relevant comparisons and outcomes were explicitly stated for only 7 percent and 16 percent of gaps respectively.

Conclusions: Almost 80 percent of the gaps were not topics identified as future research needs in the guidelines documents but rather were topics with insufficient evidence for making recommendations. Although we used cystic fibrosis in the United States as an example, the method we developed could be applied in other settings, including other countries and for different diseases.

Objectives: The aim of this study was to critically examine the current guidance for conducting ethics analysis in health technology assessment (HTA) and to offer recommendations for how to improve this practice.

Methods: MEDLINE, Philosopher's Index, and Google Scholar were searched for articles and reports using the keywords “ethics” and “health technology assessment” and related terms. Bibliographies of all relevant articles were also examined for additional references. A philosophical analysis of the existing guidance was conducted.

Results: We offer three recommendations for improving ethics analysis in HTA. First, ethical and legal issues must be clearly separated so that all policy-relevant questions that the technology raises can be considered clearly and systematically. Second, analysts must make better use of ethics theory and discuss better how particular theoretical approaches and associated analytic tools are selected to make transparent which alternative approaches were considered and why they were rejected. Third, the necessity for philosophical expertise to adequately conduct ethics analysis needs to be acknowledged.

Conclusions: To act on these recommendations for ethics analysis, we offer these three steps forward: acknowledge and use relevant expertise, further develop models for conducting and reporting ethics analyses, and make use of untapped resources in the literature.

Background: For non-drug technologies, there is often residual uncertainty following systematic review, mainly due to inadequate evidence of efficacy. The unwillingness to make decisions in the presence of uncertainty may lead to passive diffusion and intuitive decision making with or without public pressure. This may affect health system sustainability. There is increasing interest in post-market evaluation through processes that include coverage with evidence development (CED) to address residual uncertainty regarding effectiveness and cost-effectiveness. Global experience of CED has been slow to develop despite their potential contribution to decision making.

Methods: Ontario's field evaluation program to better inform decision making represents a collaboration between physicians, policy decision makers and academic centers. We report results of the first ten CEDs from this program to assess whether they achieved their objective of influencing policy by addressing residual uncertainty following systematic review.

Results: Since 2003, nineteen field evaluation studies to resolve residual uncertainty following systematic review have been completed, ten of which met the criteria of CED and are the focus of this report. There was more than one patient subgroup or intervention in three of the CEDs. This provided the basis for evaluating thirteen outcomes. In each case, the CED addressed the uncertainty and led to a decision based on the systematic review and CED result. The CEDs led to adoption of the technology in six instances, modified adoption in three instances and withdrawal in four instances.

Conclusions: CED makes an important contribution to translating evidence to decision making. Methodologies are needed to increase the scope and reduce timelines for CEDs, such as the use of linked comprehensive and robust data sets and collaborative studies with other jurisdictions. CED before making long-term funding decisions, especially where there is uncertainty of effectiveness, safety or cost-effectiveness, should be increasingly funded by health systems.

Italy is one of the few countries that have matured substantial experience of risk-sharing agreements so far. The first performance-based arrangement was agreed in July 2006, and as of October 2010, eighteen contracts have been in force.

The complex management of discount schemes is entirely based on Web registries run by AIFA, the Italian drug agency. The system validates each prescription and automatically requests the hospital pharmacy by e-mail to release the drug. If a patient meets nonresponder criteria, the hospital pharmacist should apply for pay-back to the manufacturer.

There are still some important question marks to address. First of all, nonresponders have to be documented by health authorities, otherwise any undocumented nonresponder will be paid as a success. Another question concerns pre-set timing. Although the scientific rationale of the nonresponder criteria for each drug has not been made public, time frames appear too short to allow a reliable assessment. Another question is whether regions, which are financially accountable in Italy for pharmaceutical expenditure, are really able to claw back refunds from manufacturers. Unfortunately here again there are no official figures, and regions do not seem yet able to quantify the amount of pay-back matured in the 4 previous years. The delayed and incomplete availability of pay-back procedures may be one explanation.

Background: The relationship between regulatory approval on the one hand and health technology assessment (HTA) and coverage on the other is receiving growing attention. Those responsible for regulatory approval, HTA, and coverage have different missions and their information requirements reflect these. There is nonetheless an increasingly popular view that improved communication and coordination between these functions could allow them all to be undertaken effectively with a lower overall burden of evidence requirements, thus speeding patient access to new products and reducing unnecessary barriers to innovation. This study summarizes the main points emerging from a recent discussion of this topic at the HTAi Policy Forum.

Results and Conclusions: After considering the roles of the various bodies, stakeholder perspectives and some current practical initiatives, those present at the Forum meeting discussed possible goals and challenges for improved interactions—in general and at specific stages of the product development life cycle. Opportunities for progress were seen in: continuing the dialogue to promote understanding and interaction between the different bodies and stakeholders; working to align scientific advice for manufacturers on the design and data requirements of pre- and post-marketing evaluation of products (specifically phase 2/3 and phase 4 trials for drugs); and extending the current dialogue to include discussion of product development to address unmet health needs.

Objectives: The objective of this study is to describe how patient aspects are assessed in Danish HTA reports, thereby contributing to the ongoing international debates concerning patient aspects in HTAs.

Methods: Fifty-eight Danish HTA reports published from 1999 to 2010 were read systematically, focusing on the inclusion of patient aspects, the methods used to generate data, and if and how clinical recommendations concerning patient aspects were drawn.

Results: Most of the fifty-eight HTA reports had a separate chapter about patient aspects. All reports included literature reviews for patient aspects. There was, however, much variation in the ways in which the authors presented and discussed their review methods. The most frequently used databases for the literature search were medical and HTA databases. More than half of the reports included primary research, either quantitative or qualitative methods, a few used both kinds of methods. Whereas some mentioned the importance of including patient aspects in daily clinical practice, others provided ideas on how to improve patient information or recommended changes in healthcare practices.

Conclusions: Danish HTA reports do, to a large extent, include patient aspects in the assessment and in the final conclusions of the reports. If health policy and decision making is to be patient-focused, it is important that HTAs in the future integrate patient aspects in recommendations of HTA. Further improvement in assessments of patient aspects, in relation to use and description of methodologies and theoretical considerations, will assist producing the relevant evidence for these recommendations.

Background: In many countries, the incremental cost-effectiveness ratio (ICER) is used to assess whether an intervention is worth its costs. At the same time, policy makers often feel uncomfortable with refusing reimbursement of any intervention purely on the basis of the fact that the ICER exceeds a specific threshold value. Reluctance to define a single threshold value for the ICER seems to have been stronger in social security systems than in national healthcare services systems. This study explores how basic differences between healthcare systems impact upon the potential usefulness of an ICER threshold value.

Methods: This study is a narrative review of literature about the theoretical foundations of the ICER threshold value approach and its practical relevance in different types of healthcare systems.

Results: A single ICER threshold value cannot be maintained, defined, or measured and should not be used as a policy-making tool. None of the solutions presented up until now to make the ICER threshold approach a valuable policy-making tool overcome the important weaknesses of the approach.

Conclusions: ICERs and ICER threshold values are insufficient for assessing interventions' value for money. Rather, they should be considered as one element in the decision-making process. Complete rationalization of the decision-making process by means of quantitative decision criteria is undesirable and not feasible. Increasing transparency in the criteria used for a decision and explicitness about the relative importance of each criterion should, therefore, be the major goal.