Background: Uncertainty about the applicability of controlled trial findings is an increasing concern for clinicians and policy decision makers. This study aimed to determine whether information reported in studies included in systematic reviews was adequate enough to assess their applicability.

Methods: We used the databases of four recently conducted systematic reviews on the comparative efficacy and safety of second-generation antidepressants, inhaled corticosteroids, Alzheimer's drugs, and targeted immune modulators. We developed and pilot-tested a questionnaire to assess the adequacy of reporting with respect to seven previously validated criteria of study design that distinguish explanatory from pragmatic studies. For each of the 137 included studies, two reviewers independently assessed the adequacy of reporting.

Results: Overall, only 12 percent of the included studies provided sufficient information to reliably distinguish explanatory from pragmatic studies. The areas with the greatest lack of reporting were the setting of the study, methods of adverse event assessment, and sample size considerations to determine a minimally important difference from a patient perspective.

Conclusions: Substantial shortcomings in reporting exist in aspects of study design important to determine whether a study is applicable to specific populations of interest.

Objectives: To describe the development and application of health technology assessment (HTA) in Australia.

Methods: Review of relevant literature and other documents related to HTA in Australia.

Results: Most HTA activity in Australia has been associated with provision of advice for the two national subsidy programs, Medicare, and the Pharmaceutical Benefits Scheme (PBS). National advisory bodies established by the federal government have had a prominent role. Assessments from the advisory bodies have had a major influence on decisions related to Medicare and the PBS, and in some other areas. Technologies without links to the national subsidy schemes, and those that are widely distributed, have been less well covered by HTA. To some extent these are addressed by evaluations supported by state governments, but details of approaches taken are not readily available.

Conclusions: HTA in Australia now has a long history and is well established as a source of advice to health decision makers. Challenges remain in extending the scope of assessments, developing more transparent approaches in some areas, and consistently applying appropriate standards.

Objectives: The aim of this study was to assess the status of translating research findings into practice at a major academic healthcare system in Central Texas.

Methods: We conducted a cross-sectional survey addressing knowledge of and participation in translational research of physicians, residents, nurses and third- and fourth-year medical students in a major academic healthcare system in Central Texas.

Results: Out of 508 respondents, 428 (84.3 percent) completed all questions. A total of 68.9 percent of faculty reported having sufficient education and training to conduct research versus 44.4 percent of residents and 35.6 percent of nurses. Fifty-eight percent of faculty, 53 percent of residents and 9 percent of nurses reported current involvement in research activity. A total of 55.6 percent of residents reported that their departments provide them with protected time for research versus 18.4 percent of faculty and 10.3 percent of nurses. In addition, 33.9 percent of nurses reported interest in participating in research but do not know how to start. There were 86.4 percent of faculty, 77.8 percent of residents, and 58 percent of nurses who indicated they were familiar with translational research. However, only 42.7 percent of faculty, 46.7 percent of residents and 35.6 percent of nurses indicated they were aware of any changes in the delivery of care that resulted from research projects.

Conclusions: The study results suggested failure to leverage members of the healthcare team in a systematic process to ensure translation of research findings into practice. Results highlighted the need to merge culture of safety and quality improvement with research while dealing with the daily pressures of patient care.

Objectives: Excessive use of laboratory diagnostics has been common. This study aimed to evaluate whether clinical decision rules for the use of liquor diagnostics would enable cost containment without affecting medical care.

Methods: This was a single-center, retrospective, cost-minimization study based on the records of all 16,319 patients hospitalized and discharged at a Neurology Clinic in Austria between 2004 and 2006. Cost of liquor diagnostics, discharge diagnosis, duration of hospital stay, and mortality were compared along the line before, during, and after implementation of decision rules in mid-2005.

Results: There were no significant changes in patient characteristics over time, not in the diagnoses at discharge, nor in the percentage of patients undergoing liquor diagnostics. The average number of tests per patient significantly decreased. Standard tests largely replaced serological tests for infections, regardless of diagnosis. Annual costs for liquor diagnostics decreased by 32.9 percent. Overall, the duration of hospital stay and mortality significantly decreased as well; however, differences were not significant for any single diagnosis-related group.

Conclusions: Diagnostic algorithms may allow cost containment without affecting medical care.

Objectives: The use and implementation of health technologies do influence and are influenced by organizational structures and processes as well as patient's experiences and actions. The objective of this study is to discuss the content and managing of organizational and patient-related assessments in HTAs.

Methods: Quantitative and qualitative analysis were performed based on a review of organizational and patient-related assessments included in a random sample of fifty HTA reports. The reports were identified from INAHTA members’ Web sites where homepages of HTA agencies where searched for full HTA reports, published in English or Scandinavian languages.

Results: HTA reports including organizational assessments mainly comprised issues related to process and structure, while issues such as control and evaluation of the technology were included less often. Cultural and environmental issues were included in a little over half of the reports. Reports including patient-related assessments mainly comprised psychological issues. Patients’ perceptions of the technology's effect on their health, patient-information, and social and ethical issues were included less often. Few reports included implications for the patient's significant others. There was considerable variation in how comprehensive the issues were managed as there was in the methodological transparency of the HTA reports.

Conclusions: There is room for improvement in the assessments of organizational and patient-related issues. Absence of a description of the considerations made when determining the content and methods of the assessments limits the usefulness of an HTA. The reader is left uncertain of the relevance and validity of the organizational and patient-related assessments.

Objectives: This article explains how the issue of stakeholder involvement was addressed in the European network for Health Technology Assessment (EUnetHTA) Project and describes the structures of future stakeholder involvement in the EUnetHTA Collaboration.

Methods: Initiatives led to a dialogue with stakeholders and exchanging views and expectations on health technology assessment (HTA) processes and the future development of EUnetHTA. The methods of involving different stakeholder groups in EUnetHTA included general information to stakeholders about EUnetHTA, targeted information on a Web site, analysis of stakeholder opinions on HTA and EUnetHTA, and development of a draft stakeholder policy.

Results: First steps were taken to organize processes to consolidate the legitimacy of EUnetHTA and its products and encourage the representation of interests, thus contributing to promoting the utilization of HTA in national/regional policy making. A stakeholder Web site, analyses of stakeholder opinions on HTA and EUnetHTA in a discussion topic catalog, and a draft stakeholder policy resulted from the work.

Conclusions: Stakeholder involvement in EUnetHTA is necessary to ensure the legitimacy and prospects for utilization of EUnetHTA and its products. The described activities and results create the foundation for a continued dialogue with, and involvement of, stakeholders. The EUnetHTA stakeholder meeting can be considered as a successful experience of dialogue between EUnetHTA and stakeholders, which should be continued. Our experience shows the challenge of obtaining balanced stakeholder representation across the identified stakeholder groups. Continued attention should be given to achieving balanced stakeholder representation.

Objectives: The aim of this study was to present a review of economic evaluations conducted from a Canadian perspective and to characterize sources of evidence and statistical methods to analyze effectiveness measures, resource utilization, and uncertainty.

Methods: A search strategy was developed to identify Canadian economic evaluations published between January 2001 and June 2006. A standardized abstraction form was used to extract key data (e.g., study design, data sources, statistical methods).

Results: A total of 153 unique studies were included for review, of which 75 were evaluations of drug therapies and less than half were funded by industry. Cost-effectiveness analysis was the most common type of economic evaluation and 80 percent of the studies used modeling techniques. A single source of evidence for effectiveness measures was used in half of the studies. Statistical methods were commonly reported to compare effectiveness measures when the economic evaluation was conducted alongside a clinical trial but less commonly when determining effectiveness input parameters in model-based economic evaluations, or to analyze resource utilization data. Authors relied mostly on univariate sensitivity analyses to explore uncertainty.

Conclusions: This review identifies the need to improve the conduct and reporting of statistical methods for economic evaluations to improve confidence in the results.

This study summarizes the experience with health technology assessment (HTA) at the European level. Geographically, Europe includes approximately fifty countries with a total of approximately 730 million people. Politically, twenty-seven of these countries (500 million people) have come together in the European Union. The executive branch of the European Union is named the European Commission, which supports several activities, including research, all over Europe and in many other parts of the world. The European Commission has promoted HTA by several policy positions and has funded a series of projects aimed at strengthening HTA in Europe. Around fifteen of the European countries now have formal national programs on HTA and some also have regional public programs. All countries that are members of the European Union and do not have a national approach to HTA have an interest in becoming more involved. The HTA projects sponsored by the European Commission have focused on networking and collaboration among established agencies and institutions for HTA, however, also on capacity building, support, and facilitation in creating mechanisms for HTA in European countries that still do not have any program in the field.

Objectives: The aim of cost-effectiveness analysis is to maximize health benefits from a given budget, taking a societal perspective. Consequently, the comparison of alternative treatments or technologies is solely based on their expected effectiveness and cost. However, the expectation, or mean, poses important limitations as it might be a poor summary of the underlying distribution, for instance when the effectiveness is a categorical variable, or when the distributions of either effectiveness or cost present a high degree of asymmetry. Clinical variables often present these characteristics.

Methods: In this study, we present a framework for cost-effectiveness analysis based on the whole posterior distribution of effectiveness and cost.

Results: An application with real data is included to illustrate the analysis. Decision-making measures such as the incremental cost-effectiveness ratio, incremental net-benefit, and cost-effectiveness acceptability curves, can also be defined under the new framework.

Conclusions: This framework overcomes limitations of the mean and offers complementary information for the decision maker.

Objectives: Fatigue is the most common symptom reported by cancer patients. The inclusion of health-related quality of life (HRQL) measures in routine clinical care of cancer patients may improve the management of fatigue. The primary objective of this study is to provide evidence on the magnitude of change in fatigue subscale scores using the Functional Assessment of Cancer Therapy-Fatigue (FACT-F) that is clinically important.

Methods: Consecutive patients with advanced primary lung cancer attending a Canadian tertiary care cancer and, prior to undergoing palliative chemotherapy, were enrolled in the study. Patients completed a battery of questionnaires [FACT-F, Qualitative Patients Self-report of Fatigue Level (QPSRF)] at baseline, follow-up and 2 weeks after their final cycle of chemotherapy. Clinicians assessed the patients using the Eastern Cooperative Oncology Group (ECOG) Performance Status Scale at baseline and each follow-up visit. FACT-F change scores were computed as the mean change in score (end of study score minus baseline score).

Results: A total of 43 patients with mean age of 59 years were enrolled in the study. Results revealed a mean change in FACT-F subscale score of 5.0 (SE 1.06) for those who rated themselves as more tired, 1.28 (SE 1.00) for those who rated themselves as the same (no change), and −1.52 (SE 0.84) for those patients who rated themselves as less tired.

Conclusions: We provide evidence on the magnitude of change in FACT-F score that is associated with the perception by patients of improvement in fatigue and magnitude of change in score that is associated with worsening in fatigue.

Objectives: Controversy exists regarding the optimal method of providing dialysis in critically ill patients with acute renal failure. We sought to determine the cost-effectiveness of treatment strategies.

Methods: Adult subjects requiring renal replacement therapy in a critical care setting who are candidates for intermittent hemodialysis (IHD) or continuous renal replacement therapy (CRRT) were considered within a Markov model. Alternative strategies including IHD, and standard or high dose CRRT were compared. The model considered relevant clinical and economic outcomes, and incorporated data on clinical effectiveness from a recent systematic review and high quality micro-costing data.

Results: In the base-case analysis, CRRT was associated with similar health outcomes but higher costs by ($3,679 more than IHD per patient). In scenarios considering alternate cost sources, and higher intensity of IHD (including daily and longer duration IHD), CRRT remained more costly. Sensitivity analysis indicated that even small differences in the risk of mortality or need for long-term chronic dialysis therapy among surviving patients benefits led to dramatic changes in the cost-effectiveness of the modalities considered.

Conclusions: Given the higher costs of providing CRRT and absence of demonstrated benefit, IHD is the preferred modality in critically ill patients who are candidates for either IHD or CRRT, although this conclusion should be revisited if future clinical trials establish differences in clinical effectiveness between modalities. Future interventions that are proven to improve renal recovery after acute renal failure are likely to be cost-effective, even if very resource intensive.

Objectives: The aim of this study was to facilitate effective internal and external communication of an international network and to explore how to support communication and work processes in health technology assessment (HTA).

Structure and Methods: European network for Health Technology Assessment (EUnetHTA) connected sixty-four HTA Partner organizations from thirty-three countries. User needs in the different steps of the HTA process were the starting point for developing an information system. A step-wise, interdisciplinary, creative approach was used in developing practical tools.

Results: An Information Platform facilitated the exchange of scientific information between Partners and with external target groups. More than 200 virtual meetings were set up during the project using an e-meeting tool. A Clearinghouse prototype was developed with the intent to offering a single point of access to HTA relevant information. This evolved into a next step not planned from the outset: Developing a running HTA Information System including several Web-based tools to support communication and daily HTA processes. A communication strategy guided the communication effort, focusing on practical tools, creating added value, involving stakeholders, and avoiding duplication of effort.

Conclusions: Modern technology enables a new information infrastructure for HTA. The potential of information and communication technology was used as a strategic tool. Several target groups were represented among the Partners, which supported collaboration and made it easier to identify user needs. A distinctive visual identity made it easier to gain and maintain visibility on a limited budget.

Objectives: This research integrates existing literature on information technology (IT) in hospitals, and proposes and validates a comprehensive IT capacities assessment tool in these settings.

Methods: A comprehensive literature review was conducted on Medline until September 2006 to identify studies that used specific IT measures in hospitals. The results were mapped and used as a basis for the development of the proposed instrument, which was tested through a survey of Canadian healthcare organizations (N = 221).

Results: A total of seventeen studies provided indicators of clinical and administrative IT capacities in hospitals. Based on the mapping of these indicators, a comprehensive IT capacities assessment instrument was developed including thirty-four items exploring computerized processes, thirteen items assessing contemporary technologies, and eleven items investigating internal and external information sharing. A time frame was inserted in the tool to reflect “plans for” versus “current” implementation of IT; in the latter, the extent of current use of computerized processes and technologies was measured on a (1–7) scale. Overall, the survey yielded a total of 106 responses (52.2 percent response rate), and the results demonstrated a good level of reliability and validity of the instrument.

Conclusions: This study unifies existing work in this area, and presents the psychometric properties of an IT capacities assessment tool in hospitals. By developing scores for capturing IT capacities in hospitals, it is possible to further address important research questions related to the determinants and impacts of IT sophistication in these settings.

Objectives: The aim of this article is to describe and analyze the stages toward recognition and implementation of health technology assessment (HTA).

Methods: System analysis of structures and institutions and their use of HTA.

Results: Austria is a latecomer in implementing evaluations/HTA as decision support. It can to a certain degree absorb the increasing international knowledge. Austria had a long time to observe the successes and failures of HTA in other countries and to learn from other countries. The implementation of HTA within the Austrian healthcare system ran through stages of uptake: starting 1989 with a systematic review on international activities, first international networking and collaboration since 1991, proposed assessments until the late 1990s, followed by reactive assessments on demand mostly on high volume and costly technologies since then. Since 2000, HTA is used on a regular basis for investment and reimbursement decisions by several players, namely the Ministry of Health, the Social Insurance and hospital cooperations. In 2006, the Austrian HTA-institute was founded.

Conclusions: It took approximately 15 years from first research activities in HTA to an institutionalization. HTA in Austria is not only product- but also process-oriented: The actual production of assessments for decision support is as important as the structuring and accompanying of the process of decisions making. In addition, shaping the public understanding of science (characterized by the intrinsic belief that all new medical interventions provide added value to the healthcare system) is part of Austrian HTA.

Objectives: The aim of this study was to examine the potential cost-effectiveness of calcium chemoprevention post-polypectomy as a substitute or adjunct for surveillance.

Methods: We constructed a Markov model of post-polypectomy adenoma recurrence and colorectal cancer (CRC) development, calibrated to data from prospective chemoprevention trials of fiber, calcium, antioxidants, and aspirin. We modeled four scenarios for 50-year-old patients immediately after polypectomy: (i) natural history with no further intervention; (ii) elemental calcium 1,200 mg/day from age 50–80; (iii) surveillance colonoscopy from age 50–80 every 5 years, or 3 years for large adenoma; (iv) calcium + surveillance. Patients were followed up until age 100 or death.

Results: Calcium was cost-effective compared to natural history ($49,900/life-year gained). However, surveillance was significantly more effective than calcium (18.729 versus 18.654 life-years/patient; 76 percent versus 14 percent reduction in CRC incidence) at an incremental cost of $15,900/life-year gained. Calcium + surveillance yielded a very small benefit (0.0003 incremental life-years/patient) compared with surveillance alone, at a substantial incremental cost of $3,090,000/life-year gained.

Conclusion: Post-polypectomy calcium chemoprevention is unlikely to be a reasonable substitute for surveillance. It may be cost-effective in patients unwilling or unable to undergo surveillance.

Objectives: The internal evaluation studied the development of the European network for Health Technology Assessment (EUnetHTA) Project in achieving the general objective of establishing an effective and a sustainable network of health technology assessment (HTA) in Europe.

Methods: The Work Package 3 group was dedicated to this task and performed the work. Information on activities during the project was collected from three sources. First, three yearly cross-sectional studies surveyed the participants’ opinions. Responses were by individuals or by institutions. The last round included surveys to the Steering Committee, the Stakeholder Forum, and the Secretariat. Second, the Work Package Lead Partners were interviewed bi-annually, five times in total, to update the information on the Project's progress. Third, additional information was sought in available documents.

Results: The organizational structure remained stable. The Project succeeded in developing tools aimed at providing common methodology with intent to establish a standard of conducting and reporting HTA and to facilitate greater collaboration among agencies. The participants/agencies expressed their belief in a network and in maintaining local/national autonomy. The Work Package Leaders expressed a strong belief in the solid base of the Project for a future network on which to build, but were aware of the need for funding and governmental support.

Conclusions: Participants and Work Package Leaders have expressed support for a future network that will improve national and international collaboration in HTA based on the experience from the EUnetHTA project.

Objectives: Due to potential advantages (e.g., using all available evidence), Bayesian methods have been proposed to assist healthcare decision making. This review provides a detailed description of how Bayesian methods have been applied to economic evaluations of patient level data. The results serve both as a reference and as a means by which to examine the appropriate application of Bayesian methods to inform decision making.

Methods: MEDLINE, EMBASE, and Cochrane Economic Evaluation databases were searched to identify studies, published up to November 2007, meeting three inclusion criteria: (i) the study conducted an economic evaluation, (ii) sampling uncertainty was incorporated using Bayesian methods, (iii) the likelihood function was informed by patient level data from a single source. Data were collected on key study characteristics (e.g., prior distribution, likelihood function, presentation of uncertainty).

Results: The search identified 366 potentially relevant studies, from which 103 studies underwent full-text review. Sixteen studies met the inclusion criteria. Half of the studies used uninformative priors; most studies incorporated the potential dependence between costs and effects, and presented cost-effectiveness acceptability curves. Results were sensitive to changes in the priors and likelihoods.

Conclusions: Limited use of informative priors, among the included studies, gives policy makers little guidance on one of the main benefits of Bayesian methods, the ability to integrate all available evidence to capture the uncertainty inherent in decision making.

Objectives: The research objectives were two-fold: first, to systematically review the literature on the cost-effectiveness of home telehealth for chronic diseases, and second to develop a framework for the conduct of economic evaluation of home telehealth projects for patients with chronic diseases.

Methods: A comprehensive literature search identified twenty-two studies (n = 4,871 patients) on home telehealth for chronic diseases published between 1998 and 2008. Studies were reviewed in terms of their methodological quality and their conclusions.

Results: Home telehealth was found to be cost saving from the healthcare system and insurance provider perspectives in all but two studies, but the quality of the studies was generally low. An evaluative framework was developed which provides a basis to improve the quality of future studies to facilitate improved healthcare decision making, and an application of the framework is illustrated using data from an existing program evaluation of a home telehealth program.

Conclusions: Current evidence suggests that home telehealth has the potential to reduce costs, but its impact from a societal perspective remains uncertain until higher quality studies become available.

Objectives: This study investigates the use of information graphics to display the outputs of health technology assessment (HTA) in the United Kingdom and proposes a more structured approach founded in an analysis of the decision-making requirements of the key stakeholders.

Methods: A scoping review of HTA reports was conducted to investigate current practice in the use of information graphics in HTA literature. A classification framework using dimensions of report section, graphical type, and originating research center was devised and used for a full content analysis of the graphical figures in the fifty most recent reports produced for the UK National Health Service's HTA process.

Results: Our survey shows that graphical tools are used extensively in HTA reports although less frequently than tables. Use of information graphics varies widely between different report sections and tends to follow conventional lines with little evidence of variance from established practice. The largest variance was found between the quantities of graphics used by different research centers responsible for authoring the reports.

Conclusions: HTA makes extensive use of graphics; however, there is little evidence of a systematic or standardized approach, or of much innovation. Significant potential exists to explore the application of information graphics in this field, but there are many research challenges. A contextually based, structured approach to the design of effective information graphics in HTA is proposed as a basis both to investigate the application of existing graphical tools in HTA, and to explore the considerable scope for innovation.

Verhagen et al. (5) suggest, in a study done in 2002, that methodologic quality of individual trials do not influence the conclusions of a landmark meta-analysis on thrombolytic therapy. This meta-analysis was studied because it was believed that it represents the true effect because its conclusions remain valid 15 years after publication. They incorporated the results of quality assessment in five different ways in the calculation of the pooled odd ratios (ORs): (i) component analysis, (ii) visual plot, (iii) quality score as a threshold score, (iv) quality score as a weighting factor, and (v) cumulative pooling. They did not find much discrepancy using either of these methods of quality assessment.