Objectives: The objective of this study is to profile the health technology assessments (HTAs) produced in Canada and other selected countries and assess their potential to inform policy making about health systems in jurisdictions other than the ones for which they were produced, and to develop and pilot test prototypes for packaging and assessing the relevance of HTAs for health system managers and policy makers.

Methods: We compiled an inventory of all HTAs that were produced by nine HTA agencies between September 2003 and August 2006; coded the title and abstract of each HTA according to the technologies assessed, methods used, and whether or not context-specific actionable messages were provided; developed a prototype for a structured, decision-relevant HTA summary and for a relevance-assessment form; and pilot-tested the prototypes using semistructured telephone interviews with a purposive sample of Canadian healthcare managers and policy makers.

Results: Our review of the 223 HTAs identified that: (i) 44 HTAs addressed health system arrangements (20 percent); (ii) 205 incorporated a systematic review (92 percent), whereas only 12 incorporated a sociopolitical assessment using explicit methods (5 percent); and (iii) 50 contained context-specific actionable messages (22 percent). Our interviews identified significant support for both the general idea of an HTA summary and the prototype's specific elements, but mixed views about using peer assessments of relevance.

Conclusions: Those involved in supporting the use of HTAs in policy making about health systems may wish to produce structured decision-relevant summaries for their systematic review-containing HTAs to increase the prospects for their HTAs being used outside the jurisdiction for which they were produced.

The objective of health technology assessment (HTA) is to support decision making in health care. HTA does not claim to provide a definite solution to a health care problem, but to assist decision makers with evidence-based information about the clinical, ethical, social, and economic implications of the development, diffusion, and use of health care technology.

The purpose of this study was to determine how horizon-scanning organizations can encourage the implementation of recommendations contained in their early warning messages about emerging health technologies. We reviewed the conclusions of the EUR-ASSESS Project Subgroup report on dissemination and impact, an overview of systematic reviews of interventions to promote implementation of research findings by health care professionals, and various Cochrane Effective Practice and Organisation of Care Group protocols and reviews. The evidence on the effectiveness of different implementation strategies aimed at distinct target groups is of variable quality. There is some evidence from rigorous study designs on the effectiveness of strategies designed to influence the behavior of health care professionals; the quality of the evidence relating to policy makers and the general public is more limited. Horizon-scanning organizations can improve the likelihood of their recommendations being acted upon by developing active implementation strategies based on the best available evidence, establishing links with key groups, and directing early warning messages at specific target audiences. Given the relative lack of good quality evidence, it is important that implementation strategies be rigorously evaluated to determine their effectiveness.

Objectives: In an interrupted time series (ITS) design, data are collected at multiple instances over time before and after an intervention to detect whether the intervention has an effect significantly greater than the underlying secular trend. We critically reviewed the methodological quality of ITS designs using studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies).

Methods: Quality criteria were developed, and data were abstracted from each study. If the primary study analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression.

Results: Twenty mass media studies and thirty-eight guideline studies were included. A total of 66% of ITS studies did not rule out the threat that another event could have occurred at the point of intervention. Thirty-three studies were reanalyzed, of which eight had significant preintervention trends. All of the studies were considered “effective” in the original report, but approximately half of the reanalyzed studies showed no statistically significant differences.

Conclusions: We demonstrated that ITS designs are often analyzed inappropriately, underpowered, and poorly reported in implementation research. We have illustrated a framework for appraising ITS designs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.

Objective: To compare the effects of computerized and paper-based versions of guidelines on recently qualified physicians' consultation practices.

Methods: Two arm cluster randomized controlled trial. Physicians were randomized to receive computerized or textbook-based versions of the same guidelines for a 4-week study period. Physicians' compliance with guideline recommendations about laboratory, radiological, physical and other examinations, procedures, nonpharmacologic and pharmacologic treatments, physiotherapy, and referrals were measured by case note review.

Results: There were 139 recently qualified physicians working in 96 primary healthcare centers in Finland who participated in the study. Data on 4,633 patient encounters were abstracted, of which 3,484 were suitable for further analysis. Physicians' compliance with guidelines was high (over 80% for use of laboratory, radiology, physical examinations, and referrals). There were no significant differences in physicians' consultation practices in any of the measured outcomes between the computerized and textbook group.

Conclusion: Guidelines are a useful source of information for recently qualified physicians working in primary care. However, the method of presentation of the guidelines (electronic or paper) does not have an effect on guideline use or their impact on decisions. Other factors should be considered when choosing the method of presentation of guidelines, such as information-seeking time, ease of use during the consultation, ability to update, production costs, and the physicians' own preferences.

The framework for this chapter stems from the proposal that the basic steps of the quality cycle should be combined with the practical stages of the scientific method to form an extended quality cycle comprising nine steps. The scientific and methodological implications of each step are considered in turn. The discussion of the three steps relating to the development, dissemination and implementation of clinical standards is based on a recent review of rigorous evaluations of clinical guidelines. The discussion of the three steps relating to the comparison of clinical performance with agreed standards is based on the existing principles of medical research. The final step is concerned with the management of change. The chapter ends by applying the steps of the extended quality cycle to the topic of thrombolytic therapy after acute myocardial infarction. The resulting hypothetical quality assurance exercise appears simple and potentially very effective. We conclude that, if quality assurance is pursued with a combination of scientific rigour, imagination and persistence, it has the potential to bring substantial benefits to patients and health professionals.