Advances in the digitization of health systems and expedited regulatory approvals of innovative treatments have led to increased potential for the use of real-world data (RWD) to generate real-world evidence (RWE) to complement evidence from clinical trials. However, health technology assessment (HTA) bodies and payers have concerns about the ability to generate RWE of sufficient quality to be pivotal evidence of relative treatment effectiveness. Consequently, there is a growing need for HTA bodies and payers to develop guidance for the industry and other stakeholders about the use of RWD/RWE to support access, reimbursement, and pricing. We therefore sought to (i) understand barriers to the use of RWD/RWE by HTA bodies and payers; (ii) review potential solutions in the form of published guidance; and (iii) review findings with selected HTA/payer bodies. Four themes considered key to shaping the generation of robust RWE for HTA bodies and payers were identified as: (i) data (availability, governance, and quality); (ii) methodology (design and analytics); (iii) trust (transparency and reproducibility); and (iv) policy and partnerships. A range of guidance documents were found from trusted sources that could address these themes. These were discussed with HTA experts. This commentary summarizes the potential guidance solutions available to help resolve issues faced by HTA decision-makers in the adoption of RWD/RWE. It shows that there is alignment among stakeholders about the areas that need improvement in the development of RWE and that the key priority to move forward is better collaboration to make data usable for multiple purposes.

Objectives: Although there is increased awareness of patient and public involvement (PPI) among health technology assessment (HTA) organizations, evaluations of PPI initiatives are relatively scarce. Our objective as members of Health Technology Assessment International's (HTAi's) Patient and Citizen Involvement Group (PCIG) was to advance understanding of the range of evaluation strategies adopted by HTA organizations and their potential usefulness.

Methods: In March 2016, a survey was sent to fifty-four HTA organizations through the International Network of Agencies for Health Technology Assessment (INAHTA) and contacts of members of HTAi's PCIG. Respondents were asked about their organizational structure; how patients and members of the public are involved; whether and how PPI initiatives have been evaluated, and, if so, which facilitators and challenges to evaluation were found and how results were used and disseminated.

Results: Fifteen (n = 15) programs from twelve countries responded (response rate 27.8 percent) that involved patients (14/15) and members of the public (10/15) in HTA activities. Seven programs evaluated their PPI activities, including participant satisfaction (5/7), process (5/7) and impact evaluations (4/7). Evaluation results were used to improve PPI activities, identify education and training needs, and direct strategic priorities. Facilitators and challenges revolved around the need for stakeholder buy-in, sufficient resources, senior leadership, and including patients in evaluations.

Conclusions: A small but diverse set of HTA organizations evaluate their PPI activities using a range of strategies that reflect the range of rationales and approaches to PPI in HTA. It will be important for HTA organizations to draw on evaluation theories and methods.

Objectives: Health Technology Assessment (HTA) needs to address the challenges posed by high cost, effective technologies, expedited regulatory approaches, and the opportunities provided by collaborative real-world evaluation of technologies. The Health Technology Assessment International (HTAi) Policy Forum met to consider these issues and the implications for evidence production to inform HTA. This paper shares their discussion to stimulate further debate.

Methods: A background paper, presentations, group discussions, and stakeholder role play at the 2015 HTAi Policy Forum meeting informed this paper.

Results: HTA has an important role to play in helping improve evidence production and ensuring that the health service is ready to adopt effective technologies. It needs to move from simply informing health system decisions to also working actively to align stakeholder expectations about realistic evidence requirements. Processes to support dialogue over the health technology life cycle need to be developed that are mindful of limited resources, operate across jurisdictions and learn from past processes. Collaborations between health technology developers and health systems in different countries should be encouraged to develop evidence that will inform decision making. New analytical techniques emerging for real-world data should be harnessed to support modeling for HTA.

Conclusions: A paradigm shift (to “Health Innovation System 2.0”) is suggested where HTA adopts a more central, proactive role to support alignment within and amongst stakeholders over the whole life cycle of the technology. This could help ensure that evidence production is better aligned with patient and health system needs and so is more effective and efficient.

Objectives: Rare diseases are often heterogeneous in their progression and response to treatment, with only a small population for study. This provides challenges for evidence generation to support HTA, so novel research methods are required.

Methods: Discussion with an expert panel was augmented with references and case studies to explore robust approaches for HTA evidence generation for rare disease treatments.

Results: Traditional RCTs can be modified using sequential, three-stage or adaptive designs to gain more power from a small patient population or to focus trial design. However, such designs need to maintain important design aspects such as randomization and blinding and be analyzed to take account of the multiple analyses performed. N-of-1 trials use within-patient randomization to test repeat periods of treatment and control until a response is clear. Such trials could be particularly valuable for rare diseases and when prospectively planned across several patients and analyzed using Bayesian techniques, a population effect can be estimated that might be of value to HTA. When the optimal outcome is unclear in a rare disease, disease specific patient reported outcomes can elucidate impacts on patients’ functioning and wellbeing. Likewise, qualitative research can be used to elicit patients’ perspectives, with just a small number of patients.

Conclusions: International consensus is needed on ways to improve evidence collection and assessment of technologies for rare diseases, which recognize the value of novel study designs and analyses in a setting where the outcomes and effects of importance are yet to be agreed.

HTA can be described as a research-based, practice-oriented assessment of relevant evidence and knowledge on the direct and intended effects of healthcare technologies, as well as the indirect and unintended consequences (4). It is also viewed as a multidisciplinary field of policy analysis that studies the medical, ethical, social, and economic implications of the development, diffusion, and use of a health technology (5).

Background: To ensure rapid access to new potentially beneficial health technologies, obtain best value for money, and ensure affordability, healthcare payers are adopting a range of innovative reimbursement approaches that may be called Managed Entry Agreements (MEAs).

Methods: The Health Technology Assessment International (HTAi) Policy Forum sought to identify why MEAs might be used, issues associated with implementation and develop principles for their use. A 2-day deliberative workshop discussed key papers, members' experiences, and collectively addressed four policy questions that resulted in this study.

Results: MEAs are used to give access to new technologies where traditional reimbursement is deemed inappropriate. Three different forms of MEAs have been identified: management of budget impact, management of uncertainty relating to clinical and/or cost-effectiveness, and management of utilization to optimize performance. The rationale for using these approaches and their advantages and disadvantages differ. However, all forms of MEA should take the form of a formal written agreement among stakeholders, clearly identifying the rationale for the agreement, aspects to be assessed, methods of data collection and review, and the criteria for ending the agreement.

Conclusions: MEAs should only be used when HTA identifies issues or concerns about key outcomes and/or costs and/or organizational/budget impacts that are material to a reimbursement decision. They provide patient access and can be useful to manage technology diffusion and optimize use. However, they are administratively complex and may be difficult to negotiate and their effectiveness has yet to be evaluated.

Background: The way in which a health technology is used in any particular health system depends on the decisions and actions of a variety of stakeholders, the local culture, and context. In 2009, the HTAi Policy Forum considered how health technology assessment (HTA) could be improved to optimize the use of technologies (in terms of uptake, change in use, or disinvestment) in such complex systems.

Methods: In scoping, it was agreed to focus on initiatives to implement evidence-based guidance and monitoring activities. A review identified systematic reviews of implementation initiatives and monitoring activities. A two-day deliberative workshop was held to discuss key papers, members’ experiences, and collectively address key questions. This consensus paper was developed by email and finalized at a postworkshop meeting.

Results: Evidence suggests that the impact and use of HTA could be increased by ensuring timely delivery of relevant reports to clearly determined policy receptor (decision-making) points. To achieve this, the breadth of assessment, implementation initiatives such as incentives and targeted, intelligent dissemination of HTA result, needs to be considered. HTA stakeholders undertake a variety of monitoring activities, which could inform optimal use of a technology. However, the quality of these data varies and is often not submitted to an HTA.

Conclusions: Monitoring data should be sufficiently robust so that they can be used in HTA to inform optimal use of technology. Evidence-based implementation initiatives should be developed for HTA, to better inform decision makers at all levels in a health system about the optimal use of technology.

There is increasing emphasis on providing patient-focused health care and ensuring patient involvement in the design of health services. As health technology assessment (HTA) is meant to be a multidisciplinary, wide-ranging policy analysis that informs decision making, it would be expected that patients’ views should be incorporated into the assessment. However, HTA is still driven by collection of quantitative evidence to determine the clinical and cost effectiveness of a health technology. Patients’ perspectives about their illness and the technology are rarely included, perhaps because they are seen as anecdotal, biased views. There are two distinct but complementary ways in which HTAs can be strengthened by: (i) gathering robust evidence about the patients’ perspectives, and (ii) ensuring effective engagement of patients in the HTA process from scoping, through evidence gathering, assessment of value, development of recommendations and dissemination of findings. Robust evidence eliciting patients’ perspectives can be obtained through social science research that is well conducted, critically appraised and carefully reported, either through meta-synthesis of existing studies or new primary research. Engagement with patients can occur at several levels and we propose that HTA should seek to support effective patient participation to create a fair deliberative process. This should allow two-way flow of information, so that the views of patients are obtained in a supportive way and fed into decision-making processes in a transparent manner.

As more countries use HTA to inform decisions on the reimbursement of health technologies, harmonization of evidence requirements between jurisdictions has been proposed, mainly on the grounds of improved efficiency. Harmonization has the potential to avoid duplication of effort for both manufacturers and HTA bodies involved in preparing and reviewing HTA submissions for innovative technologies. However, it also carries risks of loss of local control over decisions, the application of general data standards which are not universally accepted and slowing the rate of development of innovation in the analytical disciplines supporting HTA. This study reviews the issues associated with harmonization taking into account the perspectives of the multiple stakeholders. This study draws on experiences from recent initiatives intended to promote the harmonization of HTA and experience from related fields, particularly regulatory approval of new medical technologies.

In the article entitled “Rapid reviews versus full systematic reviews: An inventory of current methods and practice in health technology assessment,” by Watt et al. in volume 24 number 2 (Spring 2008) of International Journal of Technology Assessment in Health Care, the affiliation of Stephen Blamey is incorrectly listed as Department of Health & Ageing. Dr. Blamey is the current Chair of the Medical Services Advisory Committee (MSAC). MSAC is an independent scientific committee comprising individuals with expertise in clinical medicine, health economics, and consumer matters. The Department of Health & Ageing administers funding and operations for MSAC. However, members of MSAC act independently of the Department. As Chair of MSAC, Dr. Blamey can be contacted through the Department. Dr. Blamey is not affiliated with the Department of Health and Ageing and his contribution to the above-mentioned article does not reflect its policy. Dr. Blamey wishes to apologize for this misunderstanding.

Objectives: This review assessed current practice in the preparation of rapid reviews by health technology assessment (HTA) organizations, both internationally and in the Australian context, and evaluated the available peer-reviewed literature pertaining to the methodology used in the preparation of these reviews.

Methods: A survey tool was developed and distributed to a total of fifty International Network of Agencies for Health Technology Assessment (INAHTA) members and other selected HTA organizations. Data on a broad range of themes related to the conduct of rapid reviews were collated, discussed narratively, and subjected to simple statistical analysis where appropriate. Systematic searches of the Cochrane Library, EMBASE, MEDLINE, and the Australian Medical Index were undertaken in March 2007 to identify literature pertaining to rapid review methodology. Comparative studies, guidelines, program evaluations, methods studies, commentaries, and surveys were considered for inclusion.

Results: Twenty-three surveys were returned (46 percent), with eighteen agencies reporting on thirty-six rapid review products. Axiomatic trends were identified, but there was little cohesion between organizations regarding the contents, methods, and definition of a rapid review. The twelve studies identified by the systematic literature search did not specifically address the methodology underpinning rapid review; rather, many highlighted the complexity of the area. Authors suggested restricted research questions and truncated search strategies as methods to limit the time taken to complete a review.

Conclusions: Rather than developing a formalized methodology by which to conduct rapid reviews, agencies should work toward increasing the transparency of the methods used for each review. It is perhaps the appropriate use, not the appropriate methodology, of a rapid review that requires future consideration.