Wales has ambitious health, wealth, and innovation policies and a clear goal to use the economic muscle of the Welsh National Health Service (NHS) to support its strong life sciences sector. Health Technology Wales (HTW) has a clear remit to appraise technologies over the span of their lifecycle from innovation to obsolescence. HTW is collaborating with the Bevan Commission through their national Health Technology Exemplars (HTEs), which partners NHS and industry stakeholders to strengthen innovation within the Welsh health system.

Health technology assessment (HTA) methods were used to produce topic exploration reports for assessing the evidence underpinning applicant innovations. A “Dragons’ Den” expert panel was convened to select the successful HTEs.

Fourteen Bevan HTEs were awarded funds, which were matched by industry partners. Application of HTA methods resulted in more critical consideration of technology value propositions, including: developing pull models of innovation focused on delivering health technology solutions for current problems facing NHS Wales; supporting early dialogue between the NHS and industry partners around demonstrating evidence of improved patient outcomes; and focusing on transformative rather than incremental innovation. The most promising innovations will progress to rapid HTA, where the evidence generated will be used to develop guidance for NHS Wales.

HTA methods were productively deployed at the innovation phase of the technology lifecycle to support evidence-informed allocation of scarce innovation resources. In this way, HTW is working with key stakeholders to identify and offer early support to the most promising innovations, with the aim of expediting their adoption and realizing health benefits for patients as quickly as possible. The Bevan Commission has partnered with HTW to routinely build in HTA and evidence considerations in its future innovation calls and competitions. Thus, HTW has established a “feeder” pipeline for assessing bottom-up service-led innovations and encouraging evidence consideration throughout the lifecycle of innovative technologies.

Randomized trials and similarly robust research methods generate evidence in carefully controlled settings, often with strict inclusion criteria. But patients in the ‘real world’ often have multiple comorbidities, and treatments are delivered within diverse environments. Trials are also difficult to fund, and rarely collect longitudinal data. Because of these, and other limitations, researchers are increasingly recognizing the inherent value of real world evidence (RWE). This is not only true for pharmaceutical products, and may have even more relevance in the evaluation of complex interventional procedures and non-medicines healthcare technologies. The Idea, Development, Exploration, Assessment, Learning (IDEAL) Framework guides the developmental ‘pipeline’ of surgical (and other) procedures, as well as medical device research (IDEAL-D). IDEAL informs the production of high-quality evidence of safety and effectiveness, but there is potential to further expand its applications.

Our aim is to investigate the feasibility of using of RWE alongside the IDEAL Framework in the assessment of procedures and devices. Methodological experts from the IDEAL Collaboration, HTA agencies and other healthcare research organisations are contributing their unique perspectives and experiences to explore these methods. As part of this work, Cedar Healthcare Technology Research Centre (Cedar) has attempted to retrospectively apply the IDEAL criteria to a series of RWE projects conducted on behalf of the National Institute for Health and Care Excellence (NICE) Interventional Procedures and Medical Technologies Evaluation Programmes.

Cedar's experience indicates that there may be options for using retrospective routinely-collected linked data and other existing sources to address some of the requirements of IDEAL. Likewise, the IDEAL Framework is expected to be a helpful reference when designing new databases and clinical registries for prospective collection of relevant and informative evidence. Examples from several projects will be shared at the Health Technology Assessment International (HTAi) conference.

Initial signs are that there are likely to be a number of ways in which IDEAL and RWE could complement one another.

Evidence supporting the use of pacemakers is well established. However, evidence about the optimal use of pacemaker telemonitoring for disease management in heart failure is not. Health Technology Wales (HTW) held a national adoption event to encourage implementation and best practice in use of pacemaker telemonitoring in the National Health Service (NHS) Wales to improve patient outcomes in heart failure.

Multi-stakeholder national adoption workshop using a mixture of expert presentations, case studies and interdisciplinary group and panel discussions to agree key actions to understand the value and promote optimal use of pacemakers for remote disease monitoring in patients with heart failure in Wales.

The workshop was attended by forty-five senior professionals with an interest in improving care of patients with heart failure. Actions to progress included: providing a centralized Welsh system to support technical issues that arise with telemonitoring; considering interoperability with other NHS Wales systems; encouraging value-based procurement with collection of a core outcome set; agreeing implementation issues with both professionals and patients; audit to understand experience, resource use and outcomes; and sharing manufacturer evidence on the accuracy of telemanagement algorithms. It was suggested that these actions be progressed via an All-Wales multi-stakeholder approach, led by the Welsh Cardiac Network.

Developing a more agile, lifecycle approach to technology appraisal is currently advocated; recalibrating the focus from technology assessment to technology management across the complete technology lifecycle. HTW will endeavour through regular adoption events to facilitate such a paradigm shift that aims to understand value and optimise use of evidence-based technologies.

Health Technology Wales (HTW) is a relatively new Health Technology Assessment (HTA) agency which focuses on non-medicines. In common with other HTA organizations, it identifies and appraises a range of technologies. However, HTW is also looking beyond the publication of guidance, to assess the adoption of advice and its eventual impact.

HTW commissioned development of an Evaluation Plan from independent experts (Matter of Focus). A literature review was carried out to inform an options appraisal of methods for assessing impact. The selected approach was Contribution Analysis, which estimates the counterfactual through engagement of stakeholders.

Whilst it is too early to report the full impact of HTW's guidance, a number of activities have taken place to prepare for evaluation. The core HTW team developed a series of logic models to describe the anticipated impact, the mechanisms by which it would be achieved, and key assumptions. Stakeholders were consulted for insight from a range of perspectives, and to manage expectations. This was achieved through individual interviews, presentation and discussion at committee meetings, and the sharing of written materials for feedback. This information was collated to populate bespoke software (OutNav). The collection of data relating to processes, outputs and outcomes is already an ongoing routine task of researchers and support staff.

HTW has an opportunity to build impact evaluation into its culture from the beginning. This will facilitate the future reporting of HTW's influence using a well-designed, evidence-based approach. Furthermore, this pioneering work will clearly demonstrate the value of HTA to funders, commissioners, governments, and other decision-making bodies.

Effective communication is vital for engaging stakeholders in health technology assessment (HTA), as well as the successful dissemination and adoption of HTA research and guidance. As a relatively new organization, Health Technology Wales (HTW) has an ideal opportunity to take an effective, strategic approach to communication and stakeholder engagement from the outset.

HTW commissioned Pagoda Public Relations to develop an informed communications strategy and delivery framework. The strategy used OASIS methodology for public relations planning: Objectives, Audience insight, Strategy, Implementation, and Scoring (evaluation). Initial objectives were developed with input from the HTW team and members of the HTW Assessment Group and Appraisal Panel. Stakeholder insights were collected through an online survey and telephone interviews. These insights were used to inform the communications strategy and framework, outlining key audiences, key messages, communication objectives, methods, tactics, and evaluations.

Seven key objectives were identified, each of which were supported by recommended actions. These were underpinned by the key aims and messages reflecting how we will achieve these objectives. National Health Service boards, government, clinicians, the technology and research sector, patients, and the general public were identified as priority audiences. Various different communication channels and activities were identified, aimed at various audiences. These included the website, social media, traditional media, and exhibitions or workshops, as well as targeted e-mail dissemination of guidance. Evaluation of HTW communications will be aligned with the wider HTW evaluation strategy, and evidence will be recorded through OutNav software (Matter of Focus Ltd).

HTW is committed to a strategic, effective approach to communication and engagement. We now have an audience-informed communications strategy and plan that outlines our key objectives, and how to achieve and evaluate these objectives. Successful implementation will raise awareness of and value in profile and outputs of HTW, both in Wales and internationally.