Health Technology Wales (HTW) is a relatively new Health Technology Assessment (HTA) agency which focuses on non-medicines. In common with other HTA organizations, it identifies and appraises a range of technologies. However, HTW is also looking beyond the publication of guidance, to assess the adoption of advice and its eventual impact.

HTW commissioned development of an Evaluation Plan from independent experts (Matter of Focus). A literature review was carried out to inform an options appraisal of methods for assessing impact. The selected approach was Contribution Analysis, which estimates the counterfactual through engagement of stakeholders.

Whilst it is too early to report the full impact of HTW's guidance, a number of activities have taken place to prepare for evaluation. The core HTW team developed a series of logic models to describe the anticipated impact, the mechanisms by which it would be achieved, and key assumptions. Stakeholders were consulted for insight from a range of perspectives, and to manage expectations. This was achieved through individual interviews, presentation and discussion at committee meetings, and the sharing of written materials for feedback. This information was collated to populate bespoke software (OutNav). The collection of data relating to processes, outputs and outcomes is already an ongoing routine task of researchers and support staff.

HTW has an opportunity to build impact evaluation into its culture from the beginning. This will facilitate the future reporting of HTW's influence using a well-designed, evidence-based approach. Furthermore, this pioneering work will clearly demonstrate the value of HTA to funders, commissioners, governments, and other decision-making bodies.