Digital health care is a rapidly growing and increasingly diverse area of medicine and social care. Health technology assessment (HTA) agencies around the world are engaging with digital health technologies and developing methods for appropriately assessing their value. Given our experience in HTA for pharmaceuticals and how methodologies must differ when assessing medical devices and diagnostics, we must be proactive in rapidly developing useful methods for assessing a variety of digital health technologies.

We will first present the results of two completed HTAs from the UK and contrast their methods and results, while looking at the UK approach to assessing digital health care in general. Next, we will present preliminary results from an ongoing consensus conference study of HTA’s for integrated digital technologies. This study will use a modified Delphi methodology to develop statements for discussion at a consensus conference. The primary research question is “how digital technologies can help build resilient healthcare systems and how HTAs can assess their value”.

An HTA conducted in the UK of an artificial intelligence-enabled ambulatory electrocardiogram device resulted in a recommendation for further real-world evidence generation. Another HTA of a digital therapeutic app for insomnia, which was also conducted in the UK, resulted in a positive recommendation but the process was lengthy and needed to be flexible. The preliminary results of an upcoming consensus conference study will also be presented. The consensus panel will consist of a range of stakeholders from digital experts, HTA practitioners, clinicians, health economists, industry representatives, and policy makers.

Digital technology is now a broad field with a wide range of very different technologies included in its definition. Questions remain as to whether digital technologies should be assessed using a traditional pharma-based framework for HTA, a MedTech model, or something else entirely. Given our past experience of developing flexible methods for HTA, we have an opportunity to develop useful recommendations and best practices from the ground up.

Health technology assessments (HTAs) of robotic assisted surgery (RAS) face several challenges in assessing the value of robotic surgical platforms. As a result of using different assessment methods, previous HTAs have reached different conclusions when evaluating RAS. While the number of available systems and surgical procedures is rapidly growing, existing frameworks for assessing MedTech provide a starting point, but specific considerations are needed for HTAs of RAS to ensure consistent results. This work aimed to discuss different approaches and produce guidance on evaluating RAS.

A consensus conference research methodology was adopted. A panel of 14 experts was assembled with international experience and representing relevant stakeholders: clinicians, health economists, HTA practitioners, policy makers, and industry. A review of previous HTAs was performed and seven key themes were extracted from the literature for consideration. Over five meetings, the panel discussed the key themes and formulated consensus statements.

A total of ninety-eight previous HTAs were identified from twenty-five total countries. The seven key themes were evidence inclusion and exclusion, patient- and clinician-reported outcomes, the learning curve, allocation of costs, appropriate time horizons, economic analysis methods, and robotic ecosystem/wider benefits.

Robotic surgical platforms are tools, not therapies. Their value varies according to context and should be considered across therapeutic areas and stakeholders. The principles set out in this paper should help HTA bodies at all levels to evaluate RAS. This work may serve as a case study for rapidly developing areas in MedTech that require particular consideration for HTAs.

Gaining the perspective of patients is invaluable in the design, management and reporting of research. As part of the process of facilitating clinical research into the effectiveness of a digital colposcope in a cervical cancer pathway, patients were involved from the outset.

Using funding made available by a Public Involvement Fund, a patient consultation group was established. The group's initial discussions informed the design of a feasibility study and funding application, which was submitted to the UK National Institute of Health Research (NIHR). A Patient and Public Involvement (PPI) representative was recruited and along with the consultation group, contributed to the ethical approvals for the study. The Patient Information Sheet and Consent Form were reviewed by the patients, to ensure readability, understandability and accessibility. The patient questionnaires and interview topics that are part of the feasibility study were also developed in conjunction with the PPI group, to make sure that women's concerns are being addressed in the research design and protocols.

The PPI consultation group's contributions helped strengthen the funding application and funding for a feasibility study was granted as part of the NIHR's Research for Patient Benefit funding scheme. Part of the grant will be used for training and reimbursement for time spent for the PPI representative. Data collection for the study is due to commence in the summer of 2021. The PPI group will be consulted at the beginning and end of the data collection period and will contribute to the data analysis and dissemination of the research output, including a Plain English Summary.

Involving patients greatly amplified the quality of the funding and ethical applications and will continue to benefit the ongoing research. Resources were widely available within the researcher's University and also through UK-wide schemes. Such resources are crucial and should be encouraged as part of all clinical research.

Zio XT Service was one of five Digital Health Technologies (DHTs) to be assessed by the National Institute for Health and Care Excellence (NICE) as part of their evaluation pilot. The King's Technology Evaluation Centre (KiTEC) act as an External Assessment Centre for NICE and worked on this pilot evaluation. The service comprises a I-Lead ECG patch, an inbuilt software that makes use of artificial intelligence (AI) algorithms to record, store and analyze ECG traces, and a team of cardiac physiologists.

Although the methods were based on NICE's existing Medical Technologies Guidance Process, they were modified to suit the assessment of DHTs. The process was split into two sections, with the option to discontinue the assessment if it was considered that insufficient evidence was available for the technology. Clinical experts and patients were consulted through the process and clinical, economic and technical evidence was considered. Costs for three care pathways were modelled.

A total of thirty relevant clinical studies were identified, with a further study being reviewed as part of a separate technical assessment, focusing on the AI component of the technology. Four of the studies were considered to be pivotal to the decision problem, one of which was a Randomized Controlled Trial. The technology was found to have a greater diagnostic yield than a standard ambulatory monitor, however diagnostic accuracy measures were absent in the literature. Three economic models were developed to represent three care pathways: patients with syncope, patients who have had a stroke or transient ischaemic attack and a third model assessing downstream costs associated with stroke treatment.

Digital Health Technologies and Artificial Intelligence Technologies pose novel and unique challenges to health technology assessment (HTA) bodies. Zio XT Service is a diagnostic tool, with both human and AI input, making it a particularly complex technology to assess. This work serves as a case-study in the evaluation of DHTs and AI and the lessons learned may contribute to the development of guidelines for such technologies.

The Evelina London Children's Hospital (ELCH) is undergoing a period of growth, including a new building planned to be completed within the next five years. Due to limited space and ambitions to be a state-of-the-art hospital, Horizon Scanning (HS) was considered important to ‘future-proof’ new facilities. As the aim of HS is to identify signals of coming change, ‘scanning’ the previous five years’ trends may be beneficial to an iterative HS methodology. Thus, it was thought that capital bids could provide a range of useful information required to make procurement decisions.

King's Technology Evaluation Centre (KiTEC) provided hospital-based HTA and HS support for the expansion of a London-based paediatric hospital. KiTEC focused on imaging technology due to its large spatial requirements and high-costs and assessed all capital bids made over the previous five years. A capital bidding system is used within GSTT to allocate funding for medical equipment that costs more than GBP5000 (USD 6540.70). Information was collated for all imaging equipment bid for over the previous five years and assessed for trends in imaging modalities and purchase costs.

A total of 135 bids were made in the period 2013-2018, eight of which were by ECLH. Bids for ultrasound equipment were most common and rose over the period. Bids for CT scanners also rose, while bids for MRI scanners and x-ray technology were consistent and bids for fluoroscopy fell. The total cost of imaging bids over the interval rose steadily from GBP5.4 million to GBP6.9 million.

Due to the lifespan of imaging technology, some trends may not emerge within a five year window. While some interesting findings were made, a ten to fifteen year period may require to be scanned for a robust analysis. This methodology is best applied in an iterative fashion along with standard HS techniques.

The strategic MedTech investment for the expansion of a central London paediatric hospital must sustain its ambitions to remain a state-of-the-art hospital, whilst implementing recent and future MedTech innovations and taking into account spatial and financial limitations. Horizon scanning (HS) is an important health technology assessment (HTA) tool to achieve these goals. To this end, we developed a methodology to help decide the suitability of investing in the following imaging-based MedTech: a hybrid theatre incorporating a biplane, intra-operative MRI (iMRI), multi-detector computed tomography (CT) scanners, and an EOS imaging system and predict the complementary technologies required for the decade to come. These technologies not only require adequate spatial resources but a significant upfront capital investment.

Three sources of information were used: i) a literature search, selected journals and other horizon scanning resources that examined current efficiency, safety, and cost-effectiveness for the proposed technologies, ii) expert elicitation in the form of user-group meetings and one-to-one discussions with clinical and service management teams and iii) hospital data consisting of audit and information from capital equipment bids.

With the exception of limited comparative data on iMRI (mainly including adults), little evidence exists to support investment in the proposed technologies. However, the decision of whether to adopt these technologies was influenced not only by existing evidence on the proposed technologies and associated cost but other factors such as local disease burdens, hospital staff requirements (training, expertise), space requirements for the new MedTech, and its impact on organizing healthcare services and hospital workflows. Complementary technologies associated with radiation monitoring image visualization and control were identified.

Strategic MedTech investment requires a holistic approach that assigns equal weight to information arising by expert elicitation and hospital audit data with existing literature evidence. The decision for adoption is heavily influenced by the clinical expertise and hospital workflows.

Mobile Health (mHealth) apps offer potential to promote greater public engagement in health, improve efficiency and open up new care pathways and models of care. However, the volume and heterogeneity of apps has led to uncertainty and lack of standardization around app definitions. Some mobile apps carry minimal risks to consumers, but others can carry significant risks. Work has been carried out to develop a framework for assessment (for example, for the NHS app library [beta version]). We discuss work helping to inform a preliminary framework of categorizing mHealth apps for proportionate assessment and validation, and the challenges involved.

A literature review was carried out to identify different types of categorizations used to define health apps and the most important dimensions for their assessment. A taxonomy of apps and a process for routing them towards appropriate methods of evaluation was developed through iterative review, discussion and refinement.

Fourteen types of mHealth apps were established which were categorized by app function and by the potential risk involved with use. Subsequently, this research suggested a method of routing apps towards the most appropriate and proportionate method of evaluation, by using four example dimensions of impact (population size, disease burden, priority of clinical condition, and innovation), and four levels of risk.

The outcome of an evaluation framework should be to enable healthcare professionals and patients to select and use safe and effective mHealth apps with greater confidence. A preliminary taxonomy and method of routing apps towards appropriate assessment are presented. Both need larger scale discussion, iterative testing and refining. This research faced significant challenges, including a high volume of heterogeneous apps with poorly standardized app definitions and associated nomenclature.