The adoption of electronic health records (EHR), which contain large volumes of aggregated longitudinal clinical data, guarantees substantial benefits, including better care, improved safety and decreased clinical risks; however, it is also associated with significant costs and large technical and organizational impacts. For these reasons, it is important to conduct a comprehensive evaluation of health care delivery outcomes. The purpose of the study is to gather evidence on the safety and overall effectiveness of EHR implementation at Bambino Gesù Children's Hospital.

A decision-oriented health technology assessment (HTA) method was applied to assess the technology on clinical, technical, organizational, economic, legal, ethical and safety domains. It is a new implementation of the EUnetHTA CoreModel integrated with the Analytic Hierarchy Process. The evaluation structure was a hierarchical decision tree filled with indicators of a technology's performance, where each indicator was weighted based on its relative impact on decision making. Finally, the alternatives’ ranking was defined. A subgroup of these indicators has been included in a checklist for the evaluation of six EHR implementation projects. This checklist was used as a tool by each involved professional during demo sessions.

The assessment took into consideration all the recommendations about the benefits and disadvantages of EHR. In particular, EHR seems to offer many benefits in terms of safety and clinical effectiveness, such as improved continuity, quality of care and accessibility of the data. Its implementation resulted in important organizational outcomes such as EHR configuration, learning curve and training; usability was the main technical characteristics of the technology taken into account. Finally, legal aspects on privacy and data security assumed a key role.

A detailed technology evaluation of EHR has permitted the hospital's decision-makers to knowingly assess its introduction in the hospital.