Many factors affect patient outcome after congenital heart surgery, including the complexity of the heart disease, pre-operative status, patient specific factors (prematurity, nutritional status and/or presence of comorbid conditions or genetic syndromes), and post-operative residual lesions. The Residual Lesion Score is a novel tool for assessing whether specific residual cardiac lesions after surgery have a measurable impact on outcome. The goal is to understand which residual lesions can be tolerated and which should be addressed prior to leaving the operating room. The Residual Lesion Score study is a large multicentre prospective study designed to evaluate the association of Residual Lesion Score to outcomes in infants undergoing surgery for CHD. This Pediatric Heart Network and National Heart, Lung, and Blood Institute-funded study prospectively enrolled 1,149 infants undergoing 5 different congenital cardiac surgical repairs at 17 surgical centres. Given the contribution of echocardiographic measurements in assigning the Residual Lesion Score, the Residual Lesion Score study made use of a centralised core lab in addition to site review of all data. The data collection plan was designed with the added goal of collecting image quality information in a way that would permit us to improve our understanding of the reproducibility, variability, and feasibility of the echocardiographic measurements being made. There were significant challenges along the way, including the coordination, de-identification, storage, and interpretation of very large quantities of imaging data. This necessitated the development of new infrastructure and technology, as well as use of novel statistical methods. The study was successfully completed, but the size and complexity of the population being studied and the data being extracted required more technologic and human resources than expected which impacted the length and cost of conducting the study. This paper outlines the process of designing and executing this complex protocol, some of the barriers to implementation and lessons to be considered in the design of future studies.