Clinicians working in the field of congenital and paediatric cardiology have long felt the need for a common diagnostic and therapeutic nomenclature and coding system with which to classify patients of all ages with congenital cardiac disease. In the early 2000s, this desire was heightened by high-profile enquiries examining the outcome of paediatric cardiac surgery at the congenital cardiology centres in Bristol, United Kingdom
; Sydney, Australia
; Winnipeg, Canada
; Denver, Colorado
; and Lexington, Kentucky.
The incorporation of effective clinical governance and best practice into our speciality requires the harvesting of accurate and validated data on the diagnosis, treatment, and outcome of these patients from prenatal life through to adulthood. Such data gathering facilitates the comparison of outcomes following interventions between individual centres, while taking into account the mix of cases involved, any accompanying risk factors and comorbidities, as well as postprocedural complications. Benchmarking against those units who perform best allows analysis of relevant and genuine factors underlying differing outcomes, and instigation of improvements, in terms of both mortality and morbidity. For this objective to be achieved, it is essential to have a comprehensive and standardised system of coding and classification, using mutually exclusive and unambiguous terms. The system must be easy to use, preferably in multiple languages, be digitally compatible with different software database systems, and also fulfil the needs and expectations of widely different cultures of practice.
A cohesive and comprehensive system of nomenclature, which is suitable for setting a global standard for multicentric analysis of outcomes and stratification of risk, has been published, namely the freely available International Paediatric and Congenital Cardiac Code (IPCCC),
as developed and copyrighted by the International Society for Nomenclature of Paediatric and Congenital Heart Disease (International Nomenclature Society). The IPCCC is available for download without cost from the Society’s website at http://www.IPCCC.net. The IPCCC has been used extensively as the coding system underlying international and national comparisons of outcomes between individual centres undertaking congenital heart surgery and transcatheter interventions.
Unfortunately, while coding systems such as the IPCCC have been used by specialist clinicians for quality assurance, outcome analysis, and research, health administrators often follow their own coding schemes, most commonly the International Classification of Diseases (ICD) of the World Health Organization. In so doing, efforts are thwarted to have a uniform identification of patients with CHD using a single up-to-date and coordinated nomenclature system.
An ideal classification scheme for CHD would be one that could serve the purposes of both administrators and clinicians. In the last decade, the development of such a scheme was made possible when it was announced that a new 11th iteration of the International Classification of Diseases (ICD-11) was planned, and that input from clinicians and researchers was sought. After discussion with senior representatives of the ICD-11 revision team in 2007, the International Nomenclature Society was given the mandate to provide a list of CHD to be incorporated into ICD-11, and in 2010 the Society set about achieving this task. The final product of 7 years of deliberation is published in this issue of Cardiology in the Young.
This article reviews the history of the International Classification of Diseases, and of the IPCCC. It discusses the content of the congenital cardiology section of ICD-11, a classification tree that will permit the widespread use of a single diagnostic list for CHD both for administrative and clinical purposes.
The International Classification of Diseases
Although the gathering of population-based information, or censuses, originated in Babylonian times (3800 BC), it was not until the early 17th century that the focus shifted away from taxation and military conscription to causes of death, with the weekly publication from 1603 through the 1830s of the Bills of Mortality in London.
In 1662, these statistics were first systematically analysed by John Graunt (1620–1674), when he estimated, for instance, the mortality of children before the age of 6 years to be 36%, an estimate later proven to be quite accurate.
In 1837, the General Register Office of England and Wales was established, with William Farr (1807–1883) as its first medical statistician. Farr revolutionised the principles of statistical classification and nomenclature, emphasising the need for a common international lexicon to allow the epidemiological study of diseases and thereby their causes, while incorporating medical advances. His initial report, published in 1839, resonates through to the late 20th century:
“The advantages of a uniform statistical nomenclature, however imperfect, are so obvious, that it is surprising that no attention has been paid to its enforcement in Bills of Mortality. Each disease has, in many instances, been denoted by three or four terms, and each term has been applied to as many different diseases: vague inconvenient names have been employed, or complications registered instead of primary diseases. The nomenclature is of as much importance in this department of inquiry, as weights and measures in the physical sciences, and should be settled without delay.”
The eminence of Farr in the field was recognised at the first International Statistical Congress, held in Brussels in 1853, when he proposed a classification of causes of mortality based on five groups, which formed the basis of the 139 categories accepted by the Congress in 1855:
∙ epidemic diseases
∙ constitutional, or general, diseases
∙ local diseases arranged according to anatomical site
∙ developmental diseases
∙ diseases that are the direct result of violence.
These held for the subsequent four revisions through to 1886, and the later International Classification of Diseases, particularly the principle of classifying causes of death by anatomical site, in distinction to generalised processes of disease. As there was limited take-up of this classification, in 1891 the International Statistical Institute (the former Congress) commissioned a committee chaired by Jacques Bertillon (1851–1922), Chief of Statistical Services of the City of Paris, to create what became the Bertillon [International] Classification of Causes of Death, based on the principles propounded by Farr, with associated sequential numeric codes.
Over the following decades, this classification scheme was adopted by many countries in the Americas and Europe, with conferences for revision occurring roughly decennially to take note of medical advances (Fig 1).
Figure 1 This bar chart documents the time interval between each Revision of the International Classification of Diseases (ICD), 1900–2018. Bertillon presented his (International) Classification of Causes of Death at the meeting of the International Statistical Institute in Chicago in 1893, where it was adopted by several cities and countries. In 1898, the American Public Health Association recommended its adoption in North America and that the classification be revised every 10 years. The First International Conference to revise the Bertillon Classification of Causes of Death was held in Paris in 1900. In 1909 non-fatal diseases were added (morbidity) and in 1948 the World Health Organization (WHO) took over its promotion and management with the sixth revision of the International Classification of Diseases, Injuries and Causes of Death. The horizontal lower bar indicates the number of CHD terms listed in each ICD version.
Farr was also instrumental in promoting the need for a similar system to account for non-fatal disease, submitting a list of these to the second International Statistical Congress in 1856. He stated the need to
“extend the same system of nomenclature to diseases which, though not fatal, cause disability in the population, and now figure in the tables of the diseases of armies, navies, hospitals, prisons, lunatic asylums, public institutions of every kind, and sickness societies, as well as in the census of countries like Ireland, where the diseases of all the people are enumerated.”
At the fourth International Statistical Congress held in 1860, Florence Nightingale urged its adoption for the tabulation of hospital morbidity in her paper Proposals for a uniform plan of hospital statistics.
In 1900 and 1909, at the first two Conferences to revise the Bertillon International Classification of Causes of Death, a parallel classification of diseases for use in statistics of sickness was adopted, but with limited scope. The Health Organization of the League of Nations, and the International Statistical Institute, recommended at the fourth and fifth revision conferences of the International List of Causes of Death – as the International Classification of Diseases was then known, held in 1928 and 1938, respectively, that individual countries develop and promote their own systems of classification for morbidity statistics, using the Causes of Death structure as a template. It was not until the sixth revision conference, held in Paris in 1948, when the recently constituted World Health Organization took over its promotion and management, that a single comprehensive list was endorsed by the First World Health Assembly as the International Classification of Diseases, Injuries and Causes of Death, the sequential four-digit code being augmented by lists of causes of accidents and injuries (E-codes). The ninth International Classification of Diseases revision (ICD-9), in 1975, saw further expansion to nearly 18,000 codes, as well as structural change, with a new five-digit sequential numeric designation for each entity, the optional fifth digit facilitating greater detail where needed by subspecialities. The dagger and asterisk notation system was introduced to allow the specific linkage of anatomical sites of disease to more generalised diseases. The first International Classification of Procedures in Medicine was also published, in response to international pressure for a lead in this important area.
By the time of the tenth revision (ICD-10), in 1993, the decennial revision schedule was abandoned (Fig 1) because of the enormous amount of work and international consultation involved with each revision, and a new chapter-based alphanumeric coding structure was introduced, retaining the overall four-digit structure. ICD-10, whose full name is International Statistical Classification of Diseases and Related Health Problems, consists of over 14,400 basic codes, and is currently used in 115 countries for reporting mortality. It has been translated into 43 languages, with the code ensuring that the data derived are language independent. Adoption of ICD-10 has been very variable, with the United States of America only abandoning ICD-9 in 2015. This is largely because of the expense of adapting data processing systems with each ICD revision, and despite the fact that World Health Organization member states are expected to use the most current version of ICD to report death and illness, as agreed by an international treaty in 1967. Many countries have created their own versions of ICD-9 and ICD-10 to produce Clinical Modifications (ICD-9-CM, ICD-10-CM), so as to expand the lexicons sufficiently for their own bespoke clinical use and administrative purposes, such as billing. In the United States Clinical Modification, for example, there are over 68,000 ICD-CM codes with an up to seven-digit coding structure. The International Classification of Diseases is one of the World Health Organization Family of International Classifications designed to support national and international health information systems, statistics, and evidence.
The other two key World Health Organization classification systems are the International Classification of Functioning, Disability and Health, and the International Classification of Health Interventions, which is under development.
The development of classification schemes specific to the malformed heart has a shorter history, which began with Maude Abbott’s pioneering work in Montreal in the early 1900s. Her 1936 Atlas of Congenital Cardiac Disease was the first formal attempt to classify CHD.
Following this initial effort, centres across the world developed or adopted different schemes to produce more hospital-specific databases. By the late 20th century, a multitude of centre-derived congenital heart classification systems for internal, national, and even international comparisons were developed, but these were dissimilar enough to preclude the large-scale studies needed to benchmark the outcomes of the heterogeneous population of patients with congenitally malformed hearts. Aided by advances in information technology to facilitate the entry of data and exchange of information, two independent international endeavours began in the 1990s and resulted in the near simultaneous publication in 2000 of two separate congenital cardiology nomenclature lexicons
The European Paediatric Cardiac Code of the Association for European Paediatric and Congenital Cardiology
The nomenclature system of the International Congenital Heart Surgery Nomenclature and Database Project of the Society of Thoracic Surgeons in North America, and the European Congenital Heart Defects Database of the European Congenital Heart Surgeons Foundation – renamed the European Congenital Heart Surgeons Association in 2003.
Both systems of nomenclature included a comprehensive Long List, with thousands of terms, and a so-called Short List designed to be used as part of a minimum data set for audit and research purposes. Both Long Lists mapped fully to their respective Short Lists. The nearly simultaneous publication of these two complementary systems of nomenclature led to the problematic situation of having two lexicons that were to be widely adopted, with the potential risks of invalidating multicentric projects owing to confusion between the two systems, and duplicate or inaccurate entries within institutions.
Following a meeting between the involved society representatives in October, 2000 in Frankfurt, Germany, the International Nomenclature Committee for Paediatric and Congenital Heart Disease was established to include representatives of the three societies, as well as representatives from the remaining continents of the world – Africa, Australia, Asia, and South America. It was tasked to work in partnership and produce a reconciliatory bidirectional map between the two systems. In 2005, the International Nomenclature Committee became the International Nomenclature Society and was legally incorporated as a not-for-profit corporation in Canada (Fig 2). Fortuitously, the International Congenital Heart Surgery Nomenclature and Database Project did not feature a numerical code, and the six-digit numerical code derived from the European Paediatric Cardiac Code was used as the backbone for mapping the two systems.
Figure 2 The logo of the International Society for Nomenclature of Paediatric and Congenital Heart Disease.
The feasibility of this project was established by the creation of a rule-based bidirectional crossmap between the two Short Lists by the Nomenclature Working Group of the International Nomenclature Society.
Over the next 8 years, the Nomenclature Working Group met 10 times, over a combined period of 47 days, to achieve the main goal of mapping the two comprehensive lists to each other to create the two dominant versions of IPCCC.
∙ The IPCCC derived from the European Paediatric Cardiac Code of the Association for European Paediatric and Congenital Cardiology;
∙ The IPCCC derived from the International Congenital Heart Surgery Nomenclature and Database Project of the Society of Thoracic Surgeons, the European Congenital Heart Surgeons Association, and the European Association for Cardio-Thoracic Surgery.
For all terms within the two versions of the IPCCC, a unique six-digit code corresponds to a single entity, whether it be a morphological phenotype, procedure, symptom, or genetic syndrome, with the mapped terms in each of the two versions being synonymous to each other. By 2013, there were 12,168 items in the IPCCC version derived from the European Paediatric Cardiac Code, and 17,176 in the IPCCC version derived from the International Congenital Heart Surgery and Nomenclature Database Project, with additional hundreds of qualifiers, some specific (such as anatomical sites), and some generic (such as gradings of severity).
It is primarily the Short Lists, rather than the Long Lists, of the two crossmapped versions of the IPCCC that have been used for analyses of multi-institutional and international outcomes following operations and procedures for patients with congenitally malformed hearts, with close to a million patients in registries worldwide. Both versions of the IPCCC Short Lists have been used to develop empirical systems for the stratification of risk following surgical procedures, based on the operation type, comorbidities, and diagnoses, for quality assurance and quality improvement purposes.
Both risk adjustment systems depend upon the IPCCC for all variables, to ensure a common lexicon of terms between institutions submitting data, and both perform better than the systems based on the subjective assessment of risk.
Although the Long Lists of each version have been crossmapped, the two Short Lists emanating from their respective Long-List versions are not the same in terms of structure or content. One Short List has a distinct surgical orientation, with the other serving more cardiological needs. The International Nomenclature Society recognised this disparity, and believed that the creation of a congenital cardiology subset within ICD-11 would help resolve the situation by presenting a single comprehensive and hierarchical list of diagnostic terms that could serve all communities dealing with CHD.
Apart from the treaty-bound necessity of the World Health Organization “to establish and revise as necessary international nomenclatures of diseases, of causes of death and of public health practices”,
ICD-10 is no longer fit for purpose to address the needs of the digital healthcare systems in the 21st century with the move to the electronic health record and a paperless environment. This requires a standardised system for gathering and retaining accurate and detailed clinical information, free from duplication of data entry, enabling comparative studies across units and nations of clinical and cost-related outcomes for risk-adjusted benchmarking to best practice. The prerequisite for this accomplishment is an underlying, all encompassing, common nomenclature and system of coding for healthcare, with clinician-led and validated entry of data. The ICD-11 mission, therefore, is “To produce an international disease classification that is ready for electronic health records that will serve as a standard for scientific comparability and communication”.
Its purpose is to allow the systematic recording, analysis, interpretation, and comparison of mortality and morbidity data collected in different countries or areas and at different times. The ICD-11 project began in earnest in 2007 and is planned to achieve its first stage of implementation in May, 2018, after endorsement at the 71st World Health Assembly of the ICD-11 version for Mortality and Morbidity Statistics, known as ICD-11-MMS. Uniquely for the International Classification of Diseases, it incorporates textual definitions, and there are logical linkages to other standard terminologies, particularly the Systematised Nomenclature of Medicine Clinical Terms (SNOMED-CT).
In addition, the system is to be explicitly stratified to cater to different users, such as primary care, traditional medicine, and public health, producing so-called linearisations or “Tabular Lists”. For the first time, the revision process has moved away from reliance on large meetings of national delegations of health statisticians with manual archiving – curation, limited field testing, working solely in the English language, wherein those who voiced their opinion strongest would dominate the content of the paper-based output – “decibel” diplomacy. In contrast, the ICD-11 revision process is dependent upon international expert clinicians, with digital curation, the incorporation of wide peer review, and extensive field testing. Although working in English, multiple translations are integral to the process. The work was divided into content-specific Topic Advisory Groups, with related Working Groups led by Managing Editors and chaired by specialist clinicians with an intentionally wide geographic spread (Fig 3). From 2009 through to 2016, the Managing Editor coordinated a series of meetings, some face-to-face but mostly teleconferences, to populate and update the classification of specialist areas, beginning with the hierarchical structure and terms within ICD-10, and initially producing an evolving alpha draft. In 2012, a beta draft was published online (https://icd.who.int/dev11/f/en), coinciding with the authoring process moving to a web-based platform for its entire content. This is known as the ICD Collaborative Authoring Tool (iCAT), whereby the ICD-11 content and structure is enhanced and edited by the designated Managing Editors.
The tool also allows online global peer review and submission of comments by both the authors and worldwide interested parties in the current field-testing stage.
Figure 3 Diagram illustrating the workflow for the collaborative authoring ICD-11 revision process, led by the Revision Steering Group and its Small Executive Group. Branches of ICD classification are assigned to at least one Content-Specific Topic Advisory Group (TAG), each led by a Managing Editor and two clinical chairs. Congenital cardiology falls under the remit of the Cardiovascular Working Group of the Internal Medicine TAG with collaborative input and ratification from the Pediatric TAG. The members of each working group and TAG have the task of entering the content of the diseases in their domain of expertise, such as textual definition, synonyms, and so on. Cross-Sectional TAGs are involved with specific linearisations for their area, using terms from the Content-Specific TAGs as appropriate, the dominant one being the Joint Linearization for Morbidity and Mortality Statistics (JLMMS). Data entry by Managing Editors uses the ICD Collaborative Authoring Tool (iCAT) web platform, which can also be used for commentary and feedback during field testing.
From the start, clinicians involved in the Topic Advisory Groups have been encouraged to enlist the advice of specialist Societies to aid the process, and to ensure that the content was both up-to-date and had Societal endorsement. This has resulted in a huge increase in the number of individual terms within ICD-11, with secondary expanse of the hierarchical structure when compared with ICD-10. An example is congenital cardiology, with an increase from 73 CHD diagnostic terms in ICD-10 to 318 codes submitted by the International Nomenclature Society for incorporation into ICD-11. Populating ICD-11 by the content-specific Topic Advisory Groups was not always without controversy, with at times, for example, heated and prolonged discussions between the Rare Diseases Topic Advisory Group and several Internal Medicine Topic Advisory Workings Groups, including the Cardiovascular Working Group, over the hierarchy to be included or excluded.
ICD-11 uses an ontological content model derived from computer science. Ontology in this context can be defined as the explicit, operational description of the concepts within a domain – in other words, its qualities, properties, and attributes. In ICD-11, the evidence-based attributes of each individual disease are delineated, with the current emphasis on textual definitions, synonyms, inclusions, and exclusions. There is the ambition to extend the content to include diagnostic criteria, genetics, causal mechanisms, functional impact, and so on. This ontological content model will result in a common vocabulary with robust data standards, enabling a shared understanding and exchange of information (data) between individuals and institutions, permitting interoperability and like-for-like comparisons across all fields of medicine.
An important additional feature in ICD-11 is the ability to add a large-range additional clinically useful detail to a primary diagnosis or “stem code”, using “extension codes”, to produce a “cluster code”, also called post-coordination. Extension codes are housed in a specific ICD-11 Chapter (Chapter X) and include details of severity, size, temporality, aetiology, for example pathogen causing infection, anatomical detail, type of device related to a postprocedural complication, and so on. These equate to qualifier codes in the IPCCC. Other extension codes cover more administrative and broader health interactions, such as reason for health encounter/admission, for example illness, screening, diagnosis confirmed/ ruled out, and patient related such as family history of a condition. Of course not all extension codes can be used with every stem code. It is planned that each stem code will have a list of valid extension codes visible for use when such detail is available and required for patient management or research purposes.
ICD-11 and the International Nomenclature Society
Up until the ICD-11 iteration, congenital cardiac diseases remained very poorly represented. Outdated terminology, inconsistent logic, and little appreciation of the inherent complexity of lesions have meant that the ICD system of classification has never been robust enough for the purposes of relevant clinical governance. In the ninth and tenth revisions, there are a total of only 29 and 73 individual codes, respectively, for congenital cardiovascular lesions (Fig 1). Evidence suggests that the validity of coding of lesions seen in the congenitally malformed heart via the ninth revision of the International Classification of Diseases (ICD-9), as currently used in administrative databases in the United States, is poor.
For example, in a series of 373 infants with congenital cardiac defects at Children’s Hospital of Wisconsin, investigators reported that only 52% of the cardiac diagnoses in the medical records had a corresponding code from the International Classification of Diseases in the hospital discharge database.
A subsequent study of infants and fetuses with cardiac defects, born in Atlanta from 1988 to 2003, compared administrative coding using ICD-9 and the clinical nomenclature used in the Society of Thoracic Surgeons Congenital Heart Surgery Database.
This study concluded that analyses based on the 29 codes available in ICD-9 are likely to “have substantial misclassification” of congenital cardiac disease. This prompted the organisation of a linkage study using patient data (2004–2010) from the Society of Thoracic Surgeons Congenital Heart Surgery (STS-CHS) Database, a clinical registry, and the Pediatric Health Information Systems administrative database in order to evaluate differential coding/classification of operations between data sets and subsequent impact on outcomes assessment.
The cohort included 59,820 patients from 33 centres. There was a greater than 10% difference in the number of cases identified between data sources for half of the benchmark operations. The negative predictive value of the administrative, versus clinical, data was high (98.8–99.9%); the positive predictive value was lower (56.7–88.0%). These differences translated into significant differences in case ascertainment and outcomes assessment. These ranged from an underestimation of mortality associated with common arterial trunk repair by 25.7% in the administrative versus clinical data (7.01% versus 9.43%; p=0.001) to an overestimation of mortality associated with ventricular septal defect repair by 31.0% (0.78% versus 0.60%; p=0.1). An extension to this study demonstrated similar concerns about federal recommendations in the United States of America that suggest administrative data be used to assess hospital-level mortality rates as a measure of centre performance.
Surgical volume was underreported, with lower mortality using administrative data compared with registry data, with rank positions differing by at least five positions for eight of the 33 centres evaluated. This produced a change in mortality tertile classification – high, middle, or low mortality – for 6 units, and a change in statistical outlier classification for four hospitals. Despite their limitations with respect to CHD, nonetheless, both ICD-9 and ICD-10 remain in use, mostly for the purpose of “billing”, returns to central governments, and crude epidemiological surveillance.
Discussions began in 2007, in Tokyo Japan, between the International Nomenclature Society and representatives of the steering group responsible for the creation of ICD-11, administered by the World Health Organization. Both parties were interested in greatly expanding the CHD section within this new iteration of the International Classification of Diseases, in order to include enough terms to be useful for clinicians and researchers who work in the field of congenital cardiology. By mutual agreement, the International Nomenclature Society took on the task of developing the list of diagnostic terms for CHD for ICD-11 that was to include definitions and synonyms for each diagnosis.
Developing the CHD list for ICD-11
To fulfil this new mandate, the International Nomenclature Society formed the Definitions Working Group, which was tasked with creating a comprehensive list of diagnostic terms to be used in ICD-11 to describe CHD in sufficient detail for clinical use by specialists in the field. This was to be achieved by, and from the perspective of, an international body of experts from congenital and paediatric cardiology, cardiovascular surgery, and cardiac morphology.
In ICD-11, congenital and paediatric cardiac diseases are under the auspices of the Cardiovascular Working Group of the Internal Medicine Topic Advisory Group, in liaison with the Pediatric Topic Advisory Group (Fig 3). The Definitions Working Group worked therefore in liaison with both Topic Advisory Groups to produce a large subset of terms derived from the IPCCC and presented in a hierarchical manner. The Definitions Working Group was also tasked with providing definitions and synonyms for each term. Definitions were to be used further to validate the selection of different codes so as to accurately describe and code CHD in all its forms.
Methodological approach to defining the ICD-11 terms
Early in the deliberations of the Definitions Working Group of the International Nomenclature Society in 2010, considerable discussion was devoted to achieving agreement on what constitutes a useful definition. Fundamentally, the group sought definitions that provided a concise set of findings that were essential to the diagnosis, while avoiding inclusion of variable elements of the disease. For example, although right ventricular dilation is usually associated with an atrial septal defect, it is the presence of a communication between the atrial chambers that constitutes the definition. This is also an example of defining anatomic abnormalities based on morphology, whereas physiologic abnormalities such as aortic regurgitation are defined based on their functional abnormalities. The definitions were then systematically formulated during nine meetings of the committee in the ensuing 7 years. The group first created a systematic hierarchical list of terms headed by “congenital anomaly of heart and great vessels” (Table 1). Although the terminological hierarchy was developed first, the definitions process at times resulted in reconsideration of this hierarchy. In addition to defining the terms themselves, the definitions committee also took responsibility for development of the related elements required by the ICD-11 system. These include acceptable synonyms, unacceptable synonyms, relevant historical terms that are useful for search purposes, and abbreviations. Some areas of the congenital cardiology sections were found to be prepopulated with terms and synonyms as found in ICD-10 or input from non-specialists in congenital cardiology. These discrepancies were rectified on an item-by-item basis, submitting the rationale for change to the Managing Editor of the Internal Medicine Topic Advisory Group. The ICD-11 system also includes a “notes” or commentary option, permitting the inclusion of explanatory material to describe the phenotype in more detail, to further explain differences between individual codes, and to point the user to an appropriate alternative term (Table 2, italicised text after formal definition). This option was only occasionally used by the Definitions Working Group, although as field testing of the ICD-11 system proceeds the committee may use this section to clarify usage.
Table 1 Systematic hierarchical list of congenital cardiology terms submitted to the World Health Organization for inclusion in the Foundation Component of the 11th revision of the International Classification of Diseases.
Table 2 Systematic list of congenital cardiology terms submitted to the World Health Organization for inclusion in the Foundation Component of the 11th revision of the International Classification of Diseases with their definitions and acceptable synonyms,
and their IPCCC six digit numbers.
The procedural method adopted by the Definitions Working Group was to devote 2.5 to 3 days at in-person committee meetings to achieve consensus on the individual definitions. Before each meeting, each attendee was asked to develop proposed definitions for a group of terms and these preliminary definitions then served as the basis for discussion and refinement by the committee at large. Outside information, including reference to earlier literature, imaging data, intraoperative experience, and observations from pathologic specimens, was often used. The struggle for consensus was indeed a struggle, but generally with sufficient time the committee achieved agreement. There was also no policy precluding reconsideration of definitions that had previously been agreed to. Several of the more challenging terms achieved consensus definitions that were then subsequently reconsidered. The terminology and definitions for the ventricular septal defects, for example, were reconsidered at nearly all of the in-person meetings.
One of the most important obstacles to progress in establishing common terminology is the historical differences in usage at many of the centres, and the fear that deviations from the existent standards could lead to rejection of terminology that was at odds with the “preferred” local terminology. In cases in which different names are used for the same entity, a cross-mapping between the terminologies can be used to avoid conflicts, effectively treating the terms as synonyms, as the code itself is independent of differing terminologies for the same phenotype. Many of these synonyms have been documented as such in the ICD-11 system to help avoid this conflict, as shown in Table 2. When the same name has been used to refer to different entities, however, the committee found it preferable to drop the name altogether. Inevitably, there are also cases in which there is fundamental disagreement as to the specific morphologic or physiologic nature of the disease, leading different experts to conclude that specific names are wrong. Despite these differences, generally a cross-mapping of these terms is possible such that, although unresolvable divergences of opinion will persist, the ability to appropriately group these entities for epidemiological and outcome analyses can still be preserved.
After 7 years the Definitions Working Group of the International Nomenclature Society began working on ICD-11 congenital cardiology content, the Group has submitted to the World Health Organization a list of over 300 diagnostic terms, all items being based upon the IPCCC and accompanied by definitions and synonyms. This list of CHD terms, along with definitions, synonyms, and the IPCCC six-digit numerical codes, is presented in this issue of Cardiology in the Young. It is accessible on the World Health Organization website in its beta draft form https://icd.who.int/dev11/f/en#/, under ‘Diseases of the circulatory system’, and then ‘Developmental anomalies of the circulatory system’. They are also available on the International Nomenclature Society website www.ipccc.net, where additional lists of the IPCCC will be published with the terms contained in ICD-11 being highlighted in the long lists of both versions of the IPCCC.
Navigating ICD-11 to find CHD terms, synonyms and definitions
The full ICD-11 content is known as the Foundation Component, and represents the entire ICD universe. The diagnostic terms for CHD that were submitted by the International Nomenclature Society reside in the Foundation Component of ICD-11. The Foundation Component has been divided into 26 sections, including one on Developmental Anomalies, which includes Structural developmental anomalies of the circulatory system. The list of diagnostic terms for CHD is found under a further subsection “Structural developmental anomaly of heart and great vessels”.
It is worth noting that, unlike previous ICD versions, all Foundation Component terms are not necessarily mutually exclusive, as double parenting is permitted such that an entity may exist in more than one hierarchy. For example, heart tumours are found under the sections both for neoplasms and diseases of the circulatory system, but should retain the same numerical designation.
The list of CHD terms that has been submitted by the International Nomenclature Society to the World Health Organization is published in two tables in this issue of Cardiology in the Young. Table 1 contains the list of diagnostic terms presented in a hierarchical manner. Table 2 contains a column of the same list of terms, but includes corresponding definitions and comments for each term, synonyms, as well as the six-digit IPCCC number that has been attributed to each term by the International Nomenclature Society. When ICD-11 is officially released, another set of ICD-specific numeric codes will be given to each term, but as yet these are not available for publication. In the discussion below, the bracketed number that follows each term is its IPCCC six-digit identification code.
As stated previously, the list of congenital heart malformations submitted by the International Nomenclature Society for incorporation into ICD-11 contains 318 terms. The hierarchy begins with the “level 0” term: Structural developmental anomaly of heart and great vessels (01.01.59). There are then 10 “level 1” items starting with those describing position and connections, and followed, in segmental sequential order, by veins, atria, atrioventricular valves, ventricles, arterial valves, and great arteries.
Congenital anomaly of position or spatial relationships of thoraco-abdominal organs (03.01.13).
Congenital anomaly of an atrioventricular or ventriculo-arterial connection (01.03.09).
Congenital anomaly of mediastinal vein (04.00.07).
Congenital anomaly of an atrium or atrial septum (05.00.02).
Congenital anomaly of an atrioventricular valve or atrioventricular septum (06.00.15).
Congenital anomaly of a ventricle or the ventricular septum (07.00.00).
Functionally univentricular heart (01.01.22).
Congenital anomaly of a ventriculo-arterial valve or adjacent regions (09.04.29).
Congenital anomaly of great arteries including arterial duct (09.04.28).
Congenital anomaly of coronary arteries (09.46.03).
Each of the 10 “level 1” terms is subdivided into finer and finer branches as needed, to a maximum of seven levels of detail and a total of 318 terms (see Table 1).
Although the CHD list is designed to enable the diagnosis of any congenital heart anomaly, 10 “normal” terms were included within the list. These are to be used when describing a complex congenital malformation, as some parts of the anomaly include normal findings. These “normal” terms are listed below, with the secondary terms denoting notable synonyms.
Usual atrial arrangement – atrial situs solitus (01.03.00)
Right-hand pattern ventricular topology – D-loop ventricles (02.03.11)
Aortic root posterior and rightward to pulmonary root (02.06.06)
Spiralling course of great arteries (02.07.00)
Concordant atrioventricular connections (01.04.01)
Concordant ventriculo-arterial connections (01.05.00)
Patent oval foramen – patent foramen ovale (05.03.01)
Patent arterial duct – patent ductus arteriosus (09.27.21)
Left aortic arch (09.28.22)
What the CHD list of ICD-11 does not contain
The section Structural developmental anomaly of heart and great vessels includes only congenital malformations. Thus, heart lesions that are not clearly congenital in nature do not appear in this section. For example, “Congenital aortic valvar stenosis” (09.15.01) appears in the structural developmental section, but “Aortic valvar stenosis” (09.15.13) does not. Terms dealing with cardiac lesions that are not necessarily congenital in nature can be found in the ICD-11 section dealing with Diseases of the Circulatory System. With few exceptions, particularly cardiac diseases that are not paediatric or congenital in nature, the cardiac terms listed in both these sections are found also in the IPCCC. Acquired cardiac lesions are also listed in Diseases of the Circulatory System within ICD-11, and those acquired cardiac lesions that pertain to CHD were developed and submitted separately for incorporation into ICD-11 by the International Nomenclature Society.
Principles used in drawing up the IPCCC and the CHD section of ICD-11
As described previously,
members of the International Nomenclature Society followed certain principles in developing the IPCCC and they have been incorporated into the Foundation Component submission for ICD-11.
The International Nomenclature Society submitted the CHD list of terms in English. American and British spelling was used interchangeably and words with dual spellings were treated as synonyms. A French translation of the CHD list has been completed by a sub-committee of the International Nomenclature Society. The World Health Organization will be assuming the task of translating the list into other languages, although it is anticipated that advice will be sought to corroborate CHD-related terms and definitions in other languages from the International Nomenclature Society, whose members speak a wide variety of languages. Of note, Latin terms have largely been translated into English, and are almost never used as a primary term. The Latin version of a term has by and large been listed as a synonym (e.g. “Common arterial trunk”=“Truncus arteriosus”=09.01.01).
Right and left
When discussing cardiac chambers, such as atriums and ventricles, and spatial relationships, the words left and right can be confusing. Rules were therefore created to provide consistency and accuracy of descriptive terms of anatomical phenotypes. For cardiac chambers, the words left and right do not imply sidedness or position but refer to morphologically left and morphologically right. Thus, left ventricle and left atrium mean the morphologically left ventricle and morphologically left atrium, respectively. If one wishes to describe the position or sidedness of a cardiac chamber, it is necessary to use terms such as left-sided ventricle. In contrast, when describing the superior caval vein, and using the adjectival prefix left or right, it is the spatial position that is being alluded to, rather than any other connection or phenotypic variation that may exist.
“Other” and “Unspecified”
The term “other”, meaning “other specified”, is largely eschewed in the IPCCC with the logic that if one cannot find a particular lesion in a given list one needs to go to the higher-order term to code for an entity. The term “congenital right ventricular diverticulum” exists in the IPCCC but was not included in ICD-11. If one wishes to code for this lesion from ICD-11, the higher-order term to be used will be “Right ventricular anomaly” but not “Right ventricular anomaly, other”, as depending on the list from which this term was taken “Right ventricular anomaly, other” may have different meanings and content.
All terms in the nomenclature lists theoretically end in Not Otherwise Specified “(NOS)” or “(unspecified)”, in that one can always create further subdivisions for virtually any diagnosis or procedure. It follows that the generic term on its own is self-explanatory, without the need for such clarifying nomenclature being affixed. Although a subtle distinction could be made to suggest that intentionally stating “unspecified” may mean that there really is no additional specification possible, it would be impossible to know whether this is the case or whether the coder or physician involved simply did not bother to code more specifically to save time and effort. These suffixes are consequently not necessary and are not used within the Foundation level listing in ICD-11. Despite advice against this by the International Nomenclature Society, nonetheless, the ICD-11 Revision Steering Group, along with the Mortality and Morbidity Topic Advisory Groups, have opted for the retention of both “Other” and “Unspecified”, both designated as “residual categories”, within the Mortality and Morbidity Statistics linearisation, partly because there have always been such terms in the lexicon, going right back to 1900. Consequently, the International Nomenclature Society greatly favours the use of the congenital heart list in its unabridged and unaltered form as found in the Foundation Component of ICD-11, and published in this issue of Cardiology in the Young, where such “residual” terms do not feature.
Conforming to ICD standards
The International Classification of Diseases has certain rules that caused some of the IPCCC terms to be changed very slightly when they were incorporated into ICD-11.
1. No parentheses are allowed within the list. Thus, if a term such as “Truncus Arteriosus (Common arterial trunk)” is listed as such in the IPCCC, signifying that they are identical and used interchangeably in congenital cardiology parlance, one of the terms was consigned to the status of synonym in another column of ICD-11. This was particularly the case for Latinisms.
2. The words “and” and “or” in ICD-11 are used with their meanings in formal logic, and are not found together as “and/or”. Thus “and” means that both terms must be present in the condition, whereas “or” means that either one, or the other, or both terms may be present. Thus “or” equates to “and/or”.
3. In ICD-11, as in the IPCCC, no eponyms are followed by a possessive apostrophe and “s”. Therefore, in ICD-11, what may be known to many as Ebstein’s anomaly of the tricuspid valve will read Ebstein anomaly of the tricuspid valve.
4. In ICD-11, the plural form of a term in parentheses – for example, “Congenital anomaly of coronary artery(ies)” – is not allowed, and thus only the singular form is identified in ICD-11, even though a given entry in IPCCC may include both the singular and plural forms of the entity.
These minor changes have not led to any fundamental changes in definitions of the terms or in the six-digit code associated with each term in the IPCCC but are noted here to explain the slight discrepancies that may be noticed between the terms in ICD-11 and the IPCCC.
Extension codes, qualifiers, and modifiers
The IPCCC contains added codes for qualifying lesions, be it for size, degree, or subtypes. In the version of the IPCCC derived from the European Paediatric Cardiac Code, these are known as “qualifiers”, whereas in the version of the IPCCC derived from the International Congenital Heart Surgery Nomenclature and Database Project, these are called “modifiers”. As noted above, in ICD-11, there will be the opportunity to qualify entries within the CHD list through the use of Extension Codes that are available in a separate chapter of ICD-11. Thus, a ventricular septal defect can be qualified as “small”, “moderate”, or “large” by using the qualifiers provided within the Extension Code chapter (chapter X). Extension codes can never be used on their own and must always be linked to a “stem” diagnosis code in both ICD-11 and the IPCCC, whereas a diagnosis may feature cluster of extension codes to describe different characteristics, such as severity and time in life; for instance, congenital aortic stenosis can be described as “severe” and “neonatal”.
The CHD diagnostic tree, the foundation component of ICD-11, and the morbidity and mortality linearisation in ICD-11
It is from the Foundation Component of ICD-11 that linearisations, also now known as Tabular Lists, are constructed. This is achieved by building a specific hierarchical tabular list from selected entities, including speciality versions of ICD-11 and country-specific modifications. Figure 3 shows the six cross-sectional Topic Advisory Groups whose purpose is to ensure structural uniformity and oversee the creation of specific linearisations, such as the joint linearisation known as morbidity and mortality statistics. The Morbidity and Mortality Statistics tabular list is designed for international mandatory reporting of data for global disease-related statistics and can be considered as the ICD-11 equivalent of ICD-10 in terms of scope, targeting mostly statistical and administrative stakeholders.
The Morbidity and Mortality Statistics Linearisation is the one part of ICD-11 that will be issued in hard copy, and because of this strict content restrictions have been imposed. As a direct consequence of these restrictions, the list of 318 congenital heart terms that was submitted by the International Nomenclature Society for the Foundation Component of ICD-11 has been pared down to 94 terms in the CHD section of the Morbidity and Mortality Statistics linearisation. Nevertheless, it is the Foundation component of ICD-11 with its full complement of 318 terms for CHD that the International Nomenclature Society will be promoting internationally to serve as a diagnostic tree to be used by all communities interested in congenital cardiology, including administrators. As countries adopt ICD-11 into their healthcare systems, the International Nomenclature Society will want to ensure uniformity in data gathering by encouraging the use of the full list of CHD terms found in the Foundation component of ICD-11 and as published in this issue of Cardiology in the Young, over the use of shortened or altered lists that may be found elsewhere in ICD-11.
ICD-11 maintenance and future enhancements
In the past, the latest version of the International Classification of Diseases could not be changed until the next iteration many years later. This is not the case for ICD-11, where there will be a mechanism for enhancing the diagnostic lists as frequently as annually. Field testing of the CHD list in ICD-11 has begun. The International Nomenclature Society fully anticipates the addition of new terms, and more accurate or elaborate definitions are likely to ensue. As “keepers” of the IPCCC and the ICD-11 list of CHD terms, requests for changes to the list will be handled through the International Nomenclature Society through a bidirectional relationship with the World Health Organization ICD-11 team. Requests for change, known as proposals, are to be submitted online via the ICD-11 website, either by an individual or a Society, to a Proposal Platform where they will be triaged to ensure first that they are correct and complete. If the proposal requires a scientific or medical view, as opposed to an error or classification query, it will be forwarded to the newly established Medical and Scientific Advisory Committee, whose 15 to 20 members are expert clinicians in a variety of fields drawn from the previous Topic Advisory Groups, which were discontinued in October, 2016. The new committee fortuitously includes a member of the International Nomenclature Society. A proposal relating to congenital cardiology could then be relayed to the International Nomenclature Society for advice in its role as a specialist editorial reviewer group. If a change is recommended, the proposal is then sent to the Classification and Statistics Advisory Committee for assessment, particularly with respect to how the proposal may affect the classification structure, as well as the collection of data for mortality and morbidity statistics, before forwarding to an implementation team. Revisions to the proposal may be made at any stage to improve it, or it may be rejected. The International Nomenclature Society, therefore, continues its commitment to this long-term enterprise, even though it has achieved the goal of providing a list of congenital heart anomaly terms, with definitions and synonyms, for ICD-11.
Future projects of the International Nomenclature Society
The International Nomenclature Society has recently been tasked with a new World Health Organization project related to the ICD-11 sister classification the International Classification of Health Interventions. The last publication in this respect from the World Health Organization was in 1978, namely the International Classification of Procedures in Medicine alongside ICD-9. A revision has not since been undertaken, as it was judged to be too complex a task. Many countries who might resource this, furthermore, had already developed their own medical procedure coding system. Despite this, in 2001 a need was perceived for those countries without such a system. After a failed attempt to use the Australian Classification of Health Interventions, in 2006 a decision was made to commence a parallel project to ICD-11 to create a classification for interventions. A health intervention is here defined as “an act performed for, with or on behalf of a person or a population, whose purpose is to assess, improve, maintain, promote or modify health, functioning or health conditions”. It is profession neutral, and includes interventions related to mental and allied health, nursing and public health, in addition to surgery and medical procedures. Owing to a lack of resources, progress has been slow, and specialist clinical input limited. This has now changed, and there is now engagement with clinicians from a variety of backgrounds, as well as a tight timeline with the hope of a finished product by 2020. It is already evident that much work is to be done to encompass the needs of congenital cardiology with respect to transcatheter interventions and cardiac surgery, as well as electrophysiology. The International Nomenclature Society has, therefore, formed a new working group specifically for this task. The first meeting of this Group will take place in Miami, Florida, between 31 January and 4 February, 2018, to discuss the goals of this new mandate and how to achieve them. Other working groups of the Society will be continuing their work in an ongoing and parallel manner, including the Archiving Working Group, which is developing a virtual atlas to illustrate each of the congenital heart malformations within the ICD-11 lexicon. These will link to the definitions, and have the potential to create hyperlinks to these from the World Health Organization ICD-11 website. The Definitions Working Group will continue to define or perfect the already existing definitions of all the items in the IPCCC.
The latest iteration of the International Classification of Diseases of the World Health Organization (ICD-11) is due to begin its release in 2018. Through the International Nomenclature Society, experts in the field were called upon to create a list of diagnostic terms for CHD to be included in ICD-11. This list, drawn from the Society’s IPCCC, was submitted to the World Health Organization in a hierarchical manner. It contains definitions and synonyms for each term further to enhance the accuracy of coding.
Although the widely used IPCCC comes in two versions, which have been crossmapped one to the other, whereby either version can be used with easy convertibility from one to the other, the so-called “Short Lists” that have been extracted from each version vary considerably in the number of terms and the actual diagnoses selected for data compilation. With the incorporation of a thorough and unique list of diagnostic terms for congenital heart malformations into the internationally used classification scheme of ICD-11, it is expected that the many societies and organisations that use various subsets of the IPCCC for the purpose of data gathering adopt the congenital heart list of ICD-11 as their “short list” for all purposes, including those related to administrative, research, and clinical work.
The International Nomenclature Society anticipates that the use of its diagnostic list for CHD in ICD-11 for future cooperative multi-institutional studies and epidemiological surveys will facilitate accurate data gathering, and ultimately enable the optimisation of the quality and effectiveness of healthcare for our patients with congenital cardiac malformations.
The Definitions Working Group of the International Society for Nomenclature of Paediatric and Congenital Heart Disease wishes to acknowledge Rodney Franklin, chief liaison between the Nomenclature Society and the World Health Organization, and chairpersons Steven Colan and Henry (Hal) Walters, for their steadfast work and leadership since the inception of the ICD-11 CHD project.
The Definitions Working Group gratefully acknowledges the “Heart of a Child, Michigan” through the Children’s Hospital of Michigan Foundation, and “Angela’s Big Heart for Little Kids Fund” through the Montreal Children’s Hospital Foundation, for donations given in support of the ICD-11 CHD Nomenclature project.
Conflicts of Interest
Smith, R. All changed, changed utterly. Br Med J
1998; 316: 1917–1918.
4. The Report of the Manitoba Pediatric Cardiac Surgery Inquest: An Inquest into twelve deaths at the Winnipeg Health Sciences Centre in 1994. Retrieved October 14, 2013 from http://www.pediatriccardiacinquest.mb.ca/.
Franklin, RC, Beland, MJ, Krogmann, ON. Mapping and coding of nomenclatures for paediatric and congenital heart disease. Cardiol Young
2006; 16: 105–106.
Brown, KL, Crowe, S, Franklin, RC, et al. Trends in 30-day mortality rate and case mix for paediatric cardiac surgery in the UK between 2000 and 2010. Open Heart
2015; 2: e000157.
Jacobs, JP, Mayer, JE Jr, Mavroudis, C, et al. The Society of Thoracic Surgeons Congenital Heart Surgery Database: 2016 Update on Outcomes and Quality. Ann Thorac Surg
2016; 101: 850–862.
Jacobs, JP, O’Brien, SM, Pasquali, SK, et al. The Society of Thoracic Surgeons Congenital Heart Surgery Database Mortality Risk Model: Part 2-clinical application. Ann Thorac Surg.
2015; 100: 1063–1070.
Jacobs, JP, Mayer, JE Jr, Mavroudis, C, et al. The Society of Thoracic Surgeons Congenital Heart Surgery Database: 2017 Update on Outcomes and Quality. Ann Thorac Surg
2017; 103: 699–709.
Greenwood, M. Medical statistics from Graunt to Farr. Cambridge University Press, Cambridge, 1948.
Farr, W. First annual report. Registrar General of England and Wales, London, 1839: 99.
17. Farr W. Sixteenth annual report. Registrar General of England and Wales, London, 1856: 75.
Nightingale, F. Proposals for a Uniform Plan of Hospital Statistics. Fourth International Statistical Congress, London, 1860.
Franklin, RC, Jacobs, JP, Krogmann, ON, et al. Nomenclature for congenital and paediatric cardiac disease: historical perspectives and The International Pediatric and Congenital Cardiac Code. Cardiol Young
2008; 18 (Suppl 2): 70–80.
Franklin, RCG, Anderson, RH, Daniëls, O, et al. The European Paediatric Cardiac Code. Cardiol Young
1999; 9: 633–657.
Association for European Paediatric Cardiology. The European Paediatric Cardiac Code. Cardiol Young
2000; 10 (Suppl 1): 1–146.
Mavroudis, C, Jacobs, JP. Congenital heart surgery nomenclature and database project: overview and minimum dataset. Ann Thorac Surg
2000; 69: S2–S17.
Mavroudis, C, Jacobs, JP. (eds). Congenital Heart Surgery Nomenclature and Database Project. Ann Thorac Surg 2000: 69 Suppl: S1-S372.
Béland, M, Jacobs, JP, Tchervenkov, CI, Franklin, RCG. The International Nomenclature Project for Paediatric and Congenital Heart Disease: Report from the Executive of The International Working Group for Mapping and Coding of Nomenclatures for Paediatric and Congenital Heart Disease. Cardiol Young
2002; 12: 425–430.
Franklin, RCG, Jacobs, JP, Tchervenkov, CI, Béland, MJ. The International Nomenclature Project for Congenital Heart Disease: bidirectional crossmap of the short lists of the European Paediatric Cardiac Code and the International Congenital Heart Surgery Nomenclature Database Project. Cardiol Young
2002; 12: 431–435.
Béland, MJ, Franklin, RCG, Jacobs, JP, et al. Update from The International Working Group for Mapping and Coding of Nomenclatures for Paediatric and Congenital Heart Disease. Cardiol Young
2004; 14: 225–229.
O’Brien, SM, Clarke, DR, Jacobs, JP, et al. An empirically based tool for analyzing mortality associated with congenital heart surgery. J Thorac Cardiovasc Surg
2009; 138: 1139–1153.
Crowe, S, Brown, KL, Pagel, C, et al. Development of a diagnosis- and procedure-based risk model for 30-day outcome after pediatric cardiac surgery. J Thoracic Cardiovasc Surg
2013; 145: 1270–1278.
Rogers, L, Brown, KL, Franklin, RC, et al. Improving risk adjustment for mortality after pediatric cardiac surgery: The UK PRAiS 2 model. Ann Thorac Surg
2017; 104: 211–219.
World Health Organization. Basic Documents, 47th ed. World Health Organization, Geneva, 2009.
Tudorache, T, Falconer, S, Nyulas, C, et al. Supporting the Collaborative Authoring of ICD-11 with WebProtégé. AMIA Annu Symp Proc
2010; 2010: 802–806.
Cronk, CE, Malloy, ME, Pelech, AN, et al. Completeness of state administrative databases for surveillance of congenital heart disease. Birth Defects Res A Clin Mol Teratol
2003; 67: 597–603.
Frohnert, BK, Lussky, RC, Alms, MA, Mendelsohn, NJ, Symonik, DM, Falken, MC. Validity of hospital discharge data for identifying infants with cardiac defects. J Perinatol
2005; 25: 737–742.
Strickland, MJ, Riehle-Colarusso, TJ, Jacobs, JP, et al. The importance of nomenclature for congenital heart disease: implications for research and evaluation. Cardiol Young
2008; 18 (Suppl 2): 92–100.
Pasquali, SK, Peterson, ED, Jacobs, JP, et al. Differential case ascertainment in clinical registry versus administrative data and impact on outcomes assessment for pediatric cardiac operations. Ann Thorac Surg
2013; 95: 197–203.
Pasquali, SK, He, X, Jacobs, JP, et al. Measuring hospital performance in congenital heart surgery: administrative vs. clinical registry data. Ann Thorac Surg
2015; 99: 9332–9338.