To outline the methodological issues associated with the assessment of newborn screening for severe combined immunodeficiency, which was conducted to address the policy question of whether this test should be added to an existing newborn screening panel.

We conducted a systematic review of published primary studies and critically appraised the methodological quality of selected studies (1).

Fifteen studies were included; six focused on screening test performance, and seven on treatment effectiveness, and two on the effectiveness of a newborn screening program. The methodological issues identified included: (i) Overall poor methodological quality ratings of included studies using the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2). This tool was originally developed for assessing diagnostic accuracy studies where subjects usually receive both index test and reference standard so a 2×2 table can be constructed; however it is almost impossible to apply this cross-sectional approach to studies of a screening test for a rare disease like severe combined immunodeficiency. (ii) Case control design using healthy controls could inflate estimates of test accuracy compared to studies using a cohort of consecutive patients, possibly due to spectrum effects and limited-challenge bias. This type of study is useful in the early phase of test development, but estimates of test accuracy based on this type of study should be interpreted with caution. (iii) Some screening programs reported no false negatives, indicating a sensitivity of 100 percent. However, lack of a systematic search for “missed cases” created uncertainty in arriving at a true value for the sensitivity. (iv) Variations in inclusion of pre-term infants, races/ethnicities, and screening protocols made it difficult to compare screening test performance across different studies.

Although severe combined immunodeficiency screening was the first addition to the US Recommended Uniform Screening Panel following an evidence-based review process, caution needs to be exercised when interpreting research findings due to important methodological issues.