Skip to main content Accessibility help
×
Home

2451: Optimal study design for Diagnostic Accuracy Studies: Differential verification Versus partial verification

  • Yizhe Xu (a1), Joseph B. Stanford (a1), Kristina Allen-Brady (a1) and Nan Hu (a1)

Abstract

OBJECTIVES/SPECIFIC AIMS: To compare the accuracy and precision for estimating the diagnostic accuracies (sensitivities and specificities) between differential verification (DV) and partial verification (PV) methods. Comparisons were made under scenarios with different values of design parameters including disease prevalence, proportion of verification for positive results, proportion of verification for negative result, sensitivity and specificity of the brass standard (BS) test in DV method. Through comparing 2 different verification methods under different scenarios, we give suggestions that which verification method is optimal under different design settings. METHODS/STUDY POPULATION: For both PV and DV methods, simulation studies were performed using statistical package R, version 3.1.3. We were primarily interested in studying how the unbiasedness and precision for estimation of diagnostic accuracies (sensitivity and specificity) of an index test change with the following design parameters: disease prevalence, proportion of verification for positive test results, the proportion of verification for negative test results, and the sensitivity and specificity of a BS test. We chose different values for each of the above parameters. For each estimation, we allowed values in only 1 parameter to change by fixing the other 2 parameters, so that the effect of each design parameter on the unbiasedness and precision of both sensitivity and specificity can be determined. For the DV method, we also developed an analytical method to estimate the sensitivity and specificity of an index test using a quadratic equation with a unique solution of the specificity and sensitivity. RESULTS/ANTICIPATED RESULTS: For rare disease with prevalence less than 1%, the PV method resulted in a less biased and more precise estimate of sensitivities and specificities of the index test. If the disease prevalence was between 1% and 10%, the DV method using a BS test with moderate or high sensitivity and specificity (sensitivity and specificity >90%) resulted in a less biased and more precise estimate of diagnostic accuracies of the index test. When the disease prevalence was greater than 10%, the PV method was superior when the BS test had sensitivity and specificity <80%, and the DV method was superior when the BS test had both sensitivity and specificity >90%. When the proportion of verification of positive test results was <30% or >70%, the DV method yielded smaller bias for the estimated specificity than the PV method. However, the PV method generated a much smaller mean square error (MSE) for specificity than the DV method when the proportion of verification for positive test results was >50%. Although the disease prevalence was >10% and the proportion of verification of positive test results was <30%, the DV method resulted in a smaller MSE for specificity. DISCUSSION/SIGNIFICANCE OF IMPACT: Disease prevalence and proportions of verification for patients with positive and negative test results influence the accuracy of a new diagnostic test. If a new index test for a very rare disease is evaluated, the PV method should be used for assessing the performance of the index test. When a disease prevalence is >1%, the DV method will result in a less biased and more precise estimate of diagnostic accuracy of an index test, if the BS test itself used in the DV method has large specificity and specificity. One concern of using BS test for the DV method is the clinical cost. Depending on the disease type, the BS tests usually are imperfect, but may be less aggressive and/or less expensive than the gold standard test. Moreover, as all clinical examinations require professional personnel to perform, verification of the index test for relative large proportion of a large cohort of patients could become a burden on human resources. Thus, the future research of the optimal design method for a diagnostic accuracy study should be based on the comprehensive cost-effectiveness analysis.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      2451
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      2451
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      2451
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

2451: Optimal study design for Diagnostic Accuracy Studies: Differential verification Versus partial verification

  • Yizhe Xu (a1), Joseph B. Stanford (a1), Kristina Allen-Brady (a1) and Nan Hu (a1)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed