Book contents
- The Handbook of Medical Image Perception and Techniques, Second Edition
- Reviews
- The Handbook of Medical Image Perception and Techniques
- Copyright page
- Dedication
- Contents
- Contributors
- 1 Medical Image Perception
- Part I Historical Reflections and Theoretical Foundations
- Part II Science of Image Perception
- Part III Perception Metrology
- 14 Designing Perception Experiments
- 15 Receiver Operating Characteristic Analysis: Basic Concepts and Practical Applications
- 16 Multireader ROC Analysis
- 17 Memory Effects and Experimental Design
- 18 Observer Models as a Surrogate to Perception Experiments
- 19 Implementation of Observer Models
- 20 Value and Limitations of Observer Models
- Part IV Clinical Performance Assessment
- Part V Computational Perception
- Part VI Applied Perception
- Index
- References
16 - Multireader ROC Analysis
from Part III - Perception Metrology
Published online by Cambridge University Press: 20 December 2018
Book contents
- The Handbook of Medical Image Perception and Techniques, Second Edition
- Reviews
- The Handbook of Medical Image Perception and Techniques
- Copyright page
- Dedication
- Contents
- Contributors
- 1 Medical Image Perception
- Part I Historical Reflections and Theoretical Foundations
- Part II Science of Image Perception
- Part III Perception Metrology
- 14 Designing Perception Experiments
- 15 Receiver Operating Characteristic Analysis: Basic Concepts and Practical Applications
- 16 Multireader ROC Analysis
- 17 Memory Effects and Experimental Design
- 18 Observer Models as a Surrogate to Perception Experiments
- 19 Implementation of Observer Models
- 20 Value and Limitations of Observer Models
- Part IV Clinical Performance Assessment
- Part V Computational Perception
- Part VI Applied Perception
- Index
- References
Summary
A summary is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
- Type
- Chapter
- Information
- The Handbook of Medical Image Perception and Techniques , pp. 245 - 262Publisher: Cambridge University PressPrint publication year: 2018
References
Abbey, C.K., Samuelson, F.W., Gallas, B.D. (2013). Statistical power considerations for a utility endpoint in observer performance studies. Acad Radiol, 20, 798–806.Google Scholar
Abbey, C.K., Gallas, B.D., Boone, J.M., Niklason, L.T., Hadjiiski, L.M., Sahiner, B., Samuelson, F.W. (2014). Comparative statistical properties of expected utility and area under the ROC curve for laboratory studies of observer performance in screening mammography. Acad Radiol, 21, 481–490.CrossRefGoogle ScholarPubMed
Bamber, D. (1975). Area above ordinal dominance graph and area below receiver operating characteristic graph. J Math Psychol, 12, 387–415.CrossRefGoogle Scholar
DeLong, E.R., DeLong, D.M., Clarke-Pearson, D.L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44, 837–845.CrossRefGoogle ScholarPubMed
Dorfman, D.D., Berbaum, K.S. (2000a). A contaminated binormal model for ROC data – part III. Initial evaluation with detection ROC data. Acad Radiol, 7, 438–447.Google Scholar
Dorfman, D.D., Berbaum, K.S. (2000b). A contaminated binormal model for ROC data – part II. A formal model. Acad Radiol, 7, 427–437.Google Scholar
Dorfman, D.D., Berbaum, K.S., Metz, C.E. (1992). Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol, 27, 723–731.Google Scholar
Dorfman, D.D., Berbaum, K.S., Lenth, R.V., Chen, Y.F. (1999). Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: split plot experimental design. SPIE Proc Ser, 3663, 91–99.Google Scholar
Dorfman, D.D., Berbaum, K.S., Brandser, E.A. (2000). A contaminated binormal model for ROC data – part I. Some interesting examples of binormal degeneracy. Acad Radiol, 7, 420–426.Google Scholar
Gallas, B.D. (2006). One-shot estimate of MRMC variance: AUC. Acad Radiol, 13, 353–362.Google Scholar
Gallas, B.D. (2017). iMRMC v3p3 Application for Analyzing and Sizing MRMC Reader Studies (computer software). Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, Silver Spring, MD. Available for download from https://github.com/DIDSR/iMRMC/releases.Google Scholar
Gallas, B.D., Bandos, A., Samuelson, F.W., Wagner, R.F. (2009). A framework for random-effects ROC analysis: biases with the bootstrap and other variance estimators. Commun Statistics – Theory Methods, 38, 2586–2603.Google Scholar
Hanley, J.A., McNeil, B.J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.CrossRefGoogle Scholar
Hillis, S.L. (2007). A comparison of denominator degrees of freedom methods for multiple observer ROC analysis. Statistics Med, 26(3), 596–619.Google Scholar
Hillis, S.L. (2014). A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data. Statistics Med, 33, 330–360.Google Scholar
Hillis, S.L., Berbaum, K.S. (2005). Monte Carlo validation of the Dorfman-Berbaum-Metz method using normalized pseudovalues and less data-based model simplification. Acad Radiol, 12, 1534–1542.Google Scholar
Hillis, S.L., Schartz, K.M. (2015). Demonstration of multi- and single-reader sample size program for diagnostic studies software. In: Mello-Thoms, C.R., Kupinski, M.A. (eds.) Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment, Proc SPIE 9416 94160E.Google Scholar
Hillis, S.L., Obuchowski, N.A., Schartz, K.M., Berbaum, K.S. (2005). A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Statistics Med, 24, 1579–1607.Google Scholar
Hillis, S.L., Schartz, K.M., Berbaum, K.S. (2007). DBM MRMC 3.0 for SAS (computer software). Available at: http://perception.radiology.uiowa.edu.Google Scholar
Hillis, S.L., Berbaum, K.S., Metz, C.E. (2008). Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol, 15, 647–661.CrossRefGoogle ScholarPubMed
Hillis, S.L., Obuchowski, N.A., Berbaum, K.S. (2011). Power estimation for multireader ROC methods: an updated and unified approach. Acad Radiol, 18, 129–142.Google Scholar
Jiang, Y.L., Metz, C.E., Nishikawa, R.M. (1996). A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology, 201, 745–750.Google Scholar
McClish, D.K. (1989). Analyzing a portion of the ROC curve. Med Decis Making, 9, 190–195.Google Scholar
Metz, C.E., Pan, X.C. (1999). “Proper” binormal ROC curves: theory and maximum-likelihood estimation. J Math Psychol, 43, 1–33.CrossRefGoogle ScholarPubMed
Obuchowski, N.A. (1995). Multireader receiver operating characteristic studies: a comparison of study designs. Acad Radiol, 2, 709–716.CrossRefGoogle ScholarPubMed
Obuchowski, N.A. (2009). Reducing the number of reader interpretations in MRMC studies. Acad Radiol, 16, 209–217.Google Scholar
Obuchowski, N.A., Rockette, H.E. (1995). Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations. Commun Statistics – Simul Comput, 24, 285–308.Google Scholar
Obuchowski, N.A., Gallas, B.D., Hillis, S.L. (2012). Multi-reader ROC studies with split-plot designs: a comparison of statistical methods. Acad Radiol, 19, 1508–1517.CrossRefGoogle ScholarPubMed
Pan, X.C., Metz, C.E. (1997). The ‘‘proper’’ binormal model: parametric receiver operating characteristic curve estimation with degenerate data. Acad Radiol, 4, 380–389.Google Scholar
Schartz, K.M., Hillis, S.L. (2018). Multi-reader sample size program for diagnostic studies (computer software). Available for download from http://perception.radiology.uiowa.edu.Google Scholar
Schartz, K.M., Hillis, S.L., Pesce, L.L., Berbaum, K.S. (2018). OR-DBM MRMC (Version 2.51) (computer software). Available for download from http://perception.radiology.uiowa.edu.Google Scholar
Shao, J., Dongshen, T. (1995). The Jackknife and Bootstrap. New York, NY: Springer-VerlagGoogle Scholar
Van Dyke, C.W., White, R.D., Obuchowski, N.A., Geisinger, M.A., Lorig, R.J., Meziane, M.A. (1993). Cine MRI in the diagnosis of thoracic aortic dissection. 79th RSNA Meetings, Chicago, IL, November 28–December 3.Google Scholar
Zeger, S.L., Liang, K.Y. (1986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics, 42, 121–130.Google Scholar