Observer Variability in BI-RADS Ultrasound Features and Its Influence on Computer-Aided Diagnosis of Breast Masses
Read full paper at:
http://www.scirp.org/journal/PaperInformation.aspx?PaperID=53047#.VLSJdsnQrzE
http://www.scirp.org/journal/PaperInformation.aspx?PaperID=53047#.VLSJdsnQrzE
Author(s)
Laith R. Sultan1*, Ghizlane Bouzghar1, Benjamin J. Levenback1, Nauroze A. Faizi1, Santosh S. Venkatesh2, Emily F. Conant1, Chandra M. Sehgal1
Affiliation(s)
1Department of Radiology, University of Pennsylvania, Philadelphia, USA.
2Department of Electrical Engineering, University of Pennsylvania, Philadelphia, USA.
2Department of Electrical Engineering, University of Pennsylvania, Philadelphia, USA.
ABSTRACT
Objective:
Computer classification of sonographic BI-RADS features can aid
differentiation of the malignant and benign masses. However, the
variability in the diagnosis due to the differences in the observed
features between the observations is not known. The goal of this study
is to measure the variation in sonographic features between multiple
observations and determine the effect of features variation on
computer-aided diagnosis of the breast masses. Materials and Methods:
Ultrasound images of biopsy proven solid breast masses were analyzed in
three independent observations for BI-RADS sonographic features. The
BI-RADS features from each observation were used with Bayes classifier
to determine probability of malignancy. The observer agreement in the
sonographic features was measured by kappa coefficient and the
difference in the diagnostic performances between observations was
determined by the area under the ROC curve, Az, and interclass
correlation coefficient. Results: While some features were repeatedly
observed, κ = 0.95, other showed a significant variation, κ = 0.16. For
all features, combined intra-observer agreement was substantial, κ =
0.77. The agreement, however, decreased steadily to 0.66 and 0.56 as
time between the observations increased from 1 to 2 and 3 months,
respectively. Despite the variation in features between observations the
probabilities of malignancy estimates from Bayes classifier were robust
and consistently yielded same level of diagnostic performance, Az was
0.772-0.817 for sonographic features alone and 0.828-0.849 for
sonographic features and age combined. The difference in the
performance, ΔAz, between the observations for the two groups was small
(0.003-0.044) and was not statistically significant (p < 0.05).
Interclass correlation coefficient for the observations was 0.822 (CI:
0.787-0.853) for BI-RADS sonographic features alone and for those
combined with age was 0.833 (CI: 0.800-0.862). Conclusion: Despite the
differences in the BI-RADS sonographic features between different
observations, the diagnostic performance of computer-aided analysis for
differentiating breast masses did not change. Through continual
retraining, the computer-aided analysis provides consistent diagnostic
performance independent of the variations in the observed sonographic
features.
Cite this paper
References
Sultan,
L. , Bouzghar, G. , Levenback, B. , Faizi, N. , Venkatesh, S. , Conant,
E. and Sehgal, C. (2015) Observer Variability in BI-RADS Ultrasound
Features and Its Influence on Computer-Aided Diagnosis of Breast Masses.
Advances in Breast Cancer Research, 4, 1-8. doi: 10.4236/abcr.2015.41001.
| [1] | Kopans, D.B. (1992) The Positive Predictive Value of Mammography. American Journal of Roentgenology, 158, 521-526. http://dx.doi.org/10.2214/ajr.158.3.1310825 |
| [2] | Jiang,
Y.L., Nishikawa, R.M., Schmidt, R.A., Metz, C.E., Giger, M.L. and Doi,
K. (1999) Improving Breast Cancer Diagnosis with Computer-Aided
Diagnosis. Academic Radiology, 6, 22-33. http://dx.doi.org/10.1016/S1076-6332(99)80058-0 |
| [3] | Shen,
W.C., Chang, R.F., Moon, W.K., Chou, Y.H. and Huang, C.S. (2007) Breast
Ultrasound Computer-Aided Diagnosis Using BI-RADS Features. Academic
Radiology, 14, 928-939. http://dx.doi.org/10.1016/j.acra.2007.04.016 |
| [4] | Shen, W.C., Chang, R.F. and Moon, W.K. (2007) Computer Aided Classification System for Breast Ultrasound Based on Breast Imaging Reporting and Data System (BI-RADS). Ultrasound in Medicine & Biology, 33, 1688-1698. http://dx.doi.org/10.1016/j.ultrasmedbio.2007.05.016 |
| [5] | Moon, W.K., Lo, C.M., Chang, J.M., Huang, C.S., Chen, J.H. and Chang, R.F. (2012) Computer-Aided Classification of Breast Masses Using Speckle Features of Automated Breast Ultrasound Images. Medical Physics, 39, 6465-6473. http://dx.doi.org/10.1118/1.4754801 |
| [6] | Moon, W.K., Lo, C.-M., Chang, J.M., Huang, C.-S., Chen, J.-H. and Chang, R.-F. (2013) Quantitative Ultrasound Analysis for Classification of BI-RADS Category 3 Breast Masses. Journal of Digital Imaging, 26, 1091-1098. http://dx.doi.org/10.1007/s10278-013-9593-8 |
| [7] | Bouzghar, G., Levenback, B.J., Sultan, L.R., Venkatesh, S.S., Cwanger, A., Conant, E.F. and Sehgal, C.M. (2014) Bayesian Probability of Malignancy with Breast Ultrasound BI-RADS Features. Journal of Ultrasound in Medicine, 33, 641-648. http://dx.doi.org/10.7863/ultra.33.4.641 |
| [8] | American College of Radiology (2013) Breast Imaging Reporting and Data System: BI-RADS Atlas. 5th Edition, American College of Radiology, Reston. |
| [9] | Stavros, A.T., Thickman, D., Rapp, C.L., Dennis, M.A., Parker, S.H. and Sisney, G.A. (1995) Solid Breast Nodules: Use of Sonography to Distinguish between Benign and Malignant Lesions. Radiology, 196, 123-134. http://dx.doi.org/10.1148/radiology.196.1.7784555 |
| [10] | Cohen, J. (1960) A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20, 37-46. http://dx.doi.org/10.1177/001316446002000104 |
| [11] | Landis, J.R. and Koch, G.G. (1977) The Measurement of Observer Agreement for Categorical Data. Biometrics, 33, 159-174. http://dx.doi.org/10.2307/2529310 |
| [12] | Cary, T.W., Cwanger, A., Venkatesh, S.S., Conant, E.F. and Sehgal, C.M. (2012) Comparison of Naive Bayes and Logistic Regression for Computer-Aided Diagnosis of Breast Masses Using Ultrasound Imaging. In: Bosch, J.G. and Doyley, M.M., Eds., Medical Imaging: Ultrasonic Imaging, Tomography, and Therapy, SPIE, Bellingham. |
| [13] | DeLong,
E.R., DeLong, D.M. and Clarke-Pearson, D.L. (1988) Comparing the Areas
under Two or More Correlated ROC Curves: A Nonparametric Approach.
Biometrics, 44, 837-845. http://dx.doi.org/10.2307/2531595 |
| [14] | Abdullah, N., Mesurolle, B., El-Khoury, M. and Kao, E. (2009) Breast Imaging Reporting and Data System Lexicon for US: Interobserver Agreement for Assessment of Breast Masses. Radiology, 252, 665-672. |
| [15] | Calas, M.J., Almeida, R.M., Gutfilen, B. and Pereira, W.C. (2009) Intra-Observer Interpretation of Breast Ultrasonography Following the BI-RADS Classification. European Journal of Radiology, 74, 525-528. http://dx.doi.org/10.1016/j.ejrad.2009.04.015 |
| [16] | Park, C.S., Lee, J.H., Yim, H.W., Kang, B.J., Kim, H.S., Jung, J.I., Jung, N.Y. and Kim, S.H. (2007) Observer Agreement Using the ACR Breast Imaging Reporting and Data System (BI-RADS)-Ultrasound. Korean Journal of Radiology, 8, 397-402. |
| [17] | Lee, H.J., Kim, E.K., Kim, M.J., Youk, J.H., Lee, J.Y., Kang, D.R. and Oh, K.K. (2008) Observer Variability of Breast Imaging Reporting and Data System (BI-RADS) for Breast Ultrasound. European Journal of Radiology, 65, 293-298. http://dx.doi.org/10.1016/j.ejrad.2007.04.008 |
| [18] | Ryan, J.T., Haygood, T.M., Yamal, J.M., Evanoff, M., O’Sullivan, P., McEntee, M. and Brennan, P.C. (2011) The “Memory Effect” for Repeated Radiologic Observations. American Journal of Roent- genology, 197, W985-W991. http://dx.doi.org/10.2214/AJR.10.5859 eww150113lx |
评论
发表评论