OpenLB
Open Library of Bioscience
Advanced Search
European radiology
Jul, 2021;31 (7): 4872-4885.

Deep learning in breast radiology: current progress and future directions.

William C Ou, Dogan Polat, Basak E Dogan
Author Information
  1. William C Ou: Department of Radiology, Seay Biomedical Building, University of Texas Southwestern Medical Center, 2201 Inwood Road, Dallas, TX, 75390, USA. William.Ou@UTSouthwestern.edu.
  2. Dogan Polat: Department of Radiology, Seay Biomedical Building, University of Texas Southwestern Medical Center, 2201 Inwood Road, Dallas, TX, 75390, USA.
  3. Basak E Dogan: Department of Radiology, Seay Biomedical Building, University of Texas Southwestern Medical Center, 2201 Inwood Road, Dallas, TX, 75390, USA.
PMID: 33449174 DOI: 10.1007/s00330-020-07640-9

Abstract

This review provides an overview of current applications of deep learning methods within breast radiology. The diagnostic capabilities of deep learning in breast radiology continue to improve, giving rise to the prospect that these methods may be integrated not only into detection and classification of breast lesions, but also into areas such as risk estimation and prediction of tumor responses to therapy. Remaining challenges include limited availability of high-quality data with expert annotations and ground truth determinations, the need for further validation of initial results, and unresolved medicolegal considerations. KEY POINTS: • Deep learning (DL) continues to push the boundaries of what can be accomplished by artificial intelligence (AI) in breast imaging with distinct advantages over conventional computer-aided detection. • DL-based AI has the potential to augment the capabilities of breast radiologists by improving diagnostic accuracy, increasing efficiency, and supporting clinical decision-making through prediction of prognosis and therapeutic response. • Remaining challenges to DL implementation include a paucity of prospective data on DL utilization and yet unresolved medicolegal questions regarding increasing AI utilization.

Keywords

Artificial intelligence Breast magnetic resonance imaging Breast ultrasonography Deep learning Digital mammography

References

  1. Hamet P, Tremblay J (2017) Artificial intelligence in medicine. Metab Clin Exp 69S:S36–S40 [PMID: 28126242]
  2. Bluemke DA, Moy L, Bredella MA et al (2020) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers-from the radiology editorial board. Radiology 294:487–489 [PMID: 31891322]
  3. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444 [DOI: 10.1038/nature14539]
  4. Keen JD, Keen JM, Keen JE (2018) Utilization of computer-aided detection for digital screening mammography in the United States, 2008 to 2016. J Am Coll Radiol 15:44–48 [PMID: 28993109]
  5. Elmore JG, Carney PA (2004) Computer-aided detection of breast cancer: has promise outstripped performance? J Natl Cancer Inst 96:162–163 [PMID: 14759974]
  6. Philpotts LE (2009) Can computer-aided detection be detrimental to mammographic interpretation? Radiology 253:17–22 [PMID: 19789251]
  7. Kohli A, Jha S (2018) Why CAD failed in mammography. J Am Coll Radiol 15:535–537 [PMID: 29398499]
  8. D’Orsi CJ, Sickles EA, Mendelson EB et al (2013) ACR BI-RADS® Atlas, breast imaging reporting and data system. American College of Radiology, Reston
  9. Deo RC (2015) Machine learning in medicine. Circulation 132:1920–1930 [PMID: 26572668]
  10. Cheng B, Titterington DM (1994) Neural networks: a review from a statistical perspective. Stat Sci 9:2–30
  11. Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29:2352–2449 [PMID: 28599112]
  12. Zhang W, Doi K, Giger ML, Wu Y, Nishikawa RM, Schmidt RA (1994) Computerized detection of clustered microcalcifications in digital mammograms using a shift-invariant artificial neural network. Med Phys 21:517–524 [PMID: 8058017]
  13. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Comput Intell Neurosci 2018:7068349 [PMID: 29487619]
  14. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol 25, pp 1097–1105
  15. Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554 [PMID: 16764513]
  16. Salakhutdinov R, Hinton G (2009) Deep Boltzmann Machines. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp 448–455
  17. Russakovsky O, Deng J, Su H et al (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252
  18. Katzen J, Dodelzon K (2018) A review of computer aided detection in mammography. Clin Imaging 52:305–309 [PMID: 30216858]
  19. Henriksen EL, Carlsen JF, Vejborg IM, Nielsen MB, Lauridsen CA (2019) The efficacy of using computer-aided detection (CAD) for detection of breast cancer in mammography screening: a systematic review. Acta Radiol 60:13–18 [PMID: 29665706]
  20. Fenton JJ, Taplin SH, Carney PA et al (2007) Influence of computer-aided detection on performance of screening mammography. N Engl J Med 356:1399–1409 [PMID: 17409321]
  21. Fenton JJ, Abraham L, Taplin SH et al (2011) Effectiveness of computer-aided detection in community mammography practice. J Natl Cancer Inst 103:1152–1161 [PMID: 21795668]
  22. Lehman CD, Wellman RD, Buist DSM et al (2015) Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern Med 175:1828–1837 [PMID: 26414882]
  23. Akselrod-Ballin A, Karlinsky L, Alpert S, Hasoul S, Ben-Ari R, Barkan E (2016) A region based convolutional network for tumor detection and classification in breast mammography. In: Carneiro G et al (eds) Deep learning and data labeling for medical applications. Springer International Publishing, Cham, pp 197–205
  24. Arevalo J, González FA, Ramos-Pollán R, Oliveira JL, Guevara Lopez MA (2016) Representation learning for mammography mass lesion classification with convolutional neural networks. Comput Methods Programs Biomed 127:248–257 [PMID: 26826901]
  25. Kooi T, Litjens G, van Ginneken B et al (2017) Large scale deep learning for computer aided detection of mammographic lesions. Med Image Anal 35:303–312
  26. Sun W, Tseng T-LB, Zhang J, Qian W (2017) Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data. Comput Med Imaging Graph 57:4–9 [PMID: 27475279]
  27. Teare P, Fishman M, Benzaquen O, Toledano E, Elnekave E (2017) Malignancy detection on mammography using dual deep convolutional neural networks and genetically discovered false color input enhancement. J Digit Imaging 30:499–505 [PMID: 28656455]
  28. Kim E-K, Kim H-E, Han K et al (2018) Applying data-driven imaging biomarker in mammography for breast cancer screening: preliminary study. Sci Rep 8:2762 [PMID: 29426948]
  29. Chougrad H, Zouaki H, Alheyane O (2018) Deep convolutional neural networks for breast cancer screening. Comput Methods Programs Biomed 157:19–30 [PMID: 29477427]
  30. Rodríguez-Ruiz A, Krupinski E, Mordang J-J et al (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290:305–314 [PMID: 30457482]
  31. Rodriguez-Ruiz A, Lång K, Gubern-Merida A et al (2019) Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J Natl Cancer Inst 111:916–922 [PMID: 30834436]
  32. Shen L, Margolies LR, Rothstein JH, Fluder E, McBride R, Sieh W (2019) Deep learning to improve breast cancer detection on screening mammography. Sci Rep 9:12495 [PMID: 31467326]
  33. Wu N, Phang J, Park J et al (2020) Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Trans Med Imaging 39:1184–1194 [PMID: 31603772]
  34. McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94 [PMID: 31894144]
  35. Al-Antari MA, Al-Masni MA, Kim T-S (2020) Deep learning computer-aided diagnosis for breast lesion in digital mammogram. Adv Exp Med Biol 1213:59–72 [PMID: 32030663]
  36. Kim H-E, Kim HH, Han B-K et al (2020) Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digital Health 2:e138–e148 [PMID: 33334578]
  37. Schaffter T, Buist DSM, Lee CI et al (2020) Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open 3:e200265 [PMID: 32119094]
  38. Wang X, Liang G, Zhang Y, Blanton H, Bessinger Z, Jacobs N (2020) Inconsistent performance of deep learning models on mammogram classification. J Am Coll Radiol 17:796–803 [PMID: 32068005]
  39. Sharpe RE, Venkataraman S, Phillips J et al (2016) Increased cancer detection rate and variations in the recall rate resulting from implementation of 3D digital breast tomosynthesis into a population-based screening program. Radiology 278:698–706 [PMID: 26458206]
  40. Bernardi D, Macaskill P, Pellegrini M et al (2016) Breast cancer screening with tomosynthesis (3D mammography) with acquired or synthetic 2D mammography compared with 2D mammography alone (STORM-2): a population-based prospective study. Lancet Oncol 17:1105–1113 [PMID: 27345635]
  41. Pattacini P, Nitrosi A, Giorgi Rossi P et al (2018) Digital mammography versus digital mammography plus tomosynthesis for breast cancer screening: the Reggio Emilia tomosynthesis randomized trial. Radiology 288:375–385 [PMID: 29869961]
  42. Dang PA, Freer PE, Humphrey KL, Halpern EF, Rafferty EA (2014) Addition of tomosynthesis to conventional digital mammography: effect on image interpretation time of screening examinations. Radiology 270:49–56 [PMID: 24354377]
  43. Balleyguier C, Arfi-Rouche J, Levy L et al (2017) Improving digital breast tomosynthesis reading time: a pilot multi-reader, multi-case study using concurrent computer-aided detection (CAD). Eur J Radiol 97:83–89 [PMID: 29153373]
  44. Benedikt RA, Boatsman JE, Swann CA, Kirkpatrick AD, Toledano AY (2018) Concurrent computer-aided detection improves reading time of digital breast tomosynthesis and maintains interpretation performance in a multireader multicase study. AJR Am J Roentgenol 210:685–694 [PMID: 29064756]
  45. Zhang X, Zhang Y, Han EY et al (2018) Classification of whole mammogram and tomosynthesis images using deep convolutional neural networks. IEEE Trans Nanobioscience 17:237–242 [PMID: 29994219]
  46. Samala RK, Chan H-P, Hadjiiski L, Helvie MA, Richter CD, Cha KH (2019) Breast cancer diagnosis in digital breast tomosynthesis: effects of training sample size on multi-stage transfer learning using deep neural nets. IEEE Trans Med Imaging 38:686–696 [PMID: 31622238]
  47. Conant EF, Toledano AY, Periaswamy S et al (2019) Improving accuracy and efficiency with concurrent use of artificial intelligence for digital breast tomosynthesis. Radiol Artif Intell 1:e180096 [PMID: 32076660]
  48. Li X, Qin G, He Q et al (2020) Digital breast tomosynthesis versus digital mammography: integration of image modalities enhances deep learning-based breast mass classification. Eur Radiol 30:778–788 [PMID: 31691121]
  49. Chae EY, Kim HH, Jeong J-W, Chae S-H, Lee S, Choi Y-W (2019) Decrease in interpretation time for both novice and experienced readers using a concurrent computer-aided detection system for digital breast tomosynthesis. Eur Radiol 29:2518–2525 [PMID: 30547203]
  50. Durfee SM, Selland D-LG, Smith DN, Lester SC, Kaelin CM, Meyer JE (2000) Sonographic evaluation of clinically palpable breast cancers invisible on mammography. Breast J 6:247–251 [PMID: 11348373]
  51. Cheng J-Z, Ni D, Chou Y-H et al (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6:24454 [PMID: 27079888]
  52. Shi J, Zhou S, Liu X, Zhang Q, Lu M, Wang T (2016) Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset. Neurocomputing 194:87–94
  53. Han S, Kang H-K, Jeong J-Y et al (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714–7728 [PMID: 28753132]
  54. Becker AS, Mueller M, Stoffel E, Marcon M, Ghafoor S, Boss A (2018) Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study. Br J Radiol 91:20170576 [PMID: 29215311]
  55. Xiao M, Zhao C, Zhu Q et al (2019) An investigation of the classification accuracy of a deep learning framework-based computer-aided diagnosis system in different pathological types of breast lesions. J Thorac Dis 11:5023–5031 [PMID: 32030218]
  56. Cao Z, Duan L, Yang G, Yue T, Chen Q (2019) An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Med Imaging 19:51 [PMID: 31262255]
  57. Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur Radiol 29:5458–5468 [PMID: 30927100]
  58. Fujioka T, Kubota K, Mori M et al (2019) Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network. Jpn J Radiol 37:466–472 [PMID: 30888570]
  59. Huang Y, Han L, Dou H et al (2019) Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images. Biomed Eng Online 18:8 [PMID: 30678680]
  60. Coronado-Gutiérrez D, Santamaría G, Ganau S et al (2019) Quantitative ultrasound image analysis of axillary lymph nodes to diagnose metastatic involvement in breast cancer. Ultrasound Med Biol 45:2932–2941 [PMID: 31444031]
  61. Stoffel E, Becker AS, Wurnig MC et al (2018) Distinction between phyllodes tumor and fibroadenoma in breast ultrasound using deep learning image analysis. Eur J Radiol Open 5:165–170 [PMID: 30258856]
  62. Mann RM, Cho N, Moy L (2019) Breast MRI: state of the art. Radiology 292:520–536 [PMID: 31361209]
  63. Antropova N, Huynh BQ, Giger ML (2017) A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys 44:5162–5171 [PMID: 28681390]
  64. Rasti R, Teshnehlab M, Phung SL (2017) Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. Pattern Recogn 72:381–390
  65. Dalmiş MU, Gubern-Mérida A, Vreemann S et al (2019) Artificial intelligence-based classification of breast lesions imaged with a multiparametric breast MRI protocol with ultrafast DCE-MRI, T2, and DWI. Invest Radiol 54:325–332 [PMID: 30652985]
  66. Truhn D, Schrading S, Haarburger C, Schneider H, Merhof D, Kuhl C (2019) Radiomic versus convolutional neural networks analysis for classification of contrast-enhancing lesions at multiparametric breast MRI. Radiology 290:290–297 [PMID: 30422086]
  67. Zhou J, Luo L-Y, Dou Q et al (2019) Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images. J Magn Reson Imaging 50:1144–1151 [PMID: 30924997]
  68. Feng H, Cao J, Wang H et al (2020) A knowledge-driven feature learning and integration method for breast cancer diagnosis on multi-sequence MRI. Magn Reson Imaging 69:40–48 [PMID: 32173583]
  69. Zhou J, Zhang Y, Chang K-T et al (2020) Diagnosis of benign and malignant breast lesions on DCE-MRI by using radiomics and deep learning with consideration of peritumor tissue. J Magn Reson Imaging 51:798–809 [PMID: 31675151]
  70. Parekh VS, Macura KJ, Harvey SC et al (2020) Multiparametric deep learning tissue signatures for a radiological biomarker of breast cancer: Preliminary results. Med Phys 47:75–88 [PMID: 31598978]
  71. Ren T, Cattell R, Duanmu H et al (2020) Convolutional neural network detection of axillary lymph node metastasis using standard clinical breast MRI. Clin Breast Cancer 20:e301–e308 [PMID: 32139272]
  72. Zhang H-M, Dong B (2020) A review on deep learning in medical image reconstruction. J Oper Res Soc China 8:311–340
  73. Haskins G, Kruger U, Yan P (2020) Deep learning in medical image registration: a survey. Mach Vis Appl 31:8
  74. Ha R, Chang P, Karcich J et al (2019) Convolutional neural network based breast cancer risk stratification using a mammographic dataset. Acad Radiol 26:544–549 [PMID: 30072292]
  75. Dembrower K, Liu Y, Azizpour H et al (2020) Comparison of a deep learning risk score and standard mammographic density score for breast cancer risk prediction. Radiology 294:265–272 [PMID: 31845842]
  76. Yala A, Lehman C, Schuster T, Portnoi T, Barzilay R (2019) A deep learning mammography-based model for improved breast cancer risk prediction. Radiology 292:60–66 [PMID: 31063083]
  77. Shi B, Grimm LJ, Mazurowski MA et al (2018) Prediction of occult invasive disease in ductal carcinoma in situ using deep learning features. J Am Coll Radiol 15:527–534 [PMID: 29398498]
  78. Mutasa S, Chang P, Van Sant EP et al (2019) Potential role of convolutional neural network based algorithm in patient selection for DCIS observation trials using a mammogram dataset. Acad Radiol 27:774–779 [PMID: 31526687]
  79. Zhu Z, Harowicz M, Zhang J et al (2019) Deep learning analysis of breast MRIs for prediction of occult invasive disease in ductal carcinoma in situ. Comput Biol Med 115:103498 [PMID: 31698241]
  80. Ravichandran K, Braman N, Janowczyk A, Madabhushi A (2018) A deep learning classifier for prediction of pathological complete response to neoadjuvant chemotherapy from baseline breast DCE-MRI. In: Proc. SPIE 10575, Medical Imaging 2018: Computer-Aided Diagnosis. pp 105750C
  81. Ha R, Chin C, Karcich J et al (2019) Prior to initiation of chemotherapy, can we predict breast tumor response? Deep learning convolutional neural networks approach using a breast MRI tumor dataset. J Digit Imaging 32:693–701 [PMID: 30361936]
  82. Qu Y-H, Zhu H-T, Cao K, Li X-T, Ye M, Sun Y-S (2020) Prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer using a deep learning (DL) method. Thorac Cancer 11:651–658 [PMID: 31944571]
  83. Sun Q, Lin X, Zhao Y et al (2020) Deep learning vs. radiomics for predicting axillary lymph node metastasis of breast cancer using ultrasound images: don’t forget the peritumoral region. Front Oncol 10:53 [PMID: 32083007]
  84. Zhou L-Q, Wu X-L, Huang S-Y et al (2020) Lymph node metastasis prediction from primary breast cancer US images using deep learning. Radiology 294:19–28 [PMID: 31746687]
  85. Mayo RC, Kent D, Sen LC, Kapoor M, Leung JWT, Watanabe AT (2019) Reduction of false-positive markings on mammograms: a retrospective comparison study using an artificial intelligence-based CAD. J Digit Imaging 32:618–624 [PMID: 30963339]
  86. Yala A, Schuster T, Miles R, Barzilay R, Lehman C (2019) A deep learning model to triage screening mammograms: a simulation study. Radiology 293:38–46 [PMID: 31385754]
  87. Kyono T, Gilbert FJ, van der Schaar M (2018) MAMMO: a deep learning solution for facilitating radiologist-machine collaboration in breast cancer diagnosis. Available via https://arxiv.org/abs/1811.02661 . Accessed 12 May 2020
  88. Marcus G (2018) Deep learning: a critical appraisal. Available via https://arxiv.org/abs/1801.00631 . Accessed 19 May 2020
  89. Ren K, Zheng T, Qin Z, Liu X (2020) Adversarial attacks and defenses in deep learning. Engineering 6:346–360
  90. Nagendran M, Chen Y, Lovejoy CA et al (2020) Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 368:m689 [PMID: 32213531]
  91. Gur D, Bandos AI, Cohen CS et al (2008) The “laboratory” effect: comparing radiologists’ performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 249:47–53 [PMID: 18682584]
  92. Bluemke DA (2018) Editor’s note: publication of AI research in radiology. Radiology 289:579–580 [PMID: 30398443]
  93. Goddard K, Roudsari A, Wyatt JC (2012) Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc 19:121–127 [PMID: 21685142]
  94. Alberdi E, Povykalo A, Strigini L, Ayton P (2004) Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography. Acad Radiol 11:909–918 [PMID: 15354301]
  95. Geis JR, Brady AP, Wu CC et al (2019) Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement. Radiology 293:436–440 [PMID: 31573399]
  96. Jonmarker O, Strand F, Brandberg Y, Lindholm P (2019) The future of breast cancer screening: what do participants in a breast cancer screening program think about automation using artificial intelligence? Acta Radiol Open 8:2058460119880315 [PMID: 31839989]

MeSH Term

Artificial Intelligence
Breast
Deep Learning
Humans
Prospective Studies
Radiology
Journal Article Review
Article has an altmetric score of 1

Word Cloud

Created with Highcharts 10.0.0breastlearning•DeepDLAIcurrentdeepmethodsradiologydiagnosticcapabilitiesdetectionpredictionRemainingchallengesincludedataunresolvedmedicolegalintelligenceimagingincreasingutilizationBreastreviewprovidesoverviewapplicationswithincontinueimprovegivingriseprospectmayintegratedclassificationlesionsalsoareasriskestimationtumorresponsestherapylimitedavailabilityhigh-qualityexpertannotationsgroundtruthdeterminationsneedvalidationinitialresultsconsiderationsKEYPOINTS:continuespushboundariescanaccomplishedartificialdistinctadvantagesconventionalcomputer-aidedDL-basedpotentialaugmentradiologistsimprovingaccuracyefficiencysupportingclinicaldecision-makingprognosistherapeuticresponseimplementationpaucityprospectiveyetquestionsregardingradiology:progressfuturedirectionsArtificialmagneticresonanceultrasonographyDigitalmammography

Similar Articles

Cited By