Applications of discriminative and deep learning feature extraction methods for whole slide image analysis: A survey.

Khaled Al-Thelaya, Nauman Ullah Gilal, Mahmood Alzubaidi, Fahad Majeed, Marco Agus, Jens Schneider, Mowafa Househ
Author Information
  1. Khaled Al-Thelaya: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.
  2. Nauman Ullah Gilal: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.
  3. Mahmood Alzubaidi: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.
  4. Fahad Majeed: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.
  5. Marco Agus: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.
  6. Jens Schneider: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.
  7. Mowafa Househ: Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar.

Abstract

Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by "engineered" methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.

Keywords

References

  1. Am J Pathol. 2021 Aug;191(8):1442-1453 [PMID: 34033750]
  2. Sci Rep. 2018 Feb 1;8(1):2032 [PMID: 29391542]
  3. Cell Oncol (Dordr). 2011 Aug;34(4):343-54 [PMID: 21538025]
  4. CA Cancer J Clin. 2019 Mar;69(2):127-157 [PMID: 30720861]
  5. Med Image Anal. 2015 Feb;20(1):237-48 [PMID: 25547073]
  6. Virchows Arch. 2022 May;480(5):1009-1022 [PMID: 35076741]
  7. J Pathol Inform. 2018 Nov 14;9:38 [PMID: 30607305]
  8. Bioinformatics. 2005 Sep 1;21 Suppl 2:ii7-12 [PMID: 16204128]
  9. J Biomed Opt. 2022 May;27(5): [PMID: 35578386]
  10. Med Image Anal. 2020 Oct;65:101757 [PMID: 32623275]
  11. IEEE Rev Biomed Eng. 2009;2:147-71 [PMID: 20671804]
  12. J Am Soc Nephrol. 2021 Apr;32(4):837-850 [PMID: 33622976]
  13. IEEE J Biomed Health Inform. 2021 May;25(5):1673-1685 [PMID: 32931437]
  14. J Biophotonics. 2021 Jun;14(6):e202000499 [PMID: 33638313]
  15. Sci Rep. 2021 Jun 2;11(1):11579 [PMID: 34078928]
  16. IEEE/ACM Trans Comput Biol Bioinform. 2020 Nov-Dec;17(6):1871-1882 [PMID: 31536012]
  17. J Pathol Inform. 2017 Dec 19;8:51 [PMID: 29416914]
  18. J Pathol Inform. 2021 Nov 03;12:43 [PMID: 34881098]
  19. IEEE Trans Pattern Anal Mach Intell. 2012 Nov;34(11):2274-82 [PMID: 22641706]
  20. Nat Commun. 2020 Aug 3;11(1):3877 [PMID: 32747659]
  21. Front Physiol. 2020 Oct 19;11:583333 [PMID: 33192595]
  22. Diagn Pathol. 2011 Mar 30;6 Suppl 1:S22 [PMID: 21489193]
  23. IEEE Trans Med Imaging. 2018 Sep;37(9):2126-2136 [PMID: 29994086]
  24. Med Image Anal. 2021 Feb;68:101903 [PMID: 33352373]
  25. Diagn Cytopathol. 2009 Oct;37(10):727-31 [PMID: 19373920]
  26. IEEE J Biomed Health Inform. 2017 Jul;21(4):1114-1123 [PMID: 27662689]
  27. J Pathol Inform. 2018 Nov 21;9:40 [PMID: 30607307]
  28. Front Med (Lausanne). 2019 Nov 22;6:264 [PMID: 31824952]
  29. J Biomed Opt. 2019 Dec;24(12):1-15 [PMID: 31837128]
  30. BMC Res Notes. 2019 Feb 12;12(1):82 [PMID: 30755250]
  31. Anal Cell Pathol (Amst). 2017;2017:8428102 [PMID: 28331793]
  32. NPJ Precis Oncol. 2021 Sep 23;5(1):87 [PMID: 34556802]
  33. IEEE Trans Med Imaging. 2020 Nov;39(11):3257-3267 [PMID: 31283474]
  34. IEEE Trans Med Imaging. 2018 Mar;37(3):792-802 [PMID: 29533895]
  35. J Med Imaging (Bellingham). 2018 Oct;5(4):047501 [PMID: 30840742]
  36. Med Image Anal. 2016 Oct;33:170-175 [PMID: 27423409]
  37. IEEE Trans Med Imaging. 2021 Jul;40(7):1924-1933 [PMID: 33780334]
  38. Lancet Oncol. 2019 May;20(5):e253-e261 [PMID: 31044723]
  39. Sci Rep. 2021 Jan 8;11(1):139 [PMID: 33420322]
  40. IEEE Trans Med Imaging. 2007 Oct;26(10):1366-78 [PMID: 17948727]
  41. J Comput Assist Tomogr. 1997 Jul-Aug;21(4):554-66 [PMID: 9216759]
  42. Diagn Pathol. 2011 Mar 30;6 Suppl 1:S15 [PMID: 21489185]
  43. J Med Imaging (Bellingham). 2021 Jan;8(1):014001 [PMID: 33426152]
  44. Cancers (Basel). 2020 Mar 02;12(3): [PMID: 32131409]
  45. AJR Am J Roentgenol. 2013 Mar;200(3):665-72 [PMID: 23436860]
  46. J Am Med Inform Assoc. 2013 Nov-Dec;20(6):1099-108 [PMID: 23959844]
  47. Nat Med. 2019 Aug;25(8):1301-1309 [PMID: 31308507]
  48. Clin Diagn Pathol. 2020 Sep;4(1): [PMID: 33088926]
  49. J Pathol Inform. 2022 Sep 08;13:100138 [PMID: 36268059]
  50. J Digit Imaging. 2016 Aug;29(4):496-506 [PMID: 26961982]
  51. J Digit Imaging. 2020 Aug;33(4):1034-1040 [PMID: 32468487]
  52. IEEE Trans Med Imaging. 2016 Jan;35(1):119-30 [PMID: 26208307]
  53. Nat Commun. 2021 Sep 24;12(1):5639 [PMID: 34561435]
  54. IEEE EMBS Int Conf Biomed Health Inform. 2019 May;2019: [PMID: 32577622]
  55. J Digit Imaging. 2020 Jun;33(3):632-654 [PMID: 31900812]
  56. Comput Methods Programs Biomed. 2021 Jan;198:105807 [PMID: 33130497]
  57. Annu Int Conf IEEE Eng Med Biol Soc. 2011;2011:3427-30 [PMID: 22255076]
  58. Pac Symp Biocomput. 2021;26:285-296 [PMID: 33691025]
  59. J Am Stat Assoc. 2024;119(546):798-810 [PMID: 39280355]
  60. Front Med. 2020 Aug;14(4):470-487 [PMID: 32728875]
  61. Am J Physiol Renal Physiol. 2018 Dec 1;315(6):F1644-F1651 [PMID: 30256126]
  62. J Pathol. 2019 Nov;249(3):286-294 [PMID: 31355445]
  63. Med Image Anal. 2022 Jul;79:102462 [PMID: 35512532]
  64. Neural Comput. 1997 Nov 15;9(8):1735-80 [PMID: 9377276]
  65. Cancers (Basel). 2019 Oct 28;11(11): [PMID: 31661863]
  66. J Pathol. 1992 Apr;166(4):409-11 [PMID: 1517895]
  67. Proc SPIE Int Soc Opt Eng. 2018 Feb;10579: [PMID: 30662142]
  68. Nat Rev Clin Oncol. 2019 Nov;16(11):703-715 [PMID: 31399699]
  69. Biomed Mater Eng. 2015;26 Suppl 1:S1335-44 [PMID: 26405894]
  70. IEEE Trans Med Imaging. 2021 Jun;40(6):1531-1541 [PMID: 33600310]
  71. Comput Methods Programs Biomed. 2022 Jun;221:106914 [PMID: 35640390]
  72. Histopathology. 2012 Jul;61(1):1-9 [PMID: 21477260]
  73. Mod Pathol. 2022 Jan;35(1):23-32 [PMID: 34611303]
  74. Comput Biol Med. 2022 Mar;142:105207 [PMID: 35016101]
  75. Sci Rep. 2018 Feb 21;8(1):3395 [PMID: 29467373]
  76. Med Image Anal. 2021 Jan;67:101813 [PMID: 33049577]
  77. Bioinformatics. 2018 Mar 15;34(6):1024-1030 [PMID: 29136101]
  78. BMC Bioinformatics. 2014 Aug 26;15:287 [PMID: 25155691]
  79. J Am Soc Nephrol. 2018 Aug;29(8):2081-2088 [PMID: 29921718]
  80. Comput Biol Med. 2021 Jan;128:104129 [PMID: 33254082]
  81. Med Image Anal. 2020 Oct;65:101771 [PMID: 32769053]
  82. Breast Cancer Res Treat. 2021 Apr;186(2):379-389 [PMID: 33486639]
  83. IEEE Trans Inf Technol Biomed. 2003 Dec;7(4):249-55 [PMID: 15000351]
  84. BMC Med. 2021 Mar 23;19(1):76 [PMID: 33752648]
  85. Artif Intell Med. 2021 Apr;114:102048 [PMID: 33875159]
  86. J Pathol. 2019 Oct;249(2):143-150 [PMID: 31144302]
  87. Am J Pathol. 2021 Oct;191(10):1717-1723 [PMID: 33838127]
  88. Comput Med Imaging Graph. 2022 Jan;95:102027 [PMID: 34959100]
  89. Eur Urol Focus. 2017 Oct;3(4-5):457-466 [PMID: 28753763]
  90. IEEE Trans Image Process. 2006 Aug;15(8):2259-68 [PMID: 16900681]
  91. Med Image Anal. 2021 Oct;73:102183 [PMID: 34340108]
  92. IEEE Open J Eng Med Biol. 2022 Jul 19;3:115-123 [PMID: 35937101]
  93. BMC Bioinformatics. 2017 May 26;18(1):281 [PMID: 28549410]
  94. Comput Methods Programs Biomed. 2020 Feb;184:105273 [PMID: 31891905]
  95. Radiology. 2019 Jun;291(3):677-686 [PMID: 30912722]
  96. Comput Med Imaging Graph. 2018 Dec;70:43-52 [PMID: 30286333]
  97. Comput Methods Programs Biomed. 2023 Feb;229:107268 [PMID: 36495811]
  98. Sci Rep. 2018 Jul 10;8(1):10393 [PMID: 29991684]
  99. Annu Int Conf IEEE Eng Med Biol Soc. 2009;2009:3719-22 [PMID: 19965011]
  100. Comput Med Imaging Graph. 2018 Sep;68:40-54 [PMID: 29890404]
  101. IEEE Trans Vis Comput Graph. 2021 Feb;27(2):645-655 [PMID: 33055035]
  102. Med Image Anal. 2023 Apr;85:102755 [PMID: 36724605]
  103. Sci Rep. 2020 Jan 30;10(1):1504 [PMID: 32001752]
  104. BMC Bioinformatics. 2018 May 16;19(1):173 [PMID: 29769044]
  105. Pathobiology. 2016;83(2-3):127-39 [PMID: 27100217]
  106. IEEE J Biomed Health Inform. 2022 Jul;26(7):3163-3173 [PMID: 35196251]
  107. J Pathol Inform. 2021 Sep 16;12:34 [PMID: 34760331]
  108. IEEE Trans Med Imaging. 2016 Jan;35(1):307-15 [PMID: 26302511]
  109. IEEE Trans Med Imaging. 2016 Sep;35(9):2141-2150 [PMID: 27076354]
  110. Sci Rep. 2018 Aug 13;8(1):12054 [PMID: 30104757]
  111. IEEE Trans Vis Comput Graph. 2018 Aug 20;: [PMID: 30130214]
  112. Front Oncol. 2018 Apr 04;8:96 [PMID: 29670857]
  113. J Pathol Inform. 2023 Jan 02;14:100185 [PMID: 36691660]
  114. Comput Methods Programs Biomed. 2014 Feb;113(2):539-56 [PMID: 24433758]
  115. Comput Struct Biotechnol J. 2020 Dec 03;18:4063-4070 [PMID: 33363702]
  116. Med Image Anal. 2021 Feb;68:101938 [PMID: 33359932]

Word Cloud

Created with Highcharts 10.0.0pathologymethodsdigitalapplicationsextractionfeaturesslideWSIAIimageanalysisfeaturesurveydeeplearningDigitalincludingwholefacilitatingtissuediagnosticprognosisdiagnosisDeeplearning-basedtechniquesWSIsdiscriminativetechnologiesimagingsignificantlyimprovedmodernclinicalpracticesstoringviewingprocessingsharingscansglassslidesResearchersproposedvariousartificialintelligencesolutionsautomatedextractinformationimprovingproductivityaccuracyreproducibilityFeatureplaycrucialroletransformingrawdatameaningfulrepresentationscharacterizationstructurescellularpropertiespathologicalpatternsdiverseseveralcanceremergedpromisingapproachaccuratelyrepresentcontentsdemonstratedsuperiorperformancehistology-relatedtasksprovidecomprehensiveoverviewmanualreviewrelevantliteratureanalyzegeometriciesuitedsupportprocessextracted"engineered"opposedexplorepredictivemodelingusingexaminesadvanceschallengesopportunitiesrapidlyevolvingfieldemphasizingpotentialaccuratedecision-makingApplicationsanalysis:ImageengineeringWholeimages

Similar Articles

Cited By