OpenLB
Open Library of Bioscience
Advanced Search
Heliyon
Jul, 2023;9 (7): e17934.

Machine learning with multimodal data for COVID-19.

Weijie Chen, Rui C Sá, Yuntong Bai, Sandy Napel, Olivier Gevaert, Diane S Lauderdale, Maryellen L Giger
Author Information
  1. Weijie Chen: Medical Imaging and Data Resource Center (MIDRC), USA.
  2. Rui C Sá: Medical Imaging and Data Resource Center (MIDRC), USA.
  3. Yuntong Bai: Medical Imaging and Data Resource Center (MIDRC), USA.
  4. Sandy Napel: Medical Imaging and Data Resource Center (MIDRC), USA.
  5. Olivier Gevaert: Medical Imaging and Data Resource Center (MIDRC), USA.
  6. Diane S Lauderdale: Medical Imaging and Data Resource Center (MIDRC), USA.
  7. Maryellen L Giger: Medical Imaging and Data Resource Center (MIDRC), USA.
PMID: 37483733 DOI: 10.1016/j.heliyon.2023.e17934

Abstract

In response to the unprecedented global healthcare crisis of the COVID-19 pandemic, the scientific community has joined forces to tackle the challenges and prepare for future pandemics. Multiple modalities of data have been investigated to understand the nature of COVID-19. In this paper, MIDRC investigators present an overview of the state-of-the-art development of multimodal machine learning for COVID-19 and model assessment considerations for future studies. We begin with a discussion of the lessons learned from radiogenomic studies for cancer diagnosis. We then summarize the multi-modality COVID-19 data investigated in the literature including symptoms and other clinical data, laboratory tests, imaging, pathology, physiology, and other omics data. Publicly available multimodal COVID-19 data provided by MIDRC and other sources are summarized. After an overview of machine learning developments using multimodal data for COVID-19, we present our perspectives on the future development of multimodal machine learning models for COVID-19.

Keywords

COVID-19 Machine learning Multimodal data

References

  1. Ann Intern Med. 2019 Jan 1;170(1):51-58 [PMID: 30596875]
  2. Sci Rep. 2015 Dec 07;5:17787 [PMID: 26639025]
  3. Front Cell Infect Microbiol. 2021 Sep 14;11:744903 [PMID: 34595136]
  4. Int J Oncol. 2020 Jul;57(1):43-53 [PMID: 32467997]
  5. Cell. 2020 Dec 10;183(6):1479-1495.e20 [PMID: 33171100]
  6. Nat Commun. 2014 Jun 03;5:4006 [PMID: 24892406]
  7. Eur Radiol. 2020 Dec;30(12):6797-6807 [PMID: 32607634]
  8. J Pathol. 2020 Jul;251(3):228-248 [PMID: 32418199]
  9. J Med Imaging (Bellingham). 2015 Oct;2(4):041007 [PMID: 26835491]
  10. J Med Imaging (Bellingham). 2023 Nov;10(6):061104 [PMID: 37125409]
  11. Radiology. 2015 May;275(2):384-92 [PMID: 25734557]
  12. Nat Rev Cancer. 2022 Feb;22(2):114-126 [PMID: 34663944]
  13. Sci Rep. 2017 Jan 31;7:41674 [PMID: 28139704]
  14. Radiology. 2014 Oct;273(1):168-74 [PMID: 24827998]
  15. J Gastrointest Cancer. 2020 Dec;51(4):1165-1168 [PMID: 32844349]
  16. J Appl Physiol (1985). 2021 Mar 1;130(3):865-876 [PMID: 33439790]
  17. J Magn Reson Imaging. 2016 Jun;43(6):1269-78 [PMID: 26663695]
  18. J Vasc Interv Radiol. 2007 Jul;18(7):821-31 [PMID: 17609439]
  19. J Magn Reson Imaging. 2018 Mar;47(3):604-620 [PMID: 29095543]
  20. Radiol Imaging Cancer. 2020 May 15;2(3):e190039 [PMID: 32550599]
  21. Semin Nucl Med. 2021 Jul;51(4):312-320 [PMID: 33288215]
  22. J Physiol. 2021 Nov;599(22):4991-5013 [PMID: 34510457]
  23. Am J Crit Care. 2022 Mar 1;31(2):146-157 [PMID: 34709373]
  24. Cancer Cell. 2022 Aug 8;40(8):865-878.e6 [PMID: 35944502]
  25. NPJ Digit Med. 2021 Jun 3;4(1):94 [PMID: 34083734]
  26. Radiology. 2018 Jan;286(1):307-315 [PMID: 28727543]
  27. Eur Respir J. 2019 Mar 28;53(3): [PMID: 30635290]
  28. Postgrad Med J. 2021 Jan;97(1143):34-39 [PMID: 32895294]
  29. NPJ Digit Med. 2020 Oct 16;3:136 [PMID: 33083571]
  30. Korean J Radiol. 2020 Jul;21(7):859-868 [PMID: 32524786]
  31. Acad Radiol. 2021 Nov;28(11):1507-1523 [PMID: 34649779]
  32. Semin Radiat Oncol. 2010 Jul;20(3):149-55 [PMID: 20685577]
  33. J Med Imaging (Bellingham). 2015 Oct;2(4):041001 [PMID: 26839908]
  34. NPJ Breast Cancer. 2016;2: [PMID: 27853751]
  35. Nature. 2022 Apr;604(7907):697-707 [PMID: 35255491]
  36. Radiology. 2022 Dec;305(3):709-717 [PMID: 35608443]
  37. Br J Radiol. 2016;89(1061):20151030 [PMID: 26864054]
  38. Oncotarget. 2017 May 10;8(32):52792-52801 [PMID: 28881771]
  39. BMJ. 2020 Apr 7;369:m1328 [PMID: 32265220]
  40. Health Technol (Berl). 2021;11(6):1331-1345 [PMID: 34660166]
  41. AJR Am J Roentgenol. 2012 Sep;199(3):654-63 [PMID: 22915408]
  42. PLoS One. 2011;6(10):e25451 [PMID: 21998659]
  43. Radiol Artif Intell. 2020 Mar 25;2(2):e200029 [PMID: 33937821]
  44. Int J Mol Sci. 2020 Oct 29;21(21): [PMID: 33138181]
  45. Radiology. 2021 Jun;299(3):E262-E279 [PMID: 33560192]
  46. EBioMedicine. 2019 Jul;45:70-80 [PMID: 31255659]
  47. NPJ Digit Med. 2022 Aug 26;5(1):126 [PMID: 36028526]
  48. Cancer. 2018 Dec 15;124(24):4633-4649 [PMID: 30383900]
  49. Nat Rev Nephrol. 2021 Dec;17(12):792-793 [PMID: 34504319]
  50. Bioinformatics. 2019 Jul 15;35(14):i446-i454 [PMID: 31510656]
  51. Radiology. 2016 May;279(2):432-42 [PMID: 26653683]
  52. Cancer. 2016 Mar 1;122(5):748-57 [PMID: 26619259]
  53. IEEE Trans Pattern Anal Mach Intell. 2019 Feb;41(2):423-443 [PMID: 29994351]
  54. Radiology. 2016 Nov;281(2):382-391 [PMID: 27144536]
  55. Nat Med. 2022 Aug;28(8):1706-1714 [PMID: 35879616]
  56. Cancer Res. 2012 Aug 1;72(15):3725-34 [PMID: 22710433]
  57. AJR Am J Roentgenol. 2020 Sep;215(3):603-606 [PMID: 32319792]
  58. Front Digit Health. 2022 Oct 06;4:1007784 [PMID: 36274654]
  59. Int J Impot Res. 2022 Mar;34(2):152-157 [PMID: 35152276]
  60. Med Phys. 2021 Sep;48(9):4711-4714 [PMID: 34545957]
  61. Invest Radiol. 2020 Jun;55(6):332-339 [PMID: 32134800]
  62. IEEE Trans Biomed Eng. 2023 Mar;70(3):909-919 [PMID: 36094967]
  63. Sci Rep. 2022 Mar 2;12(1):3463 [PMID: 35236896]
  64. Nat Biomed Eng. 2020 Dec;4(12):1197-1207 [PMID: 33208927]
  65. Nat Med. 2021 Apr;27(4):601-615 [PMID: 33753937]
  66. Radiology. 2012 Aug;264(2):387-96 [PMID: 22723499]
  67. Transl Oncol. 2014 Oct 24;7(5):556-69 [PMID: 25389451]
Journal Article Review
Article has an altmetric score of 1

Word Cloud

Created with Highcharts 10.0.0COVID-19datamultimodallearningfuturemachineinvestigatedMIDRCpresentoverviewdevelopmentstudiesMachineresponseunprecedentedglobalhealthcarecrisispandemicscientificcommunityjoinedforcestacklechallengespreparepandemicsMultiplemodalitiesunderstandnaturepaperinvestigatorsstate-of-the-artmodelassessmentconsiderationsbegindiscussionlessonslearnedradiogenomiccancerdiagnosissummarizemulti-modalityliteratureincludingsymptomsclinicallaboratorytestsimagingpathologyphysiologyomicsPubliclyavailableprovidedsourcessummarizeddevelopmentsusingperspectivesmodelsMultimodal

Similar Articles

Cited By