An attention-based context-informed deep framework for infant brain subcortical segmentation.

Liangjun Chen, Zhengwang Wu, Fenqiang Zhao, Ya Wang, Weili Lin, Li Wang, Gang Li
Author Information
  1. Liangjun Chen: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
  2. Zhengwang Wu: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
  3. Fenqiang Zhao: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
  4. Ya Wang: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
  5. Weili Lin: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
  6. Li Wang: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
  7. Gang Li: Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA. Electronic address: gang_li@med.unc.edu.

Abstract

Precise segmentation of subcortical structures from infant brain magnetic resonance (MR) images plays an essential role in studying early subcortical structural and functional developmental patterns and diagnosis of related brain disorders. However, due to the dynamic appearance changes, low tissue contrast, and tiny subcortical size in infant brain MR images, infant subcortical segmentation is a challenging task. In this paper, we propose a context-guided, attention-based, coarse-to-fine deep framework to precisely segment the infant subcortical structures. At the coarse stage, we aim to directly predict the signed distance maps (SDMs) from multi-modal intensity images, including T1w, T2w, and the ratio of T1w and T2w images, with an SDM-Unet, which can leverage the spatial context information, including the structural position information and the shape information of the target structure, to generate high-quality SDMs. At the fine stage, the predicted SDMs, which encode spatial-context information of each subcortical structure, are integrated with the multi-modal intensity images as the input to a multi-source and multi-path attention Unet (M2A-Unet) for achieving refined segmentation. Both the 3D spatial and channel attention blocks are added to guide the M2A-Unet to focus more on the important subregions and channels. We additionally incorporate the inner and outer subcortical boundaries as extra labels to help precisely estimate the ambiguous boundaries. We validate our method on an infant MR image dataset and on an unrelated neonatal MR image dataset. Compared to eleven state-of-the-art methods, the proposed framework consistently achieves higher segmentation accuracy in both qualitative and quantitative evaluations of infant MR images and also exhibits good generalizability in the neonatal dataset.

Keywords

References

  1. IEEE Trans Vis Comput Graph. 2006 Jul-Aug;12(4):581-99 [PMID: 16805266]
  2. Mov Disord. 2015 Aug;30(9):1155-70 [PMID: 25772380]
  3. IEEE Trans Med Imaging. 1998 Feb;17(1):87-97 [PMID: 9617910]
  4. Med Image Comput Comput Assist Interv. 2018 Sep;11072:411-419 [PMID: 30430147]
  5. IEEE Trans Image Process. 2011 Jul;20(7):2007-16 [PMID: 21518662]
  6. Graph Learn Med Imaging (2019). 2019 Oct;11849:164-171 [PMID: 32104792]
  7. Med Image Comput Comput Assist Interv. 2020 Oct;12267:646-656 [PMID: 33564753]
  8. Neuroimage. 2020 Sep;218:116946 [PMID: 32442637]
  9. Philos Trans R Soc Lond B Biol Sci. 2009 May 12;364(1521):1223-34 [PMID: 19528003]
  10. Neuroimage. 2013 Jan 15;65:315-23 [PMID: 23000785]
  11. Neurology. 2001 Jul 24;57(2):245-54 [PMID: 11468308]
  12. Lancet Neurol. 2015 Nov;14(11):1121-34 [PMID: 25891007]
  13. Nat Neurosci. 2014 Aug;17(8):1022-30 [PMID: 25065439]
  14. IEEE Trans Med Imaging. 2010 Oct;29(10):1714-29 [PMID: 20562040]
  15. IEEE Trans Image Process. 2010 Dec;19(12):3243-54 [PMID: 20801742]
  16. Annu Int Conf IEEE Eng Med Biol Soc. 2016 Aug;2016:3342-3345 [PMID: 28269019]
  17. Med Image Anal. 2013 Oct;17(7):766-78 [PMID: 23773521]
  18. Neuroimage. 2022 Jun;253:119097 [PMID: 35301130]
  19. IEEE Trans Pattern Anal Mach Intell. 2017 Jun;39(6):1137-1149 [PMID: 27295650]
  20. Comput Biol Med. 2021 Dec 4;140:105113 [PMID: 34891094]
  21. IEEE Trans Med Imaging. 2019 Feb 27;: [PMID: 30835215]
  22. J Neurosci. 2011 Aug 10;31(32):11597-616 [PMID: 21832190]
  23. IEEE Trans Cybern. 2016 Feb;46(2):546-57 [PMID: 25781973]
  24. Neuroimage. 2019 Jan 15;185:891-905 [PMID: 29578031]
  25. IEEE Trans Med Imaging. 2020 Jul;39(7):2415-2425 [PMID: 32012001]
  26. Neuroimage. 2013 Oct 15;80:62-79 [PMID: 23684880]
  27. IEEE Trans Med Imaging. 2019 Feb;38(2):540-549 [PMID: 30716024]
  28. Magn Reson Med. 2015 Dec;74(6):1609-20 [PMID: 25533337]
  29. Neuroimage. 2021 Nov 1;241:118429 [PMID: 34311068]
  30. Neuroimage. 2012 Aug 15;62(2):782-90 [PMID: 21979382]
  31. Nat Methods. 2021 Feb;18(2):203-211 [PMID: 33288961]
  32. Proc SPIE Int Soc Opt Eng. 2015;9417: [PMID: 26612964]
  33. PLoS One. 2019 Jul 3;14(7):e0218089 [PMID: 31269041]
  34. Neuroimage. 2018 Jun;173:88-112 [PMID: 29409960]
  35. IEEE Trans Med Imaging. 2021 May;40(5):1363-1376 [PMID: 33507867]
  36. NMR Biomed. 2018 Mar;31(3): [PMID: 29315894]
  37. Neuron. 2012 Aug 9;75(3):380-92 [PMID: 22884322]
  38. Curr Alzheimer Res. 2009 Aug;6(4):347-61 [PMID: 19689234]
  39. Neurosci Biobehav Rev. 2013 Nov;37(9 Pt A):1919-31 [PMID: 23261404]
  40. IEEE Trans Pattern Anal Mach Intell. 2019 Jul;41(7):1559-1572 [PMID: 29993532]
  41. Neuroimage. 2019 Jan 15;185:906-925 [PMID: 29574033]
  42. Cereb Cortex. 2012 Nov;22(11):2478-85 [PMID: 22109543]
  43. IEEE Trans Med Imaging. 2020 Sep;39(9):2806-2817 [PMID: 32091996]
  44. Neuroimage. 2018 Apr 15;170:456-470 [PMID: 28450139]

Grants

  1. RF1 MH123202/NIMH NIH HHS
  2. K01 MH109773/NIMH NIH HHS
  3. RF1 NS128534/NINDS NIH HHS
  4. R01 MH116225/NIMH NIH HHS
  5. R01 MH117943/NIMH NIH HHS
  6. R01 AG075582/NIA NIH HHS
  7. U01 MH110274/NIMH NIH HHS

MeSH Term

Infant, Newborn
Humans
Infant
Brain
Magnetic Resonance Imaging
Brain Diseases
Image Processing, Computer-Assisted

Word Cloud

Created with Highcharts 10.0.0subcorticalinfantsegmentationimagesMRbraininformationframeworkSDMsdatasetstructuresstructuralattention-baseddeeppreciselystagemulti-modalintensityincludingT1wT2wspatialstructureattentionM2A-UnetboundariesimageneonatalPrecisemagneticresonanceplaysessentialrolestudyingearlyfunctionaldevelopmentalpatternsdiagnosisrelateddisordersHoweverduedynamicappearancechangeslowtissuecontrasttinysizechallengingtaskpaperproposecontext-guidedcoarse-to-finesegmentcoarseaimdirectlypredictsigneddistancemapsratioSDM-Unetcanleveragecontextpositionshapetargetgeneratehigh-qualityfinepredictedencodespatial-contextintegratedinputmulti-sourcemulti-pathUnetachievingrefined3DchannelblocksaddedguidefocusimportantsubregionschannelsadditionallyincorporateinnerouterextralabelshelpestimateambiguousvalidatemethodunrelatedComparedelevenstate-of-the-artmethodsproposedconsistentlyachieveshigheraccuracyqualitativequantitativeevaluationsalsoexhibitsgoodgeneralizabilitycontext-informedBrainInfantMRISubcortical

Similar Articles

Cited By