VINNA for neonates: Orientation independence through latent augmentations.

Leonie Henschel, David Kügler, Lilla Zöllei, Martin Reuter
Author Information
  1. Leonie Henschel: German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany.
  2. David Kügler: German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany.
  3. Lilla Zöllei: A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA.
  4. Martin Reuter: German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany.

Abstract

A robust, fast, and accurate segmentation of neonatal brain images is highly desired to better understand and detect changes during development and disease, specifically considering the rise in imaging studies for this cohort. Yet, the limited availability of ground truth datasets, lack of standardized acquisition protocols, and wide variations of head positioning in the scanner pose challenges for method development. A few automated image analysis pipelines exist for newborn brain Magnetic Resonance Image (MRI) segmentation, but they often rely on time-consuming non-linear spatial registration procedures and require resampling to a common resolution, subject to loss of information due to interpolation and down-sampling. Without registration and image resampling, variations with respect to head positions and voxel resolutions have to be addressed differently. In deep learning, external augmentations such as rotation, translation, and scaling are traditionally used to artificially expand the representation of spatial variability, which subsequently increases both the training dataset size and robustness. However, these transformations in the image space still require resampling, reducing accuracy specifically in the context of label interpolation. We recently introduced the concept of resolution-independence with the Voxel-size Independent Neural Network framework, VINN. Here, we extend this concept by additionally shifting all rigid-transforms into the network architecture with a four degree of freedom (4-DOF) transform module, enabling resolution-aware internal augmentations (VINNA) for deep learning. In this work, we show that VINNA (i) significantly outperforms state-of-the-art external augmentation approaches, (ii) effectively addresses the head variations present specifically in newborn datasets, and (iii) retains high segmentation accuracy across a range of resolutions (0.5-1.0 mm). Furthermore, the 4-DOF transform module together with internal augmentations is a powerful, general approach to implement spatial augmentation without requiring image or label interpolation. The specific network application to newborns will be made publicly available as VINNA4neonates.

Keywords

References

  1. PLoS One. 2011 Apr 14;6(4):e18746 [PMID: 21533194]
  2. Neuroimage. 2020 Oct 1;219:117026 [PMID: 32522665]
  3. Neuroimage. 2011 Feb 14;54(4):2750-63 [PMID: 20969966]
  4. Cereb Cortex. 2012 Nov;22(11):2478-85 [PMID: 22109543]
  5. Brain Struct Funct. 2018 Dec;223(9):4153-4168 [PMID: 30187191]
  6. Neuroimage. 2020 Oct 1;219:117012 [PMID: 32526386]
  7. Pediatr Radiol. 2023 Jul;53(8):1685-1697 [PMID: 36884052]
  8. Proc SPIE Int Soc Opt Eng. 2019 Feb;10953: [PMID: 31057202]
  9. Neuroinformatics. 2013 Apr;11(2):211-25 [PMID: 23055044]
  10. Med Image Anal. 2012 Dec;16(8):1550-64 [PMID: 22939612]
  11. Proc IEEE Int Symp Biomed Imaging. 2016;2016:1342-1345 [PMID: 27668065]
  12. Front Neurosci. 2021 Jul 12;15:666020 [PMID: 34321992]
  13. Front Neuroinform. 2019 Oct 17;13:67 [PMID: 31749693]
  14. Neuroscience. 2014 Sep 12;276:48-71 [PMID: 24378955]
  15. IEEE Trans Med Imaging. 2020 Apr;39(4):898-909 [PMID: 31449009]
  16. Proc Natl Acad Sci U S A. 2012 Oct 9;109(41):16480-5 [PMID: 23012402]
  17. Neuroimage. 2018 Apr 15;170:446-455 [PMID: 28445774]
  18. Neuroimage. 2006 Mar;30(1):184-202 [PMID: 16376577]
  19. Neuroimage. 2019 Feb 1;186:713-727 [PMID: 30502445]
  20. Front Hum Neurosci. 2015 Feb 18;9:21 [PMID: 25741260]
  21. Neuroimage. 2022 May 1;251:118933 [PMID: 35122967]
  22. Neuroimage. 2009 Aug 15;47(2):564-72 [PMID: 19409502]
  23. IEEE Trans Med Imaging. 2001 Jan;20(1):45-57 [PMID: 11293691]
  24. Neuroimage. 2012 Jul 16;61(4):1402-18 [PMID: 22430496]
  25. IEEE Trans Med Imaging. 2019 Feb 27;: [PMID: 30835215]
  26. Proc Mach Learn Res. 2022 Jul;172:1075-1084 [PMID: 36968615]
  27. J Med Imaging (Bellingham). 2017 Apr;4(2):024003 [PMID: 28439524]
  28. Patch Based Tech Med Imaging (2015). 2015 Oct;9467:28-36 [PMID: 28393146]
  29. Neuroimage. 2016 Jan 15;125:456-478 [PMID: 26499811]
  30. Neural Netw. 2019 Aug;116:25-34 [PMID: 30986724]
  31. Neuroimage. 2019 Jan 15;185:891-905 [PMID: 29578031]
  32. Hum Brain Mapp. 2023 Mar;44(4):1779-1792 [PMID: 36515219]
  33. Lancet. 2000 Sep 30;356(9236):1162-3 [PMID: 11030298]
  34. Front Neuroinform. 2016 Mar 29;10:12 [PMID: 27065840]
  35. Med Image Anal. 2012 Dec;16(8):1565-79 [PMID: 22921305]
  36. Neuroimage. 2018 Apr 15;170:434-445 [PMID: 28223187]
  37. Neuroimage. 2019 Jan 15;185:750-763 [PMID: 29852283]
  38. Nat Protoc. 2023 May;18(5):1488-1509 [PMID: 36869216]
  39. Front Neurosci. 2019 Feb 05;13:34 [PMID: 30804737]
  40. Domain Adapt Represent Transf Med Image Learn Less Labels Imperfect Data (2019). 2019 Oct;11795:243-251 [PMID: 32090208]
  41. Proc Natl Acad Sci U S A. 2018 Mar 20;115(12):3156-3161 [PMID: 29507201]
  42. Neuroimage. 2012 Aug 15;62(2):782-90 [PMID: 21979382]
  43. Med Image Anal. 2019 Dec;58:101540 [PMID: 31398617]
  44. J Magn Reson Imaging. 2021 May;53(5):1318-1343 [PMID: 32420684]
  45. Neuroimage. 2016 Jan 1;124(Pt B):1149-1154 [PMID: 25937488]
  46. Magn Reson Med. 2018 Mar;79(3):1365-1376 [PMID: 28626962]
  47. Nat Methods. 2021 Feb;18(2):203-211 [PMID: 33288961]
  48. Neuroimage. 2010 Jun;51(2):684-93 [PMID: 20171290]
  49. Neuroimage. 2015 Sep;118:628-41 [PMID: 26057591]
  50. Neuroimage. 2006 Jul 1;31(3):968-80 [PMID: 16530430]
  51. Neuroimage. 2019 Jul 1;194:105-119 [PMID: 30910724]
  52. Neuroimage. 2020 Sep;218:116946 [PMID: 32442637]
  53. Neuroimage. 2018 Oct 1;179:11-29 [PMID: 29890325]
  54. Neuropsychopharmacology. 2021 Jan;46(1):131-142 [PMID: 32541809]
  55. Neuroimage. 2012 Feb 1;59(3):2255-65 [PMID: 21985910]
  56. Comput Methods Programs Biomed. 2021 Sep;208:106236 [PMID: 34311413]
  57. Front Neurosci. 2012 Dec 05;6:171 [PMID: 23227001]
  58. Neuroimage. 2018 Jun;173:88-112 [PMID: 29409960]
  59. Med Image Anal. 2005 Oct;9(5):457-66 [PMID: 16019252]
  60. Proc SPIE Int Soc Opt Eng. 2015 Feb 21;9413: [PMID: 26089584]
  61. JAMA Psychiatry. 2021 May 1;78(5):471-472 [PMID: 33295951]
  62. IEEE Trans Med Imaging. 2017 Feb;36(2):674-683 [PMID: 27845654]
  63. IEEE Trans Med Imaging. 2021 May;40(5):1363-1376 [PMID: 33507867]
  64. Neuroimage. 2015 Mar;108:214-24 [PMID: 25562829]
  65. Magn Reson Med. 2017 Aug;78(2):794-804 [PMID: 27643791]
  66. Neuroimage. 2019 Jan 15;185:934-946 [PMID: 29522888]
  67. NMR Biomed. 2019 Jul;32(7):e4103 [PMID: 31038246]
  68. Neuron. 2002 Jan 31;33(3):341-55 [PMID: 11832223]
  69. Brain Inform. 2022 May 28;9(1):12 [PMID: 35633447]
  70. IEEE Trans Med Imaging. 2014 Sep;33(9):1818-31 [PMID: 24816548]
  71. Neuroimage. 2012 Aug 15;62(2):774-81 [PMID: 22248573]
  72. Neuroimage. 2012 Sep;62(3):1499-509 [PMID: 22713673]
  73. Comput Med Imaging Graph. 2020 Jan;79:101660 [PMID: 31785402]
  74. Neuroimage. 2021 Aug 15;237:118206 [PMID: 34048902]

Word Cloud

Created with Highcharts 10.0.0imageaugmentationssegmentationbrainspecificallyvariationsheadnewbornspatialresamplinginterpolationdeeplearningVINNAdevelopmentdatasetsMRIregistrationrequireresolutionsexternalaccuracylabelconceptnetwork4-DOFtransformmoduleinternalaugmentation0robustfastaccurateneonatalimageshighlydesiredbetterunderstanddetectchangesdiseaseconsideringriseimagingstudiescohortYetlimitedavailabilitygroundtruthlackstandardizedacquisitionprotocolswidepositioningscannerposechallengesmethodautomatedanalysispipelinesexistMagneticResonanceImageoftenrelytime-consumingnon-linearprocedurescommonresolutionsubjectlossinformationduedown-samplingWithoutrespectpositionsvoxeladdresseddifferentlyrotationtranslationscalingtraditionallyusedartificiallyexpandrepresentationvariabilitysubsequentlyincreasestrainingdatasetsizerobustnessHowevertransformationsspacestillreducingcontextrecentlyintroducedresolution-independenceVoxel-sizeIndependentNeuralNetworkframeworkVINNextendadditionallyshiftingrigid-transformsarchitecturefourdegreefreedomenablingresolution-awareworkshowsignificantlyoutperformsstate-of-the-artapproachesiieffectivelyaddressespresentiiiretainshighacrossrange5-1mmFurthermoretogetherpowerfulgeneralapproachimplementwithoutrequiringspecificapplicationnewbornswillmadepubliclyavailableVINNA4neonatesneonates:Orientationindependencelatentartificialintelligencecomputationalneuroimaginghigh-resolutionstructural

Similar Articles

Cited By (1)