OpenLB
Open Library of Bioscience
Advanced Search
Magnetic resonance in medicine
Aug, 2024;92 (2): 853-868.

Accelerated motion correction with deep generative diffusion models.

Brett Levac, Sidharth Kumar, Ajil Jalal, Jonathan I Tamir
Author Information
  1. Brett Levac: Chandra Family Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA. ORCID
  2. Sidharth Kumar: Chandra Family Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA.
  3. Ajil Jalal: Electrical Engineering and Computer Sciences, University of California at Berkeley, Berkeley, California, USA. ORCID
  4. Jonathan I Tamir: Chandra Family Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA. ORCID
PMID: 38688874 DOI: 10.1002/mrm.30082

Abstract

PURPOSE: The aim of this work is to develop a method to solve the ill-posed inverse problem of accelerated image reconstruction while correcting forward model imperfections in the context of subject motion during MRI examinations.
METHODS: The proposed solution uses a Bayesian framework based on deep generative diffusion models to jointly estimate a motion-free image and rigid motion estimates from subsampled and motion-corrupt two-dimensional (2D) k-space data.
RESULTS: We demonstrate the ability to reconstruct motion-free images from accelerated two-dimensional (2D) Cartesian and non-Cartesian scans without any external reference signal. We show that our method improves over existing correction techniques on both simulated and prospectively accelerated data.
CONCLUSION: We propose a flexible framework for retrospective motion correction of accelerated MRI based on deep generative diffusion models, with potential application to other forward model corruptions.

Keywords

MRI reconstruction deep generative diffusion models deep learning motion correction

References

  1. Levac B, Jalal A, Tamir JI. Accelerated motion correction for mri using score���based generative models. Paper presented at: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). 2023a:1���5. doi:10.1109/ISBI53787.2023.10230457
  2. Levac B, Jalal A, Tamir JI. Motion robust reconstruction with score���based generative models. Paper presented at: ISMRM Workshop on Data Sampling and Image Reconstruction. vol. 1. 2023b.
  3. Griswold MA, Jakob PM, Heidemann RM, et al. Generalized autocalibrating partially parallel acquisitions (grappa). Magn Reson Med. 2002;47:1202���1210. doi:10.1002/mrm.10171
  4. Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. Sense: sensitivity encoding for fast mri. Magn Reson Med. 1999;42:952���962. doi:10.1002/(SICI)1522���2594(199911)42:5��952::AID���MRM16��3.0.CO;2���S
  5. Sodickson DK, Manning WJ. Simultaneous acquisition of spatial harmonics (smash): fast imaging with radiofrequency coil arrays. Magn Reson Med. 1997;38:591���603. doi:10.1002/mrm.1910380414
  6. Haldar JP. Low���rank modeling of local k$$ k $$���space neighborhoods (loraks) for constrained mri. IEEE Trans Med Imaging. 2014;33:668���681. doi:10.1109/TMI.2013.2293974
  7. Lustig M, Donoho D, Pauly JM. Sparse MRI: the application of compressed sensing for rapid mr imaging. Magn Reson Med. 2007;58:1182���1195.
  8. Trzasko JD. Exploiting local low���rank structure in higher���dimensional mri applications. In: Ville DVD, Goyal VK, Papadakis M, eds. Wavelets and Sparsity XV. Vol 8858. International Society for Optics and Photonics. SPIE; 2013:885821. doi:10.1117/12.2027059
  9. Ravishankar S, Bresler Y. Mr image reconstruction from highly undersampled k���space data by dictionary learning. IEEE Trans Med Imaging. 2011;30:1028���1041. doi:10.1109/TMI.2010.2090538
  10. Tamir JI, Uecker M, Chen W, et al. T2 shuffling: sharp, multicontrast, volumetric fast spin���echo imaging. Magn Reson Med. 2017;77:180���195. doi:10.1002/mrm.26102
  11. Aggarwal HK, Mani MP, Jacob M. Modl: model���based deep learning architecture for inverse problems. IEEE Trans Med Imaging. 2019;38:394���405. doi:10.1109/TMI.2018.2865356
  12. Arvinte M, Vishwanath S, Tewfik AH, Tamir JI. Deep j���sense: accelerated mri reconstruction via unrolled alternating optimization. arXiv:2103.02087. 2021. doi:10.48550/ARXIV.2103.02087
  13. Bora A, Jalal A, Price E, Dimakis AG. Compressed sensing using generative models. In: Precup D, Teh YW, eds. Proceedings of the 34th International Conference on Machine Learning. Vol 70. Proceedings of Machine Learning Research. PMLR; 2017:537���546.
  14. Chung H, Ye JC. Score���based diffusion models for accelerated mri. Med Image Anal. 2022;80:102479. doi:10.1016/j.media.2022.102479
  15. Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated mri data. Magn Reson Med. 2018;79:3055���3071. doi:10.1002/mrm.26977
  16. Jalal A, Arvinte M, Daras G, Price E, Dimakis AG, Tamir JI. Robust compressed sensing MRI with deep generative priors. In: Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Wortman VJ, eds. Advances in Neural Information Processing Systems. Curran Associates, Inc; 2021:14938���14954.
  17. Luo G, Heide M, Uecker M. MRI reconstruction via data driven markov chain with joint uncertainty estimation. arXiv preprint arXiv:2202.01479. 2022.
  18. Zaitsev M, Maclaren J, Herbst M. Motion artifacts in MRI: a complex problem with many partial solutions. J Magn Reson Imaging. 2015;42:887���901.
  19. Slipsager JM, Glimberg SL, S��gaard J, et al. Quantifying the financial savings of motion correction in brain mri: a model���based estimate of the costs arising from patient head motion and potential savings from implementation of motion correction. J Magn Reson Imaging. 2020;52:731���738.
  20. Cheng JY, Zhang T, Ruangwattanapaisarn N, et al. Free���breathing pediatric mri with nonrigid motion correction and acceleration. J Magn Reson Imaging. 2015;42:407���420.
  21. De Zanche N, Barmet C, Nordmeyer���Massner JA, Pruessmann KP. Nmr probes for measuring magnetic fields and field dynamics in mr systems. Magn Reson Med. 2008;60:176���186.
  22. Ludwig J, Speier P, Seifert F, Schaeffter T, Kolbitsch C. Pilot tone���based motion correction for prospective respiratory compensated cardiac cine mri. Magn Reson Med. 2021;85:2403���2416.
  23. Maclaren J, Herbst M, Speck O, Zaitsev M. Prospective motion correction in brain imaging: a review. Magn Reson Med. 2013;69:621���636.
  24. Nael K, Pawha PS, Fleysher L, et al. Prospective motion correction for brain mri using an external tracking system. J Neuroimaging. 2021;31:57���61.
  25. Johnson PM, Drangova M. Conditional generative adversarial network for 3d rigid���body motion correction in mri. Magn Reson Med. 2019;82:901���910.
  26. Usman M, Latif S, Asim M, Lee B���D, Qadir J. Retrospective motion correction in multishot MRI using generative adversarial network. Sci Rep. 2020;10:4786.
  27. Levac B, Kumar S, Kardonik S, Tamir JI. Fse compensated motion correction fornbsp;mri using data driven methods. Medical Image Computing and Computer Assisted Intervention ��� MICCAI 2022. Springer Nature Switzerland; 2022:707���716.
  28. Cordero���Grande L, Ferrazzi G, Teixeira RPAG, O'Muircheartaigh J, Price AN, Hajnal JV. Motion���corrected mri with disorder: distributed and incoherent sample orders for reconstruction deblurring using encoding redundancy. Magn Reson Med. 2020;84:713���726. doi:10.1002/mrm.28157
  29. Cordero���Grande L, Teixeira RPAG, Hughes EJ, Hutter J, Price AN, Hajnal JV. Sensitivity encoding for aligned multishot magnetic resonance reconstruction. IEEE Trans Comput Imaging. 2016;2:266���280.
  30. Haskell MW, Cauley SF, Wald LL. Targeted motion estimation and reduction (TAMER): data consistency based motion mitigation for mri using a reduced model joint optimization. IEEE Trans Med Imaging. 2018;37:1253���1265.
  31. Lingala SG, DiBella E, Jacob M. Deformation corrected compressed sensing (dc���cs): a novel framework for accelerated dynamic mri. IEEE Trans Med Imaging. 2015;34:72���85. doi:10.1109/TMI.2014.2343953
  32. Rizzuti G, Sbrizzi A, van Leeuwen T. Joint retrospective motion correction and reconstruction for brain mri with a reference contrast. IEEE Trans Comput Imaging. 2022;8:490���504. doi:10.1109/TCI.2022.3183383
  33. Haskell MW, Cauley SF, Bilgic B, et al. Network accelerated motion estimation and reduction (NAMER): convolutional neural network guided retrospective motion correction using a separable motion model. Magn Reson Med. 2019;82:1452���1461.
  34. Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. 2020. doi:10.48550/ARXIV.2006.11239
  35. Song Y, Ermon S. Generative modeling by estimating gradients of the data distribution. Paper presented at: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). 2019; Vancouver, Canada:11918���11930.
  36. Luo G, Zhao N, Jiang W, Hui ES, Cao P. Mri reconstruction using deep bayesian estimation. Magn Reson Med. 2020;84:2246���2261. doi:10.1002/mrm.28274
  37. Tezcan KC, Baumgartner CF, Luechinger R, Pruessmann KP, Konukoglu E. Mr image reconstruction using deep density priors. IEEE Trans Med Imaging. 2019;38:1633���1642. doi:10.1109/TMI.2018.2887072
  38. Blau Y, Michaeli T. The perception���distortion tradeoff. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, USA, 2018, pp. 6228���6237. doi: 10.1109/CVPR.2018.00652
  39. Karras T, Aittala M, Aila T, Laine S. Elucidating the design space of diffusion���based generative models. arXiv:2206.00364. 2022.
  40. Song Y, Sohl���Dickstein J, Kingma DP, Kumar A, Ermon S, Poole B. Score���based generative modeling through stochastic differential equations. arXiv:2011.13456. 2021.
  41. Anderson BDO. Reverse���time diffusion equation models. Stoch Process Appl. 1982;12:313���326.
  42. Vincent P. A connection between score matching and denoising autoencoders. Neural Comput. 2011;23:1661���1674. doi:10.1162/NECO��a��00142
  43. Chung H, Kim J, Mccann MT, Klasky ML, Ye JC. Diffusion posterior sampling for general noisy inverse problems. arXiv:2209.14687. 2023.
  44. Efron B. Tweedie's formula and selection bias. J Am Stat Assoc. 2011;106:1602���1614. doi:10.1198/jasa.2011.tm11181
  45. Nurdinova A, Ruschke S, Karampinos DC. On the limits of model���based reconstruction for retrospective motion correction in the presence of spin history effects. Paper presented at: Annual Meeting of ISMRM 2022. 2022. Abstract 0862.
  46. Kellman P, Epstein FH, Mcveigh ER. Adaptive sensitivity encoding incorporating temporal filtering(TSENSE). Magn Reson Med. 2001;45:846���852.
  47. Pruessmann KP, Weiger M, B��rnert P, Boesiger P. Advances in sensitivity encoding with arbitrary k���space trajectories. Magn Reson Med. 2001;46:638���651. doi:10.1002/mrm.1241
  48. Ronneberger O, Fischer P, Brox T. U���net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer���Assisted Intervention���MICCAI 2015: 18th International Conference, Munich, Germany, October 5���9, 2015, Proceedings, Part III 18. Springer; 2015:234���241.
  49. Knoll F, Zbontar J, Sriram A, et al. fastMRI: A publicly available raw k���space and dicom dataset of knee images for accelerated mr image reconstruction using machine learning. Radiol Artif Intell. 2020;2.
  50. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016:770���778.
  51. Simonyan K, Zisserman A. Very deep convolutional networks for large���scale image recognition. arXiv:1409.1556. 2015.
  52. Kumar S, Saber H, Charron O, Freeman L, Tamir JI. Correcting synthetic MRI contrast���weighted images using deep learning. Magn Reson Imaging. 2024;106:43���54.
  53. Pipe JG. Motion correction with propeller mri: application to head motion and free���breathing cardiac imaging. Magn Reson Med. 1999;42:963���969. doi:10.1002/(SICI)1522���2594(199911)42:5��963::AID���MRM17��3.0.CO;2���L
  54. Uecker M, Lai P, Murphy MJ, et al. Espirit���an eigenvalue approach to autocalibrating parallel mri: where sense meets grappa. Magn Reson Med. 2014;71:990���1001. doi:10.1002/mrm.24751
  55. Shimron E, Tamir JI, Wang K, Lustig M. Implicit data crimes: machine learning bias arising from misuse of public data. Proc Natl Acad Sci. 2022;119:e2117203119. doi:10.1073/pnas.2117203119
  56. Chung H, Kim J, Kim S, Ye JC. Parallel diffusion models of operator and image for blind inverse problems. Paper presented at: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023:6059���6069.
  57. Murata N, Saito K, Lai CH, et al. Gibbsddrm: a partially collapsed gibbs sampler for solving blind inverse problems with denoising diffusion restoration. arXiv preprint arXiv:2301.12686. 2023.
  58. Singh NM, Dey N, Hoffmann M, et al. Data consistent deep rigid mri motion correction. Paper presented at: Medical Imaging with Deep Learning. 2023. https://proceedings.mlr.press/v227/singh24a.html
  59. Pan J, Rueckert D, K��stner T, Hammernik K. Learning���based and unrolled motion���compensated reconstruction for cardiac mr cine imaging. International Conference on Medical Image Computing and Computer���Assisted Intervention. Springer; 2022:686���696.
  60. Xu X, Gan W, Kothapalli SV, Yablonskiy DA, Kamilov US. Correct: a deep unfolding framework for motion���corrected quantitative r2* mapping. arXiv:2210.06330. 2022.

Grants

  1. U24EB029240/NIH HHS
  2. W911NF2110117/ARO
  3. 051242-002/ARO
  4. IFML 2019844/NSF
  5. CCF2239687(NSFCAREER)/NSF
  6. /Oracle Research Fellowship
  7. /Aspect Imaging
  8. /Google research scholars program

MeSH Term

Humans
Image Processing, Computer-Assisted
Motion
Algorithms
Bayes Theorem
Brain
Computer Simulation
Magnetic Resonance Imaging
Artifacts
Retrospective Studies
Diffusion Magnetic Resonance Imaging
Journal Article

Word Cloud

Created with Highcharts 10.0.0motiondeepacceleratedgenerativediffusionmodelscorrectionMRImethodimagereconstructionforwardmodelframeworkbasedmotion-freetwo-dimensional2DdataPURPOSE:aimworkdevelopsolveill-posedinverseproblemcorrectingimperfectionscontextsubjectexaminationsMETHODS:proposedsolutionusesBayesianjointlyestimaterigidestimatessubsampledmotion-corruptk-spaceRESULTS:demonstrateabilityreconstructimagesCartesiannon-CartesianscanswithoutexternalreferencesignalshowimprovesexistingtechniquessimulatedprospectivelyCONCLUSION:proposeflexibleretrospectivepotentialapplicationcorruptionsAcceleratedlearning

Similar Articles

Cited By

No available data.