Automatic medical report generation based on deep learning: A state of the art survey.

Xinyao Liu, Junchang Xin, Qi Shen, Zhihong Huang, Zhiqiong Wang
Author Information
  1. Xinyao Liu: College of Medicine and Biological Information Engineering, Northeastern University, 110819, China.
  2. Junchang Xin: College of Computer Science and Engineering, Northeastern University, 110819, China.
  3. Qi Shen: College of Medicine and Biological Information Engineering, Northeastern University, 110819, China.
  4. Zhihong Huang: School of Science and Engineering, University of Dundee, DD1 4HN, UK.
  5. Zhiqiong Wang: College of Medicine and Biological Information Engineering, Northeastern University, 110819, China. Electronic address: wangzq@bmie.neu.edu.cn.

Abstract

With the increasing popularity of medical imaging and its expanding applications, posing significant challenges for radiologists. Radiologists need to spend substantial time and effort to review images and manually writing reports every day. To address these challenges and speed up the process of patient care, researchers have employed deep learning methods to automatically generate medical reports. In recent years, researchers have been increasingly focusing on this task and a large amount of related work has emerged. Although there have been some review articles summarizing the state of the art in this field, their discussions remain relatively limited. Therefore, this paper provides a comprehensive review of the latest advancements in automatic medical report generation, focusing on four key aspects: (1) describing the problem of automatic medical report generation, (2) introducing datasets of different modalities, (3) thoroughly analyzing existing evaluation metrics, (4) classifying existing studies into five categories: retrieval-based, domain knowledge-based, attention-based, reinforcement learning-based, large language models-based, and merged model. In addition, we point out the problems in this field and discuss the directions of future challenges. We hope that this review provides a thorough understanding of automatic medical report generation and encourages the continued development in this area.

Keywords

MeSH Term

Deep Learning
Humans
Diagnostic Imaging

Word Cloud

Created with Highcharts 10.0.0medicalreportgenerationreviewchallengesautomaticreportsprocessresearchersdeeplearningfocusinglargestateartfieldprovidesexistinglanguageAutomaticincreasingpopularityimagingexpandingapplicationsposingsignificantradiologistsRadiologistsneedspendsubstantialtimeeffortimagesmanuallywritingeverydayaddressspeedpatientcareemployedmethodsautomaticallygeneraterecentyearsincreasinglytaskamountrelatedworkemergedAlthougharticlessummarizingdiscussionsremainrelativelylimitedThereforepapercomprehensivelatestadvancementsfourkeyaspects:1describingproblem2introducingdatasetsdifferentmodalities3thoroughlyanalyzingevaluationmetrics4classifyingstudiesfivecategories:retrieval-baseddomainknowledge-basedattention-basedreinforcementlearning-basedmodels-basedmergedmodeladditionpointproblemsdiscussdirectionsfuturehopethoroughunderstandingencouragescontinueddevelopmentareabasedlearning:surveyDeepEncoder–decoderframeworkMedicalimageNatural

Similar Articles

Cited By