OpenLB
Open Library of Bioscience
Advanced Search
Annals of biomedical engineering
Feb, 2023;51 (2): 352-362.

Intraoperative Detection of Surgical Gauze Using Deep Convolutional Neural Network.

Shuo-Lun Lai, Chi-Sheng Chen, Been-Ren Lin, Ruey-Feng Chang
Author Information
  1. Shuo-Lun Lai: Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No.1, Sec.4, Roosevelt Road, Taipei, 10617, Taiwan.
  2. Chi-Sheng Chen: Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No.1, Sec.4, Roosevelt Road, Taipei, 10617, Taiwan.
  3. Been-Ren Lin: Division of Colorectal Surgery, Department of Surgery, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei, Taiwan.
  4. Ruey-Feng Chang: Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No.1, Sec.4, Roosevelt Road, Taipei, 10617, Taiwan. rfchang@csie.ntu.edu.tw.
PMID: 35972601 DOI: 10.1007/s10439-022-03033-9

Abstract

During laparoscopic surgery, surgical gauze is usually inserted into the body cavity to help achieve hemostasis. Retention of surgical gauze in the body cavity may necessitate reoperation and increase surgical risk. Using deep learning technology, this study aimed to propose a neural network model for gauze detection from the surgical video to record the presence of the gauze. The model was trained by the training group using YOLO (You Only Look Once)v5x6, then applied to the testing group. Positive predicted value (PPV), sensitivity, and mean average precision (mAP) were calculated. Furthermore, a timeline of gauze presence in the video was drawn by the model as well as human annotation to evaluate the accuracy. After the model was well-trained, the PPV, sensitivity, and mAP in the testing group were 0.920, 0.828, and 0.881, respectively. The inference time was 11.3 ms per image. The average accuracy of the model adding a marking and filtering process was 0.899. In conclusion, surgical gauze can be successfully detected using deep learning in the surgical video. Our model provided a fast detection of surgical gauze, allowing further real-time gauze tracing in laparoscopic surgery that may help surgeons recall the location of the missing gauze.

Keywords

Convolutional neural network Deep learning Detection Gauze Laparoscopic surgery You Only Look Once (YOLO)

References

  1. Ahmed, K. R. Smart pothole detection using deep learning based on dilated convolution. Sensors (Basel). 21:8406, 2021 [DOI: 10.3390/s21248406]
  2. Bochkovskiy, A., C. Y. Wang and H. Y. M. Liao. YOLOv4: Optimal Speed and Accuracy of Object Detection. https://arxiv.org/2004.10934v1 , 2020.
  3. Chilamkurthy, S., R. Ghosh, S. Tanamala, M. Biviji, N. G. Campeau, V. K. Venugopal, V. Mahajan, P. Rao, and P. Warier. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet. 392:2388–2396, 2018 [DOI: 10.1016/S0140-6736(18)31645-3]
  4. de la Fuente, E., F. M. Trespaderne, L. Santos, J. C. Fraile and J. P. Turiel. Parallel computing for real time gauze detection in laparoscopy images. In: 2nd International Conference on Bio-engineering for Smart Technologies (BioSMART). https://doi.org/10.1109/BIOSMART.2017.8095328 .
  5. de la Fuente López, E., Á. Muñoz García, L. S. Santos Del Blanco, J. C. Fraile Marinero and J. Pérez Turiel. Automatic gauze tracking in laparoscopic surgery using image texture analysis. Comput. Methods Programs Biomed. 190:105378, 2020.
  6. Ding, Y., J. H. Sohn, M. G. Kawczynski, H. Trivedi, R. Harnish, N. W. Jenkins, D. Lituiev, T. P. Copeland, M. S. Aboian, C. M. Mari Aparici, S. C. Behr, R. R. Flavell, S. Y. Huang, K. A. Zalocusky, L. Nardo, Y. Seo, R. A. Hawkins, M. Hernandez Pampaloni, D. Hadley, and B. L. Franc. A deep learning model to predict a diagnosis of Alzheimer disease by using F-FDG PET of the Brain. Radiology. 290:456–464, 2019 [DOI: 10.1148/radiol.2018180958]
  7. Dong, X., Y. Lei, T. Wang, M. Thomas, L. Tang, W. J. Curran, T. Liu, and X. Yang. Automatic multiorgan segmentation in thorax CT images using U-net-GAN. Med. Phys. 46:2157–2168, 2019 [DOI: 10.1002/mp.13458]
  8. Everingham, M. and J. Winn. The Pascal Visual Object Classes Challenge (VOC2010) Development Kit.
  9. Ghesu, F. C., E. Krubasik, B. Georgescu, V. Singh, Y. Zheng, J. Hornegger, and D. Comaniciu. Marginal space deep learning: efficient architecture for volumetric image parsing. IEEE Trans. Med. Imaging. 35:1217–1228, 2016 [DOI: 10.1109/TMI.2016.2538802]
  10. Gibbs, V. C. Retained surgical items and minimally invasive surgery. World J. Surg. 35:1532–1539, 2011 [DOI: 10.1007/s00268-011-1060-4]
  11. Hashimoto, D. A., G. Rosman, E. R. Witkowski, C. Stafford, A. J. Navarette-Welton, D. W. Rattner, K. D. Lillemoe, D. L. Rus, and O. R. Meireles. Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann. Surg. 270:414–421, 2019 [DOI: 10.1097/SLA.0000000000003460]
  12. Kitaguchi, D., N. Takeshita, H. Matsuzaki, H. Takano, Y. Owada, T. Enomoto, T. Oda, H. Miura, T. Yamanashi, M. Watanabe, D. Sato, Y. Sugomori, S. Hara, and M. Ito. Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach. Surg. Endosc. 34:4924–4931, 2020 [DOI: 10.1007/s00464-019-07281-0]
  13. Lekadir, K., A. Galimzianova, A. Betriu, M. Del Mar Vila, L. Igual, D. L. Rubin, E. Fernandez, P. Radeva, and S. Napel. A convolutional neural network for automatic characterization of plaque composition in carotid ultrasound. IEEE J. Biomed. Health Inform. 21:48–55, 2017 [DOI: 10.1109/JBHI.2016.2631401]
  14. Liu, F., H. Jang, R. Kijowski, T. Bradshaw, and A. B. McMillan. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 286:676–684, 2018 [DOI: 10.1148/radiol.2017170700]
  15. Ma, J., F. Wu, T. Jiang, J. Zhu, and D. Kong. Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med. Phys. 44:1678–1691, 2017 [DOI: 10.1002/mp.12134]
  16. Madad Zadeh, S. M., T. Francois, L. Calvet, P. Chauvet, M. Canis, A. Bartoli, and N. Bourdel. SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology. Surg. Endosc. 34:5377–5383, 2020 [DOI: 10.1007/s00464-019-07330-8]
  17. Nema, S., A. Dudhane, S. Murala, and S. Naidu. RescueNet: an unpaired GAN for brain tumor segmentation. Biomed. Signal Process. Control. 55:101641, 2020 [DOI: 10.1016/j.bspc.2019.101641]
  18. Rahman, R., Z. B. Bin Azad and M. Bakhtiar Hasan. Densely populated traffic detection using YOLOv5 and non-maximum suppression ensembling. Lecture Notes on Data Engineering and Communications Technologies. Proceedings of the International Conference on Big Data, IoT, and Machine Learning:567–578. https://doi.org/10.1007/978-981-16-6636-0_43 , 2022.
  19. Ravishankar, H., R. Venkataramani, S. Thiruvenkadam, P. Sudhakar, and V. Vaidya. Learning and incorporating shape models for semantic segmentation. Med. Image Comput. Comput. Assist. Interv. 2017. https://doi.org/10.1007/978-3-319-66182-7_24 [DOI: 10.1007/978-3-319-66182-7_24]
  20. Redmon, J., S. Divvala, R. Girshick and A. Farhadi. You only look once: unified, real-time object detection. https://arxiv.org/1506.02640v5 , 2015.
  21. Redmon, J. and A. Farhadi. YOLO 9000: Better, Faster, Stronger. https://arxiv.org/1612.08242v1 , 2016.
  22. Redmon, J. and A. Farhadi. YOLOv3: an Incremental Improvement. https://arxiv.org/1804.02767v1 , 2018.
  23. Ren, S., K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. https://arxiv.org/1506.01497 , 2015.
  24. Rivera, N., R. Mountain, L. Assumpcao, A. A. Williams, A. B. Cooper, D. L. Lewis, R. C. Benson, J. A. Miragliotta, M. Marohn, and R. H. Taylor. ASSIST - Automated system for surgical instrument and sponge tracking. In: 2008 IEEE Int. Con. on RFID, pp. 297–302, 2008.
  25. Rodriguez-Diaz, E., G. Baffy, W. K. Lo, H. Mashimo, G. Vidyarthi, S. S. Mohapatra, and S. K. Singh. Real-time artificial intelligence-based histologic classification of colorectal polyps with augmented visualization. Gastrointest. Endosc. 93:662–670, 2021 [DOI: 10.1016/j.gie.2020.09.018]
  26. Serra, J., X. Matias-Guiu, R. Calabuig, P. Garcia, F. J. Sancho, and J. P. La Calle. Surgical gauze pseudotumor. Am. J. Surg. 155:235–237, 1988 [DOI: 10.1016/S0002-9610(88)80702-5]
  27. Shvets, A. A., A. Rakhlin, A. A. Kalinin and V. I. Iglovikov. Automatic instrument segmentation in robot-assisted surgery using deep learning. In: 17th IEEE International Conference on Machine Learning and Applications (ICMLA). https://doi.org/10.1109/ICMLA.2018.00100 .
  28. Sombune, P., P. Phienphanich, S. Phuechpanpaisal, S. Muengtaweepongsa, A. Ruamthanthong and C. Tantibundhit. Automated embolic signal detection using deep convolutional neural network. In: Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 3365–3368, 2017.
  29. Suk, H. I., C. Y. Wee, S. W. Lee, and D. Shen. State-space model with deep learning for functional dynamics estimation in resting-state fMRI. Neuroimage. 129:292–307, 2016 [DOI: 10.1016/j.neuroimage.2016.01.005]
  30. Tokuyasu, T., Y. Iwashita, Y. Matsunobu, T. Kamiyama, M. Ishikake, S. Sakaguchi, K. Ebe, K. Tada, Y. Endo, T. Etoh, M. Nakashima, and M. Inomata. Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg. Endosc. 35:1651–1658, 2021 [DOI: 10.1007/s00464-020-07548-x]
  31. Twinanda, A. P., S. Shehata, D. Mutter, J. Marescaux, M. de Mathelin, and N. Padoy. EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging. 36:86–97, 2017 [DOI: 10.1109/TMI.2016.2593957]
  32. Ultralytics. Yolov5. https://github.com/ultralytics/yolov5 , 2020.
  33. Voigtlaender, P., M. Krause, A. Osep, J. Luiten, B. Balachandar Gnana Sekar, A. Geiger and B. Leibe. MOTS: Multi-object Tracking and Segmentation. https://arxiv.org/1902.03604v2 .
  34. Yang, X., L. Yu, L. Wu, Y. Wang, D. Ni, J. Qin and P. A. Heng. Fine-grained recurrent neural networks for automatic prostate segmentation in ultrasound images. AAAI. In: Proceedings of the AAAI Conference on Artificial Intelligence 31, 2017.
  35. Zhang, R., Y. Zheng, C. C. Y. Poon, D. Shen, and J. Y. W. Lau. Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker. Pattern Recognit. 83:209–219, 2018 [DOI: 10.1016/j.patcog.2018.05.026]
  36. Zheng, Z., P. Wang, W. Liu, J. Li, R. Ye and D. Ren. Distance-IoU loss: faster and better learning for bounding box regression. AAAI. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12993–13000, 2020.

MeSH Term

Humans
Bandages
Laparoscopy
Neural Networks, Computer
Reoperation
Journal Article
Article has an altmetric score of 1

Word Cloud

Created with Highcharts 10.0.0gauzesurgicalmodel0surgerylearningvideogrouplaparoscopicbodycavityhelpmayUsingdeepneuralnetworkdetectionpresenceusingYOLOLooktestingPPVsensitivityaveragemAPaccuracyDetectionGauzeDeepConvolutionalusuallyinsertedachievehemostasisRetentionnecessitatereoperationincreaserisktechnologystudyaimedproposerecordtrainedtrainingv5x6appliedPositivepredictedvaluemeanprecisioncalculatedFurthermoretimelinedrawnwellhumanannotationevaluatewell-trained920828881respectivelyinferencetime113 msperimageaddingmarkingfilteringprocess899conclusioncansuccessfullydetectedprovidedfastallowingreal-timetracingsurgeonsrecalllocationmissingIntraoperativeSurgicalNeuralNetworkLaparoscopic

Similar Articles

Cited By