OpenLB
Open Library of Bioscience
Advanced Search
Journal of imaging informatics in medicine
Jun, 2024;37 (3): 1160-1176.

Fast Real-Time Brain Tumor Detection Based on Stimulated Raman Histology and Self-Supervised Deep Learning Model.

Zijun Wang, Kaitai Han, Wu Liu, Zhenghui Wang, Chaojing Shi, Xi Liu, Mengyuan Huang, Guocheng Sun, Shitou Liu, Qianjin Guo
Author Information
  1. Zijun Wang: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  2. Kaitai Han: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  3. Wu Liu: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  4. Zhenghui Wang: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  5. Chaojing Shi: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  6. Xi Liu: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  7. Mengyuan Huang: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  8. Guocheng Sun: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  9. Shitou Liu: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
  10. Qianjin Guo: Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China. guoqj@iccas.ac.cn. ORCID
PMID: 38326533 DOI: 10.1007/s10278-024-01001-4

Abstract

In intraoperative brain cancer procedures, real-time diagnosis is essential for ensuring safe and effective care. The prevailing workflow, which relies on histological staining with hematoxylin and eosin (H&E) for tissue processing, is resource-intensive, time-consuming, and requires considerable labor. Recently, an innovative approach combining stimulated Raman histology (SRH) and deep convolutional neural networks (CNN) has emerged, creating a new avenue for real-time cancer diagnosis during surgery. While this approach exhibits potential, there exists an opportunity for refinement in the domain of feature extraction. In this study, we employ coherent Raman scattering imaging method and a self-supervised deep learning model (VQVAE2) to enhance the speed of SRH image acquisition and feature representation, thereby enhancing the capability of automated real-time bedside diagnosis. Specifically, we propose the VQSRS network, which integrates vector quantization with a proxy task based on patch annotation for analysis of brain tumor subtypes. Training on images collected from the SRS microscopy system, our VQSRS demonstrates a significant speed enhancement over traditional techniques (e.g., 20-30 min). Comparative studies in dimensionality reduction clustering confirm the diagnostic capacity of VQSRS rivals that of CNN. By learning a hierarchical structure of recognizable histological features, VQSRS classifies major tissue pathological categories in brain tumors. Additionally, an external semantic segmentation method is applied for identifying tumor-infiltrated regions in SRH images. Collectively, these findings indicate that this automated real-time prediction technique holds the potential to streamline intraoperative cancer diagnosis, providing assistance to pathologists in simplifying the process.

Keywords

Brain tumor subtype classification Generative deep learning Intraoperative diagnosis SRS imaging system Stimulated Raman histology Vector quantization

References

  1. Ferlay J, Colombet M, Soerjomataram I, Parkin DM, Piñeros M, Znaor A, Bray F: Cancer statistics for the year 2020: An overview. Int J Cancer. 2021, 149:778–89. [DOI: 10.1002/ijc.33588]
  2. Chaya N: Poor access to health Services: Ways Ethiopia is overcoming it. Res Comment. 2007, 2:1–6.
  3. Hamilton PW, Van Diest PJ, Williams R, Gallagher AG: Do we see what we think we see? The complexities of morphological assessment. J Pathol J Pathol Soc G B Irel. 2009, 218:285–91.
  4. Novis DA, Zarbo RJ: Interinstitutional comparison of frozen section turnaround time. Arch Pathol Lab Med. 1997, 121:559. [PMID: 9199619]
  5. Gal AA, Cagle PT: The 100-year anniversary of the description of the frozen section procedure. Jama. 2005, 294:3135–7. [DOI: 10.1001/jama.294.24.3135]
  6. Wei L, Min W: Electronic Preresonance Stimulated Raman Scattering Microscopy. J Phys Chem Lett. 2018, 9:4294–301. https://doi.org/10.1021/acs.jpclett.8b00204 [DOI: 10.1021/acs.jpclett.8b00204]
  7. Xiong H, Qian N, Miao Y, Zhao Z, Min W: Stimulated Raman Excited Fluorescence Spectroscopy of Visible Dyes. J Phys Chem Lett. 2019, 10:3563–70. https://doi.org/10.1021/acs.jpclett.9b01289 [DOI: 10.1021/acs.jpclett.9b01289]
  8. Xiong H, Shi L, Wei L, Shen Y, Long R, Zhao Z, Min W: Stimulated Raman excited fluorescence spectroscopy and imaging. Nat Photonics. 2019, 13:412–7. [DOI: 10.1038/s41566-019-0396-4]
  9. Xiong H, Min W: Combining the best of two worlds: Stimulated Raman excited fluorescence. J Chem Phys. 2020, 153.
  10. Saar BG, Freudiger CW, Reichman J, Stanley CM, Holtom GR, Xie XS: Video-rate molecular imaging in vivo with stimulated Raman scattering. science. 2010, 330:1368–70.
  11. Ozeki Y, Umemura W, Otsuka Y, et al.: High-speed molecular spectral imaging of tissue with stimulated Raman scattering. Nat Photonics. 2012, 6:845–51. [DOI: 10.1038/nphoton.2012.263]
  12. Liao C-S, Slipchenko MN, Wang P, Li J, Lee S-Y, Oglesbee RA, Cheng J-X: Microsecond scale vibrational spectroscopic imaging by multiplex stimulated Raman scattering microscopy. Light Sci Appl. 2015, 4:e265–e265. [DOI: 10.1038/lsa.2015.38]
  13. Liao C-S, Wang P, Wang P, Li J, Lee HJ, Eakins G, Cheng J-X: Spectrometer-free vibrational imaging by retrieving stimulated Raman signal from highly scattered photons. Sci Adv. 2015, 1:e1500738. https://doi.org/10.1126/sciadv.1500738 [DOI: 10.1126/sciadv.1500738]
  14. Wakisaka Y, Suzuki Y, Iwata O, et al.: Probing the metabolic heterogeneity of live Euglena gracilis with stimulated Raman scattering microscopy. Nat Microbiol. 2016, 1:1–4. [DOI: 10.1038/nmicrobiol.2016.124]
  15. Kim D, Choi DS, Kwon J, Shim S-H, Rhee H, Cho M: Selective Suppression of Stimulated Raman Scattering with Another Competing Stimulated Raman Scattering. J Phys Chem Lett. 2017, 8:6118–23. https://doi.org/10.1021/acs.jpclett.7b02752 [DOI: 10.1021/acs.jpclett.7b02752]
  16. Bi Y, Yang C, Chen Y, et al.: Near-resonance enhanced label-free stimulated Raman scattering microscopy with spatial resolution near 130 nm. Light Sci Appl. 2018, 7:81. [DOI: 10.1038/s41377-018-0082-1]
  17. Xiong H, Qian N, Zhao Z, Shi L, Miao Y, Min W: Background-free imaging of chemical bonds by a simple and robust frequency-modulated stimulated Raman scattering microscopy. Opt Express. 2020, 28:15663–77. [DOI: 10.1364/OE.391016]
  18. Xiong H, Qian N, Miao Y, Zhao Z, Chen C, Min W: Super-resolution vibrational microscopy by stimulated Raman excited fluorescence. Light Sci Appl. 2021, 10:87. [DOI: 10.1038/s41377-021-00518-5]
  19. Wei L, Hu F, Shen Y, et al.: Live-cell imaging of alkyne-tagged small biomolecules by stimulated Raman scattering. Nat Methods. 2014, 11:410–2. [DOI: 10.1038/nmeth.2878]
  20. Hu F, Lamprecht MR, Wei L, Morrison B, Min W: Bioorthogonal chemical imaging of metabolic activities in live mammalian hippocampal tissues with stimulated Raman scattering. Sci Rep. 2016, 6:39660. [DOI: 10.1038/srep39660]
  21. Hu F, Wei L, Zheng C, Shen Y, Min W: Live-cell vibrational imaging of choline metabolites by stimulated Raman scattering coupled with isotope-based metabolic labeling. Analyst. 2014, 139:2312–7. [DOI: 10.1039/C3AN02281A]
  22. Alfonso-García A, Pfisterer SG, Riezman H, Ikonen E, Potma EO: D38-cholesterol as a Raman active probe for imaging intracellular cholesterol storage. J Biomed Opt. 2016, 21:061003–061003. [DOI: 10.1117/1.JBO.21.6.061003]
  23. Liao C-S, Wang P, Huang CY, et al.: In Vivo and in Situ Spectroscopic Imaging by a Handheld Stimulated Raman Scattering Microscope. ACS Photonics. 2018, 5:947–54. https://doi.org/10.1021/acsphotonics.7b01214 [DOI: 10.1021/acsphotonics.7b01214]
  24. Orringer DA, Pandian B, Niknafs YS, et al.: Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat Biomed Eng. 2017, 1:0027. [DOI: 10.1038/s41551-016-0027]
  25. Yang Y, Yang Y, Liu Z, et al.: Microcalcification-Based Tumor Malignancy Evaluation in Fresh Breast Biopsies with Hyperspectral Stimulated Raman Scattering. Anal Chem. 2021, 93:6223–31. https://doi.org/10.1021/acs.analchem.1c00522 [DOI: 10.1021/acs.analchem.1c00522]
  26. Zhang B, Xu H, Chen J, et al.: Highly specific and label-free histological identification of microcrystals in fresh human gout tissues with stimulated Raman scattering. Theranostics. 2021, 11:3074. [DOI: 10.7150/thno.53755]
  27. Wei Z, Liu X, Yan R, Sun G, Yu W, Liu Q, Guo Q: Pixel-level multimodal fusion deep networks for predicting subcellular organelle localization from label-free live-cell imaging. Front Genet. 2022, 13:1002327. [DOI: 10.3389/fgene.2022.1002327]
  28. Wei Z, Liu W, Yu W, Liu X, Yan R, Liu Q, Guo Q: Multiple Parallel Fusion Network for Predicting Protein Subcellular Localization from Stimulated Raman Scattering (SRS) Microscopy Images in Living Cells. Int J Mol Sci. 2022, 23:10827. [DOI: 10.3390/ijms231810827]
  29. Sun G, Liu S, Shi C, Liu X, Guo Q: 3DCNAS: A universal method for predicting the location of fluorescent organelles in living cells in three-dimensional space. Exp Cell Res. 2023, 433:113807. [DOI: 10.1016/j.yexcr.2023.113807]
  30. Freudiger CW, Min W, Saar BG, et al.: Label-free biomedical imaging with high sensitivity by stimulated Raman scattering microscopy. Science. 2008, 322:1857–61. [DOI: 10.1126/science.1165758]
  31. Zhang J, Zhao J, Lin H, Tan Y, Cheng J-X: High-speed chemical imaging by dense-net learning of femtosecond stimulated Raman scattering. J Phys Chem Lett. 2020, 11:8573–8. [DOI: 10.1021/acs.jpclett.0c01598]
  32. Hu F, Shi L, Min W: Biological imaging of chemical bonds by stimulated Raman scattering microscopy. Nat Methods. 2019, 16:830–42. [DOI: 10.1038/s41592-019-0538-0]
  33. Zhang L, Wu Y, Zheng B, et al.: Rapid histology of laryngeal squamous cell carcinoma with deep-learning based stimulated Raman scattering microscopy. Theranostics. 2019, 9:2541. [DOI: 10.7150/thno.32655]
  34. Tian F, Yang W, Mordes DA, et al.: Monitoring peripheral nerve degeneration in ALS by label-free stimulated Raman scattering imaging. Nat Commun. 2016, 7:13283. [DOI: 10.1038/ncomms13283]
  35. Lu F-K, Basu S, Igras V, et al.: Label-free DNA imaging in vivo with stimulated Raman scattering microscopy. Proc Natl Acad Sci. 2015, 112:11624–9. [DOI: 10.1073/pnas.1515121112]
  36. Fu D, Zhou J, Zhu WS, et al.: Imaging the intracellular distribution of tyrosine kinase inhibitors in living cells with quantitative hyperspectral stimulated Raman scattering. Nat Chem. 2014, 6:614–22. [DOI: 10.1038/nchem.1961]
  37. Freudiger CW, Min W, Holtom GR, Xu B, Dantus M, Sunney Xie X: Highly specific label-free molecular imaging with spectrally tailored excitation-stimulated Raman scattering (STE-SRS) microscopy. Nat Photonics. 2011, 5:103–9. [DOI: 10.1038/nphoton.2010.294]
  38. Wang MC, Min W, Freudiger CW, Ruvkun G, Xie XS: RNAi screening for fat regulatory genes with SRS microscopy. Nat Methods. 2011, 8:135–8. [DOI: 10.1038/nmeth.1556]
  39. Ji M, Orringer DA, Freudiger CW, et al.: Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Sci Transl Med. 2013, 5:201ra119–201ra119.
  40. Hollon TC, Pandian B, Adapa AR, et al.: Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat Med. 2020, 26:52–8. [DOI: 10.1038/s41591-019-0715-9]
  41. Liu Z, Su W, Ao J, et al.: Instant diagnosis of gastroscopic biopsy via deep-learned single-shot femtosecond stimulated Raman histology. Nat Commun. 2022, 13:4050. https://doi.org/10.1038/s41467-022-31339-8 [DOI: 10.1038/s41467-022-31339-8]
  42. Cho K, Kim KD, Nam Y, et al.: CheSS: Chest X-Ray Pre-trained Model via Self-supervised Contrastive Learning. J Digit Imaging. 2023, 1–9.
  43. Liang X, Dai J, Zhou X, et al.: An Unsupervised Learning-Based Regional Deformable Model for Automated Multi-Organ Contour Propagation. J Digit Imaging. 2023, 1–9.
  44. Goyal P, Caron M, Lefaudeux B, et al.: Self-supervised pretraining of visual features in the wild. ArXiv Prepr ArXiv210301988. 2021.
  45. Holmberg OG, Köhler ND, Martins T, et al.: Self-supervised retinal thickness prediction enables deep learning from unlabelled data to boost classification of diabetic retinopathy. Nat Mach Intell. 2020, 2:719–26. [DOI: 10.1038/s42256-020-00247-1]
  46. Batson J, Royer L: Noise2self: Blind denoising by self-supervision. In: International Conference on Machine Learning. PMLR; 2019. 524–33.
  47. Jiang C, Chowdury A, Hou X, et al.: OpenSRH: optimizing brain tumor surgery using intraoperative stimulated Raman histology. Adv Neural Inf Process Syst. 2022, 35:28502–16. [PMID: 37082565]
  48. Zhang D, Slipchenko MN, Cheng J-X: Highly sensitive vibrational imaging by femtosecond pulse stimulated Raman loss. J Phys Chem Lett. 2011, 2:1248–53. [DOI: 10.1021/jz200516n]
  49. Cao R, Nelson SD, Davis S, et al.: Label-free intraoperative histology of bone tissue via deep-learning-assisted ultraviolet photoacoustic microscopy. Nat Biomed Eng. 2023, 7:124–34. [DOI: 10.1038/s41551-022-00940-z]
  50. Harms J, Lei Y, Wang T, et al.: Paired cycle-GAN-based image correction for quantitative cone-beam computed tomography. Med Phys. 2019, 46:3998–4009. [DOI: 10.1002/mp.13656]
  51. Wu H, Flierl M: Vector quantization-based regularization for autoencoders. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2020. 6380–7.
  52. Razavi A, Van den Oord A, Vinyals O: Generating diverse high-fidelity images with vq-vae-2. Adv Neural Inf Process Syst. 2019, 32.
  53. Van Den Oord A, Vinyals O: Neural discrete representation learning. Adv Neural Inf Process Syst. 2017, 30.
  54. McInnes L, Healy J, Melville J: Umap: Uniform manifold approximation and projection for dimension reduction. ArXiv Prepr ArXiv180203426. 2018.
  55. Hussain SF, Ramazan M: Biclustering of human cancer microarray data using co-similarity based co-clustering. Expert Syst Appl. 2016, 55:520–31. https://doi.org/10.1016/j.eswa.2016.02.029 [DOI: 10.1016/j.eswa.2016.02.029]
  56. Wu Z, Zhang Y, Zhang JZ, Xia K, Xia F: Determining Optimal Coarse-Grained Representation for Biomolecules Using Internal Cluster Validation Indexes. J Comput Chem. 2020, 41:14–20. https://doi.org/10.1002/jcc.26070 [DOI: 10.1002/jcc.26070]
  57. Zelig A, Kariti H, Kaplan N: KMD clustering: Robust general-purpose clustering of biological data. 2023, 2020.10.04.325233. https://doi.org/10.1101/2020.10.04.325233
  58. Cheng Y, Church GM: Biclustering of expression data. In: Ismb. 2000. 93–103.

MeSH Term

Humans
Brain Neoplasms
Deep Learning
Spectrum Analysis, Raman
Neural Networks, Computer
Supervised Machine Learning
Journal Article

Word Cloud

Created with Highcharts 10.0.0diagnosisreal-timeRamanVQSRSbraincancerSRHdeeplearningintraoperativehistologicaltissueapproachhistologyCNNpotentialfeatureimagingmethodspeedautomatedquantizationtumorimagesSRSsystemBrainStimulatedproceduresessentialensuringsafeeffectivecareprevailingworkflowreliesstaininghematoxylineosinH&Eprocessingresource-intensivetime-consumingrequiresconsiderablelaborRecentlyinnovativecombiningstimulatedconvolutionalneuralnetworksemergedcreatingnewavenuesurgeryexhibitsexistsopportunityrefinementdomainextractionstudyemploycoherentscatteringself-supervisedmodelVQVAE2enhanceimageacquisitionrepresentationtherebyenhancingcapabilitybedsideSpecificallyproposenetworkintegratesvectorproxytaskbasedpatchannotationanalysissubtypesTrainingcollectedmicroscopydemonstratessignificantenhancementtraditionaltechniqueseg20-30 minComparativestudiesdimensionalityreductionclusteringconfirmdiagnosticcapacityrivalshierarchicalstructurerecognizablefeaturesclassifiesmajorpathologicalcategoriestumorsAdditionallyexternalsemanticsegmentationappliedidentifyingtumor-infiltratedregionsCollectivelyfindingsindicatepredictiontechniqueholdsstreamlineprovidingassistancepathologistssimplifyingprocessFastReal-TimeTumorDetectionBasedHistologySelf-SupervisedDeepLearningModelsubtypeclassificationGenerativeIntraoperativeVector

Similar Articles

Cited By