OpenLB
Open Library of Bioscience
Advanced Search
Current medical science
Mar 18, 2025;

Construction of a Multimodal 3D Atlas for a Micrometer-Scale Brain-Computer Interface Based on Mixed Reality.

Hong Zhou, Zi-Neng Yan, Wei-Hang Gao, Xiang-Xin Lv, Rui Luo, Jason Shih Hoellwarth, Lei He, Jia-Ming Yang, Jia-Yao Zhang, Hong-Lin Wang, Yi Xie, Xiao-Liang Chen, Ming-di Xue, Ying Fang, Yu-Yu Duan, Rui-Yuan Li, Xu-Dong Wang, Rui-Lin Wang, Mao Xie, Li Huang, Peng-Ran Liu, Zhe-Wei Ye
Author Information
  1. Hong Zhou: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  2. Zi-Neng Yan: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  3. Wei-Hang Gao: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  4. Xiang-Xin Lv: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  5. Rui Luo: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  6. Jason Shih Hoellwarth: Osseointegration Limb Replacement Center, Limb Lengthening and Complex Reconstruction Service, Hospital for Special Surgery, New York, 10021, USA.
  7. Lei He: University of Chicago, Chicago, 60637, USA.
  8. Jia-Ming Yang: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  9. Jia-Yao Zhang: Laboratory of Intelligent Medicine, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  10. Hong-Lin Wang: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  11. Yi Xie: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  12. Xiao-Liang Chen: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  13. Ming-di Xue: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  14. Ying Fang: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  15. Yu-Yu Duan: Laboratory of Intelligent Medicine, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  16. Rui-Yuan Li: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  17. Xu-Dong Wang: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  18. Rui-Lin Wang: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  19. Mao Xie: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China.
  20. Li Huang: Wuhan Neuracom Technology Development Co., Ltd, Wuhan, 430200, China. 13907130486@163.com.
  21. Peng-Ran Liu: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China. lprlprlprwd@163.com.
  22. Zhe-Wei Ye: Department of Orthopedics Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China. yezhewei@hust.edu.cn. ORCID
PMID: 40100553 DOI: 10.1007/s11596-025-00033-3

Abstract

OBJECTIVE: To develop a multimodal imaging atlas of a rat brain-computer interface (BCI) that incorporates brain, arterial, bone tissue and a BCI device using mixed reality (MR) for three-dimensional (3D) visualization.
METHODS: An invasive BCI was implanted in the left visual cortex of 4-week-old Sprague-Dawley rats. Multimodal imaging techniques, including micro-CT and 9.0 T MRI, were used to acquire images of the rat cranial bone structure, vascular distribution, brain tissue functional zones, and BCI device before and after implantation. Using 3D-slicer software, the images were fused through spatial transformations, followed by image segmentation and 3D model reconstruction. The HoloLens platform was employed for MR visualization.
RESULTS: This study constructed a multimodal imaging atlas for rats that included the skull, brain tissue, arterial tissue, and BCI device coupled with MR technology to create an interactive 3D anatomical model.
CONCLUSIONS: This multimodal 3D atlas provides an objective and stable reference for exploring complex relationships between brain tissue structure and function, enhancing the understanding of the operational principles of BCIs. This is the first multimodal 3D imaging atlas related to a BCI created using Sprague-Dawley rats.

Keywords

Brain-computer interface Mixed reality Multimodal imaging Three-dimensional atlas

References

  1. Chang EF. Brain–Computer Interfaces for Restoring Communication. N Engl J Med. 2024;391(7):654–657. [DOI: 10.1056/NEJMe2407363]
  2. Willett FR, Avansino DT, Hochberg LR, et al. High-performance brain-to-text communication via handwriting. Nature. 2021;593(7858):249–254. [DOI: 10.1038/s41586-021-03506-2]
  3. Liu Y, Liu R, Ge J, et al. Advancements in brain-machine interfaces for application in the metaverse. Front Neurosci. 2024;18:1383319. [DOI: 10.3389/fnins.2024.1383319]
  4. Abbasi J. Advanced brain–computer interface for people with paralysis. JAMA. 2019;321(6):537. [PMID: 30747952]
  5. Orban M, Elsamanty M, Guo K, et al. A Review of Brain Activity and EEG-Based Brain–Computer Interfaces for Rehabilitation Application. Bioengineering. 2022;9(12):768. [DOI: 10.3390/bioengineering9120768]
  6. Wu X, Metcalfe B, He S, et al. A Review of Motor Brain-Computer Interfaces Using Intracranial Electroencephalography Based on Surface Electrodes and Depth Electrodes. IEEE Trans Neural Syst Rehabil Eng. 2024;32:2408–2431. [DOI: 10.1109/TNSRE.2024.3421551]
  7. Rouzitalab A, Boulay CB, Park J, et al. Intracortical brain-computer interfaces in primates: a review and outlook. Biomed Eng Lett. 2023;13(3):375–390. [DOI: 10.1007/s13534-023-00286-8]
  8. Ma Y, Gong A, Nan W, et al. Personalized Brain–Computer Interface and Its Applications. J Pers Med. 2022;13(1):46. [DOI: 10.3390/jpm13010046]
  9. Klein F. Optimizing spatial specificity and signal quality in fNIRS: an overview of potential challenges and possible options for improving the reliability of real-time applications. Front Neuroergonomics. 2024;5:1286586. [DOI: 10.3389/fnrgo.2024.1286586]
  10. Yu L, Liu Q, Bai X, et al. A digital rat atlas of sectional anatomy. In: Xu K, Luo Q, Xing D et al., editors. SPIE Proceedings. Tianjin, China; 2006. p. 60470Y.
  11. Liao Y, Yu L, Bai X, et al. 3D model construction for arterial vessels system of digital rat. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi (Chinese). 2008;25(3):552–555. [PMID: 18693429]
  12. Perens J, Salinas CG, Roostalu U, et al. Multimodal 3D Mouse Brain Atlas Framework with the Skull-Derived Coordinate System. Neuroinform. 2023;21(2):269–286. [DOI: 10.1007/s12021-023-09623-9]
  13. Huang S, Lu Z, Huang W, et al. High-resolution in vivo Wistar rodent brain atlas based on T1 weighted image. In Proceedings of SPIE Photonics Asia 2016; Chengdu, China; 2016 Sep 26–30. Berlingham, WA: SPIE; 2016. p. 978825.
  14. Bai X, Yu L, Liu Q, et al. A high‐resolution anatomical rat atlas. J Anat. 2006;209(5):707–708. [DOI: 10.1111/j.1469-7580.2006.00645.x]
  15. Nie B, Chen K, Zhao S, et al. A rat brain MRI template with digital stereotaxic atlas of fine anatomical delineations in paxinos space and its automated application in voxel‐wise analysis. Human Brain Mapping. 2013;34(6):1306–1318. [DOI: 10.1002/hbm.21511]
  16. Sadeghi M, Ramos-Prats A, Neto P, et al. Localization and Registration of 2D Histological Mouse Brain Images in 3D Atlas Space. Neuroinform. 2023;21(3):615–630. [DOI: 10.1007/s12021-023-09632-8]
  17. Naito S, Kajiwara M, Nakashima R, et al. Application of Extended Reality (Virtual Reality and Mixed Reality) Technology in Laparoscopic Liver Resections. Cureus. 2023;15(9):e44520. [PMID: 37790042]
  18. Chytas D, Nikolaou VS. Mixed reality for visualization of orthopedic surgical anatomy. World J Orthop. 2021;12(10):727–731. [DOI: 10.5312/wjo.v12.i10.727]
  19. Qi Z, Jin H, Xu X, et al. Head model dataset for mixed reality navigation in neurosurgical interventions for intracranial lesions. Sci Data. 2024;11(1):538. [DOI: 10.1038/s41597-024-03385-y]
  20. Jeising S, Liu S, Blaszczyk T, et al. Combined use of 3D printing and mixed reality technology for neurosurgical training: getting ready for brain surgery. Neurosurg Focus. 2024;56(1):E12. [DOI: 10.3171/2023.10.FOCUS23611]
  21. Bartella AK, Kamal M, Kuhnt T, et al. Mixed reality in oral and maxillofacial surgery: a symbiosis of virtual and augmented reality or a pointless technological gadget? Int J Comput Dent. 2021;24(1):65–76. [PMID: 34006064]
  22. Jain S, Timofeev I, Kirollos RW, et al. Use of Mixed Reality in Neurosurgery Training: A Single Centre Experience. World Neurosurg. 2023;176:e68–e76. [DOI: 10.1016/j.wneu.2023.04.107]
  23. Andrews A. Integration of augmented reality and brain-computer interface technologies for health care applications: exploratory and prototyping study. JMIR Form Res. 2022;6(4):e18222.
  24. Schoch CL, Ciufo S, Domrachev M, et al. NCBI Taxonomy: a comprehensive update on curation, resources and tools. Database (Oxford). 2020;2020:baaa062. [DOI: 10.1093/database/baaa062]
  25. Zhu Y, Xu K, Xu C, et al. PET Mapping for Brain–Computer Interface Stimulation of the Ventroposterior Medial Nucleus of the Thalamus in Rats with Implanted Electrodes. J Nucl Med. 2016;57(7):1141–1145. [DOI: 10.2967/jnumed.115.171868]
  26. Charvet G, Billoint O, Gharbi S, et al. A modular 256-channel Micro Electrode Array platform for in vitro and in vivo neural stimulation and recording: BioMEA. Annu Int Conf IEEE Eng Med Biol Soc. 2010;2010:1804–1807. [PMID: 21095937]
  27. Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic Resonance Imaging. 2012;30(9):1323–1341. [DOI: 10.1016/j.mri.2012.05.001]
  28. Kleven H, Bjerke IE, Clascá F, et al. Waxholm Space atlas of the rat brain: a 3D atlas supporting data analysis and integration. Nat Methods. 2023;20(11):1822–1829. [DOI: 10.1038/s41592-023-02034-3]
  29. Papp EA, Leergaard TB, Calabrese E, et al. Waxholm Space atlas of the Sprague Dawley rat brain. NeuroImage. 2014;97:374–386. [DOI: 10.1016/j.neuroimage.2014.04.001]
  30. Liu S, Xie M, Zhang Z, et al. A 3D Hologram With Mixed Reality Techniques to Improve Understanding of Pulmonary Lesions Caused by COVID-19: Randomized Controlled Trial. J Med Internet Res. 2021;23(9):e24081. [DOI: 10.2196/24081]
  31. Lu L, Wang H, Liu P, et al. Applications of Mixed Reality Technology in Orthopedics Surgery: A Pilot Study. Front Bioeng Biotechnol. 2022;10:740507. [DOI: 10.3389/fbioe.2022.740507]
  32. Wang Q, Ding SL, Li Y, et al. The Allen Mouse Brain Common Coordinate Framework: A 3D Reference Atlas. Cell. 2020;181(4):936-953.e20. [DOI: 10.1016/j.cell.2020.04.007]
  33. Chen J, Xia Y, Zhou X, et al. fNIRS-EEG BCIs for Motor Rehabilitation: A Review. Bioengineering. 2023;10(12):1393. [DOI: 10.3390/bioengineering10121393]
  34. Chai X, Cao T, He Q, et al. Brain–computer interface digital prescription for neurological disorders. CNS Neurosci Ther. 2024;30(2):e14615. [DOI: 10.1111/cns.14615]
  35. De Miguel-Rubio A, Gallego-Aguayo I, De Miguel-Rubio MD, et al. Effectiveness of the Combined Use of a Brain–Machine Interface System and Virtual Reality as a Therapeutic Approach in Patients with Spinal Cord Injury: A Systematic Review. Healthcare. 2023;11(24):3189. [DOI: 10.3390/healthcare11243189]
  36. Lin S, Jiang J, Huang K, et al. Advanced Electrode Technologies for Noninvasive Brain–Computer Interfaces. ACS Nano. 2023;17(24):24487–24513. [DOI: 10.1021/acsnano.3c06781]
  37. Hawrylycz M, Baldock RA, Burger A, et al. Digital Atlasing and Standardization in the Mouse Brain. PLoS Comput Biol. 2011;7(2):e1001065. [DOI: 10.1371/journal.pcbi.1001065]
  38. Shcherbatyy V, Carson J, Yaylaoglu M, et al. A Digital Atlas of Ion Channel Expression Patterns in the Two-Week-Old Rat Brain. Neuroinform. 2015;13(1):111–125. [DOI: 10.1007/s12021-014-9247-0]
  39. Zhang H, Jiao L, Yang S, et al. Brain–computer interfaces: the innovative key to unlocking neurological conditions. Int J Surg. 2024;110(9):5745–5762. [DOI: 10.1097/JS9.0000000000002022]
  40. Pelanis E, Kumar RP, Aghayan DL, et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol. 2020;29(3):154–160. [DOI: 10.1080/13645706.2019.1616558]
  41. Pasquali M, Fusini L, Italiano G, et al. Feasibility study of a mixed reality tool for real 3D visualization and planning of left atrial appendage occlusion. J Cardiovasc Comput Tomog. 2022;16(5):460–462. [DOI: 10.1016/j.jcct.2022.02.010]

Grants

  1. No.82172524/the National Natural Science Foundation of China
  2. No.81974355/the National Natural Science Foundation of China
  3. No.2020021105012440/National Innovation Platform Development Program
  4. 2021BEA161/Major Program of Hubei Province
  5. JD2023BAA005/Major Key Project of Hubei Province
  6. No.2024XHYN047/Wuhan Union Hospital Free Innovation Preliminary Research Fund
Journal Article
Article has an altmetric score of 14

Word Cloud

Created with Highcharts 10.0.0BCIatlastissue3DmultimodalimagingbraindeviceMRratsratinterfacearterialboneusingrealityvisualizationSprague-DawleyMultimodalimagesstructuremodelMixedOBJECTIVE:developbrain-computerincorporatesmixedfor three-dimensional METHODS:invasiveimplantedleftvisualcortex4-week-oldtechniquesincludingmicro-CT90 TMRIusedacquirecranialvasculardistributionfunctionalzonesimplantationUsing3D-slicersoftwarefusedspatialtransformationsfollowedimagesegmentationreconstructionHoloLensplatformemployedRESULTS:studyconstructedincludedskullcoupledtechnologycreateinteractiveanatomicalCONCLUSIONS:providesobjectivestablereferenceexploringcomplexrelationshipsfunctionenhancingunderstandingoperationalprinciplesBCIsfirstrelatedcreatedConstruction3D AtlasMicrometer-ScaleBrain-ComputerInterfaceBasedRealityBrain-computerMultimodal imagingThree-dimensional

Similar Articles

Cited By