OpenLB
Open Library of Bioscience
Advanced Search
Nature communications
05 13, 2021;12 (1): 2784.

A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping.

Kang Huang, Yaning Han, Ke Chen, Hongli Pan, Gaoyang Zhao, Wenling Yi, Xiaoxi Li, Siyuan Liu, Pengfei Wei, Liping Wang
Author Information
  1. Kang Huang: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. ORCID
  2. Yaning Han: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
  3. Ke Chen: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
  4. Hongli Pan: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
  5. Gaoyang Zhao: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. ORCID
  6. Wenling Yi: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
  7. Xiaoxi Li: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
  8. Siyuan Liu: Pennsylvania State University, University Park, PA, USA.
  9. Pengfei Wei: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. pf.wei@siat.ac.cn. ORCID
  10. Liping Wang: Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. lp.wang@siat.ac.cn. ORCID
PMID: 33986265 DOI: 10.1038/s41467-021-22970-y

Abstract

Animal behavior usually has a hierarchical structure and dynamics. Therefore, to understand how the neural system coordinates with behaviors, neuroscientists need a quantitative description of the hierarchical dynamics of different behaviors. However, the recent end-to-end machine-learning-based methods for behavior analysis mostly focus on recognizing behavioral identities on a static timescale or based on limited observations. These approaches usually lose rich dynamic information on cross-scale behaviors. Here, inspired by the natural structure of animal behaviors, we address this challenge by proposing a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behavior into the feature space. In addition, we characterize the animal 3D kinematics with our low-cost and efficient multi-view 3D animal motion-capture system. Finally, we demonstrate that this framework can monitor spontaneous behavior and automatically identify the behavioral phenotypes of the transgenic animal disease model. The extensive experiment results suggest that our framework has a wide range of applications, including animal disease model phenotyping and the relationships modeling between the neural circuits and behavior.

References

  1. Science. 2020 Jan 24;367(6476):440-445 [PMID: 31974254]
  2. Neuron. 2015 Dec 16;88(6):1121-1135 [PMID: 26687221]
  3. Cell Syst. 2015 Nov 25;1(5):338-348 [PMID: 26640823]
  4. Nat Methods. 2019 Jan;16(1):117-125 [PMID: 30573820]
  5. Elife. 2018 Jun 26;7: [PMID: 29943731]
  6. Nat Neurosci. 2018 Sep;21(9):1281-1289 [PMID: 30127430]
  7. Cell. 2018 Jun 28;174(1):44-58.e17 [PMID: 29779950]
  8. Nat Neurosci. 2014 Nov;17(11):1455-62 [PMID: 25349912]
  9. BMC Biol. 2018 Feb 23;16(1):23 [PMID: 29475451]
  10. Nat Protoc. 2019 Jul;14(7):2152-2176 [PMID: 31227823]
  11. J Biol Phys. 2016 Mar;42(2):259-70 [PMID: 26782966]
  12. Front Behav Neurosci. 2023 May 30;17:1111908 [PMID: 37324523]
  13. Elife. 2019 Oct 04;8: [PMID: 31584428]
  14. Nat Methods. 2013 Sep;10(9):877-9 [PMID: 23852451]
  15. Neuron. 2017 Jun 7;94(5):978-984 [PMID: 28595054]
  16. Curr Opin Neurobiol. 2020 Oct;64:60-69 [PMID: 32203874]
  17. Science. 2014 Apr 25;344(6182):386-92 [PMID: 24674869]
  18. J Neurosci Methods. 2018 Apr 15;300:37-47 [PMID: 28456660]
  19. Nat Commun. 2020 Sep 11;11(1):4560 [PMID: 32917899]
  20. Nat Chem Biol. 2016 Jul;12(7):559-66 [PMID: 27239787]
  21. Commun Biol. 2022 Nov 18;5(1):1267 [PMID: 36400882]
  22. Nat Commun. 2015 Apr 09;6:6756 [PMID: 25854147]
  23. Front Behav Neurosci. 2017 Jul 28;11:141 [PMID: 28804452]
  24. J R Soc Interface. 2014 Oct 6;11(99): [PMID: 25142523]
  25. PLoS One. 2013 Oct 30;8(10):e78460 [PMID: 24205238]
  26. Proc Natl Acad Sci U S A. 2013 Jan 8;110(2):791-6 [PMID: 23267063]
  27. Nat Methods. 2009 Jun;6(6):451-7 [PMID: 19412169]
  28. Nature. 2011 Apr 28;472(7344):437-42 [PMID: 21423165]
  29. R J. 2016 Aug;8(1):289-317 [PMID: 27818791]
  30. Neuron. 2020 Feb 5;105(3):402-404 [PMID: 32027829]
  31. Neuropsychopharmacology. 2021 Jan;46(1):33-44 [PMID: 32599604]
  32. Cell Rep. 2019 Nov 12;29(7):2016-2027.e4 [PMID: 31722214]
  33. Neuron. 2014 Oct 1;84(1):18-31 [PMID: 25277452]
  34. J Neurosci Methods. 2019 Oct 1;326:108352 [PMID: 31415845]
  35. Nature. 2016 Feb 25;530(7591):481-4 [PMID: 26886798]
  36. J Exp Biol. 2017 Jan 1;220(Pt 1):25-34 [PMID: 28057825]
  37. Nat Commun. 2021 May 13;12(1):2784 [PMID: 33986265]
  38. Front Cell Neurosci. 2018 Oct 22;12:374 [PMID: 30405356]
  39. Cell. 2019 Aug 8;178(4):867-886.e24 [PMID: 31398341]
  40. PLoS One. 2013 Sep 18;8(9):e74536 [PMID: 24058586]
  41. Behav Res Methods Instrum Comput. 2000 Feb;32(1):63-71 [PMID: 10758665]
  42. Curr Biol. 2020 Jan 6;30(1):70-82.e4 [PMID: 31866367]
  43. Proc Natl Acad Sci U S A. 2016 Oct 18;113(42):11943-11948 [PMID: 27702892]
  44. Elife. 2019 Oct 01;8: [PMID: 31570119]
  45. Curr Biol. 2018 Jan 22;28(2):181-195.e5 [PMID: 29307558]
  46. BMC Biol. 2019 May 29;17(1):44 [PMID: 31142307]
  47. Mol Autism. 2019 Dec 24;10:50 [PMID: 31879555]
  48. Nat Protoc. 2018 Jun;13(6):1331-1347 [PMID: 29773907]
  49. Nature. 2019 Jul;571(7766):456-457 [PMID: 31337906]
  50. Physiol Behav. 2014 Aug;135:55-71 [PMID: 24910138]
  51. Nat Commun. 2010 Sep 07;1:68 [PMID: 20842193]
  52. Nat Commun. 2019 Nov 28;10(1):5416 [PMID: 31780648]
  53. Trends Cogn Sci. 2019 Apr;23(4):334-348 [PMID: 30852123]
  54. PLoS One. 2019 Aug 2;14(8):e0220751 [PMID: 31374097]
  55. Neuron. 2019 Oct 9;104(1):11-24 [PMID: 31600508]
  56. Natl Sci Rev. 2019 May;6(3):402-420 [PMID: 34691891]
  57. Annu Rev Anim Biosci. 2017 Feb 8;5:371-389 [PMID: 28199172]
  58. Neuron. 2020 Feb 5;105(3):562-576.e9 [PMID: 31786012]
  59. Front Bioeng Biotechnol. 2018 May 16;6:61 [PMID: 29868576]
  60. Hum Mol Genet. 2011 Aug 1;20(15):3093-108 [PMID: 21558424]
  61. Nat Methods. 2013 Jan;10(1):64-7 [PMID: 23202433]
  62. IEEE Trans Pattern Anal Mach Intell. 2013 Mar;35(3):582-96 [PMID: 22732658]

MeSH Term

Animals
Behavior, Animal
Biomechanical Phenomena
Machine Learning
Mice
Mice, Inbred C57BL
Mice, Knockout
Microfilament Proteins
Movement
Nerve Tissue Proteins
Video Recording

Chemicals

Microfilament Proteins
Nerve Tissue Proteins
Shank3 protein, mouse
Journal Article Research Support, Non-U.S. Gov't
Article has an altmetric score of 21

Word Cloud

Created with Highcharts 10.0.0behavioranimalhierarchicalbehaviorsframeworkdynamicsusuallystructureneuralsystembehavioral3DspontaneousdiseasemodelAnimalThereforeunderstandcoordinatesneuroscientistsneedquantitativedescriptiondifferentHoweverrecentend-to-endmachine-learning-basedmethodsanalysismostlyfocusrecognizingidentitiesstatictimescalebasedlimitedobservationsapproachesloserichdynamicinformationcross-scaleinspirednaturaladdresschallengeproposingparallelmulti-layeredlearngenerateobjectivemetricmapfeaturespaceadditioncharacterizekinematicslow-costefficientmulti-viewmotion-captureFinallydemonstratecanmonitorautomaticallyidentifyphenotypestransgenicextensiveexperimentresultssuggestwiderangeapplicationsincludingphenotypingrelationshipsmodelingcircuits3D-motionlearningmapping

Similar Articles

Cited By