Exploration biases forelimb reaching strategies.

Alice C Mosberger, Leslie J Sibener, Tiffany X Chen, Helio F M Rodrigues, Richard Hormigo, James N Ingram, Vivek R Athalye, Tanya Tabachnik, Daniel M Wolpert, James M Murray, Rui M Costa
Author Information
  1. Alice C Mosberger: Departments of Neuroscience and Neurology, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA. Electronic address: acm2246@columbia.edu.
  2. Leslie J Sibener: Departments of Neuroscience and Neurology, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  3. Tiffany X Chen: Departments of Neuroscience and Neurology, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  4. Helio F M Rodrigues: Departments of Neuroscience and Neurology, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Allen Institute, Seattle, WA 98109, USA.
  5. Richard Hormigo: Department of Neuroscience, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  6. James N Ingram: Department of Neuroscience, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  7. Vivek R Athalye: Departments of Neuroscience and Neurology, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  8. Tanya Tabachnik: Department of Neuroscience, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  9. Daniel M Wolpert: Department of Neuroscience, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
  10. James M Murray: Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA.
  11. Rui M Costa: Departments of Neuroscience and Neurology, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Allen Institute, Seattle, WA 98109, USA. Electronic address: rui.costa@alleninstitute.org.

Abstract

The brain can generate actions, such as reaching to a target, using different movement strategies. We investigate how such strategies are learned in a task where perched head-fixed mice learn to reach to an invisible target area from a set start position using a joystick. This can be achieved by learning to move in a specific direction or to a specific endpoint location. As mice learn to reach the target, they refine their variable joystick trajectories into controlled reaches, which depend on the sensorimotor cortex. We show that individual mice learned strategies biased to either direction- or endpoint-based movements. This endpoint/direction bias correlates with spatial directional variability with which the workspace was explored during training. Model-free reinforcement learning agents can generate both strategies with similar correlation between variability during training and learning bias. These results provide evidence that reinforcement of individual exploratory behavior during training biases the reaching strategies that mice learn.

Keywords

References

  1. J Neurosci Methods. 2009 Jun 30;181(1):18-26 [PMID: 19383512]
  2. J Neurophysiol. 2019 Apr 1;121(4):1561-1574 [PMID: 30811259]
  3. Sci Adv. 2022 Mar 11;8(10):eabj5167 [PMID: 35263129]
  4. Nat Rev Neurosci. 2004 Jul;5(7):532-46 [PMID: 15208695]
  5. J Neurophysiol. 1979 Jan;42(1 Pt 1):183-94 [PMID: 107279]
  6. Nat Neurosci. 2002 Nov;5(11):1226-35 [PMID: 12404008]
  7. J Neurosci. 2002 Nov 15;22(22):9656-60 [PMID: 12427820]
  8. Curr Opin Neurobiol. 2014 Apr;25:194-200 [PMID: 24566242]
  9. J Neurosci. 1982 Nov;2(11):1527-37 [PMID: 7143039]
  10. Science. 1986 Sep 26;233(4771):1416-9 [PMID: 3749885]
  11. Trends Cogn Sci. 1998 Sep 1;2(9):338-47 [PMID: 21227230]
  12. J Neurosci. 2000 Oct 15;20(20):7807-15 [PMID: 11027245]
  13. Nature. 2017 Apr 6;544(7648):96-100 [PMID: 28321129]
  14. J Integr Neurosci. 2011 Sep;10(3):353-84 [PMID: 21960307]
  15. J Neurosci. 2016 Apr 27;36(17):4832-45 [PMID: 27122039]
  16. PLoS Comput Biol. 2011 Mar;7(3):e1002012 [PMID: 21423711]
  17. Curr Opin Neurobiol. 2014 Apr;25:211-20 [PMID: 24632334]
  18. J Neurosci. 2022 Aug 10;42(32):6243-6257 [PMID: 35790405]
  19. Sci Adv. 2022 Feb 25;8(8):eabk0231 [PMID: 35213216]
  20. Sci Adv. 2019 Oct 30;5(10):eaay0001 [PMID: 31693007]
  21. J Exp Psychol Hum Percept Perform. 1998 Apr;24(2):569-91 [PMID: 9554097]
  22. Neuron. 2015 May 6;86(3):800-12 [PMID: 25892304]
  23. J Neurosci. 1990 Jul;10(7):2039-58 [PMID: 2376768]
  24. Front Behav Neurosci. 2014 Nov 18;8:398 [PMID: 25477795]
  25. J Vis Exp. 2013 Jun 09;(76): [PMID: 23770844]
  26. Neuron. 2022 Sep 7;110(17):2771-2789.e7 [PMID: 35870448]
  27. Neuropharmacology. 2000 Mar 3;39(5):788-805 [PMID: 10699445]
  28. J Neurosci Methods. 1984 May;11(1):47-60 [PMID: 6471907]
  29. J Neurosci. 1994 May;14(5 Pt 2):3208-24 [PMID: 8182467]
  30. Cell. 2020 Oct 29;183(3):605-619.e22 [PMID: 33031743]
  31. Exp Brain Res. 2007 Aug;181(2):359-65 [PMID: 17387461]
  32. Behav Brain Res. 2005 Jan 6;156(1):125-37 [PMID: 15474657]
  33. Adv Exp Med Biol. 2009;629:139-78 [PMID: 19227499]
  34. J Neurophysiol. 1989 Aug;62(2):595-608 [PMID: 2769350]
  35. J Mot Behav. 2013;45(4):351-60 [PMID: 23796130]
  36. PLoS One. 2012;7(6):e37749 [PMID: 22675490]
  37. Science. 1995 Sep 29;269(5232):1880-2 [PMID: 7569931]
  38. Cell. 2021 Jul 8;184(14):3731-3747.e21 [PMID: 34214470]
  39. Nat Neurosci. 2014 Feb;17(2):312-21 [PMID: 24413700]
  40. Nature. 2014 Jun 12;510(7504):263-7 [PMID: 24805237]
  41. Nat Methods. 2024 Jul;21(7):1316-1328 [PMID: 38918605]
  42. Q J Exp Psychol. 1972 Nov;24(4):448-53 [PMID: 4648981]
  43. Nature. 2007 Dec 20;450(7173):1240-4 [PMID: 18097411]
  44. Elife. 2015 Dec 02;4:e10774 [PMID: 26633811]
  45. Nat Rev Neurosci. 2008 Apr;9(4):292-303 [PMID: 18319728]
  46. Nature. 2013 Jul 18;499(7458):295-300 [PMID: 23868258]
  47. Behav Brain Res. 2015 Feb 1;278:569-76 [PMID: 25446755]
  48. J Neural Eng. 2005 Sep;2(3):S266-78 [PMID: 16135889]
  49. Elife. 2015 Sep 29;4:e09423 [PMID: 26417950]
  50. J Neurosci. 2015 Mar 4;35(9):4015-24 [PMID: 25740529]
  51. Nat Rev Neurosci. 2006 Dec;7(12):967-75 [PMID: 17115078]
  52. eNeuro. 2018 Aug 13;5(3): [PMID: 30105298]
  53. Nat Protoc. 2020 Mar;15(3):1237-1254 [PMID: 32034393]
  54. Nature. 2020 Jan;577(7790):386-391 [PMID: 31875851]
  55. J Neurophysiol. 1995 Jun;73(6):2563-7 [PMID: 7666162]
  56. Cell. 2019 Apr 18;177(3):669-682.e24 [PMID: 30929904]
  57. J Neurophysiol. 2019 Feb 1;121(2):500-512 [PMID: 30540551]
  58. J Neurosci. 2004 Jul 21;24(29):6515-20 [PMID: 15269262]
  59. J Neurosci. 1985 Jul;5(7):1688-703 [PMID: 4020415]
  60. J Neurosci. 2000 Dec 1;20(23):8916-24 [PMID: 11102502]
  61. Curr Biol. 2019 Nov 4;29(21):3551-3562.e7 [PMID: 31630947]
  62. J Neurophysiol. 2015 Oct;114(4):2187-93 [PMID: 26245313]
  63. Nature. 2011 Sep 28;478(7369):387-90 [PMID: 21964335]
  64. J Neurosci. 2009 Nov 4;29(44):13870-82 [PMID: 19889998]
  65. Brain. 1996 Aug;119 ( Pt 4):1183-98 [PMID: 8813282]
  66. Cell Rep. 2018 Mar 6;22(10):2767-2783 [PMID: 29514103]
  67. PLoS One. 2014 Feb 10;9(2):e88678 [PMID: 24520413]
  68. J Neurosci. 2012 Oct 24;32(43):14951-65 [PMID: 23100418]
  69. J Exp Psychol. 1946 Feb;36:13-24 [PMID: 21015338]
  70. Neuron. 2017 Mar 22;93(6):1493-1503.e6 [PMID: 28334611]
  71. Exp Brain Res. 1993;95(1):111-7 [PMID: 8405243]
  72. Annu Rev Neurosci. 2010;33:89-108 [PMID: 20367317]
  73. Front Psychol. 2019 Aug 14;10:1874 [PMID: 31474912]
  74. Nature. 1986 Apr 24-30;320(6064):748-50 [PMID: 3703000]
  75. Curr Opin Behav Sci. 2018 Apr;20:183-195 [PMID: 30035207]
  76. IEEE Int Conf Rehabil Robot. 2011;2011:5975349 [PMID: 22275553]
  77. Cell. 2015 Sep 10;162(6):1418-30 [PMID: 26359992]
  78. Behav Brain Res. 2013 Apr 15;243:267-72 [PMID: 23380676]
  79. J Neurophysiol. 2021 Nov 1;126(5):1604-1613 [PMID: 34525324]
  80. J Neurophysiol. 2006 Sep;96(3):1441-55 [PMID: 16723412]
  81. Science. 1898 Jun 17;7(181):818-24 [PMID: 17769765]
  82. Nat Methods. 2019 Jul;16(7):649-657 [PMID: 31209382]
  83. Hum Mov Sci. 2015 Dec;44:258-69 [PMID: 26415094]
  84. Nat Commun. 2023 Apr 12;14(1):1866 [PMID: 37045825]
  85. Acta Psychol (Amst). 1969;30:3-15 [PMID: 5811530]
  86. J Neurophysiol. 2012 Nov;108(9):2373-82 [PMID: 22896725]
  87. J Neurophysiol. 1989 Aug;62(2):582-94 [PMID: 2769349]
  88. J Exp Psychol. 1946 Jun;36:221-9 [PMID: 20985357]
  89. J Neurosci. 1984 Nov;4(11):2738-44 [PMID: 6502202]
  90. Annu Rev Neurosci. 2017 Jul 25;40:77-97 [PMID: 28375768]
  91. J Neurophysiol. 1998 Dec;80(6):3321-5 [PMID: 9862925]
  92. J Vis Exp. 2010 Jul 21;(41): [PMID: 20689506]
  93. J Neurophysiol. 1981 Oct;46(4):725-43 [PMID: 7288461]
  94. J Neurophysiol. 2014 Jul 15;112(2):411-29 [PMID: 24717350]
  95. J Neurophysiol. 2012 Jul;108(2):578-94 [PMID: 22514286]
  96. IEEE Trans Neural Syst Rehabil Eng. 2013 Sep;21(5):796-805 [PMID: 23335672]
  97. Front Neurol. 2013 Jul 22;4:101 [PMID: 23885254]
  98. Annu Rev Neurosci. 2017 Jul 25;40:479-498 [PMID: 28489490]
  99. Sci Rep. 2017 Nov 17;7(1):15759 [PMID: 29150620]

Grants

  1. K99 NS128250/NINDS NIH HHS
  2. U19 NS104649/NINDS NIH HHS
  3. K99 NS126307/NINDS NIH HHS
  4. F31 NS111853/NINDS NIH HHS
  5. R00 NS114194/NINDS NIH HHS
  6. F32 MH118714/NIMH NIH HHS

MeSH Term

Animals
Forelimb
Mice
Exploratory Behavior
Mice, Inbred C57BL
Learning
Male
Movement
Reinforcement, Psychology
Female
Behavior, Animal

Word Cloud

Created with Highcharts 10.0.0strategiesmicelearningcanreachingtargetlearntraininggenerateusinglearnedhead-fixedreachjoystickspecificindividualbiasvariabilityreinforcementbehaviorbiasesmotorbrainactionsdifferentmovementinvestigatetaskperchedinvisibleareasetstartpositionachievedmovedirectionendpointlocationrefinevariabletrajectoriescontrolledreachesdependsensorimotorcortexshowbiasedeitherdirection-endpoint-basedmovementsendpoint/directioncorrelatesspatialdirectionalworkspaceexploredModel-freeagentssimilarcorrelationresultsprovideevidenceexploratoryExplorationforelimbCP:Neuroscienceexplorationcontrol

Similar Articles

Cited By