Robot motor learning shows emergence of frequency-modulated, robust swimming with an invariant Strouhal number.

Hankun Deng, Donghao Li, Colin Nitroy, Andrew Wertz, Shashank Priya, Bo Cheng
Author Information
  1. Hankun Deng: Department of Mechanical Engineering, The Pennsylvania State University, University Park, PA 16802, USA. ORCID
  2. Donghao Li: Department of Mechanical Engineering, The Pennsylvania State University, University Park, PA 16802, USA.
  3. Colin Nitroy: Department of Mechanical Engineering, The Pennsylvania State University, University Park, PA 16802, USA.
  4. Andrew Wertz: Department of Mechanical Engineering, The Pennsylvania State University, University Park, PA 16802, USA.
  5. Shashank Priya: Department of Material Science and Engineering, The Pennsylvania State University, University Park, PA 16802, USA. ORCID
  6. Bo Cheng: Department of Mechanical Engineering, The Pennsylvania State University, University Park, PA 16802, USA. ORCID

Abstract

Fish locomotion emerges from diverse interactions among deformable structures, surrounding fluids and neuromuscular activations, i.e. fluid-structure interactions (FSI) controlled by fish's motor systems. Previous studies suggested that such motor-controlled FSI may possess embodied traits. However, their implications in motor learning, neuromuscular control, gait generation, and swimming performance remain to be uncovered. Using robot models, we studied the embodied traits in fish-inspired swimming. We developed modular robots with various designs and used central pattern generators (CPGs) to control the torque acting on robot body. We used reinforcement learning to learn CPG parameters for maximizing the swimming speed. The results showed that motor frequency converged faster than other parameters, and the emergent swimming gaits were robust against disruptions applied to motor control. For all robots and frequencies tested, swimming speed was proportional to the mean undulation velocity of body and caudal-fin combined, yielding an invariant, undulation-based Strouhal number. The Strouhal number also revealed two fundamental classes of undulatory swimming in both biological and robotic fishes. The robot actuators were also demonstrated to function as motors, virtual springs and virtual masses. These results provide novel insights in understanding fish-inspired locomotion.

Keywords

Associated Data

figshare | 10.6084/m9.figshare.c.7125496

References

  1. J Exp Biol. 1999 Jun;202 (Pt 11):1511-21 [PMID: 10229697]
  2. Biol Cybern. 1985;52(6):367-76 [PMID: 2996634]
  3. J Neurophysiol. 2018 Aug 1;120(2):765-780 [PMID: 29668379]
  4. J Exp Biol. 2018 Mar 29;221(Pt 7): [PMID: 29599417]
  5. Trends Neurosci. 1997 Dec;20(12):553-7 [PMID: 9416664]
  6. J Exp Biol. 2007 Dec;210(Pt 24):4265-71 [PMID: 18055615]
  7. Science. 2014 Oct 10;346(6206):196-203 [PMID: 25301621]
  8. J Exp Biol. 2003 Apr;206(Pt 7):1117-26 [PMID: 12604571]
  9. J Exp Biol. 1994 Jul;192(1):45-59 [PMID: 9317308]
  10. Bioinspir Biomim. 2014 Mar;9(1):016010 [PMID: 24451164]
  11. Sci Rep. 2023 Sep 12;13(1):15032 [PMID: 37699939]
  12. Zoology (Jena). 2011 Apr;114(2):78-84 [PMID: 21392954]
  13. Soft Robot. 2017 Sep;4(3):202-210 [PMID: 29182079]
  14. J Exp Biol. 1997;200(Pt 13):1863-71 [PMID: 9319776]
  15. J Exp Biol. 1998 Oct;201 (Pt 19):2723-33 [PMID: 9732327]
  16. J Exp Biol. 2004 Apr;207(Pt 10):1633-42 [PMID: 15073196]
  17. J Exp Biol. 1997;200(Pt 4):767-84 [PMID: 9318535]
  18. Sci Adv. 2021 May 5;7(19): [PMID: 33952525]
  19. Science. 2000 Apr 7;288(5463):100-6 [PMID: 10753108]
  20. Rep Prog Phys. 2016 Nov;79(11):110001 [PMID: 27652614]
  21. J Fish Biol. 2012 Jun;80(7):2494-516 [PMID: 22650430]
  22. J Exp Biol. 2000 Oct;203(Pt 20):3089-101 [PMID: 11003820]
  23. Nature. 2003 Oct 16;425(6959):707-11 [PMID: 14562101]
  24. J R Soc Interface. 2014 Jan 15;11(92):20131073 [PMID: 24430125]
  25. Proc Natl Acad Sci U S A. 2015 Mar 31;112(13):3874-9 [PMID: 25770221]
  26. Neural Comput. 2001 Nov;13(11):2409-63 [PMID: 11674845]
  27. Bioinspir Biomim. 2022 May 24;17(4): [PMID: 35487201]
  28. Physiol Rev. 2020 Jan 1;100(1):271-320 [PMID: 31512990]
  29. Phys Rev Lett. 2021 Jun 11;126(23):234501 [PMID: 34170168]
  30. Science. 1999 Apr 2;284(5411):87-9 [PMID: 10102823]
  31. J Exp Biol. 2002 Sep;205(Pt 18):2875-84 [PMID: 12177151]
  32. Bioinspir Biomim. 2024 Mar 14;19(3): [PMID: 38430560]
  33. Proc Natl Acad Sci U S A. 2021 Dec 7;118(49): [PMID: 34853171]
  34. J R Soc Interface. 2024 Mar;21(212):20240036 [PMID: 38531411]
  35. J Exp Biol. 2003 Aug;206(Pt 16):2749-58 [PMID: 12847120]
  36. J Exp Biol. 1995;198(Pt 2):585-602 [PMID: 9318295]
  37. Am J Physiol. 1994 Aug;267(2 Pt 2):R481-8 [PMID: 8067458]
  38. Nat Commun. 2023 Sep 9;14(1):5569 [PMID: 37689714]
  39. Sci Robot. 2019 Sep 18;4(34): [PMID: 33137777]
  40. Bioinspir Biomim. 2021 Mar 05;16(2): [PMID: 32927442]
  41. Integr Comp Biol. 2022 May 19;: [PMID: 35588062]
  42. Proc Natl Acad Sci U S A. 2010 Nov 16;107(46):19832-7 [PMID: 21037110]
  43. Sci Robot. 2021 Aug 11;6(57): [PMID: 34380756]

MeSH Term

Animals
Swimming
Robotics
Biomechanical Phenomena
Fishes
Locomotion

Word Cloud

Created with Highcharts 10.0.0swimmingmotorStrouhalnumberlearningcontrolrobotlocomotioninteractionsneuromuscularFSIembodiedtraitsfish-inspiredrobotsusedbodyparametersspeedresultsrobustinvariantalsoundulatoryvirtualFishemergesdiverseamongdeformablestructuressurroundingfluidsactivationsiefluid-structurecontrolledfish'ssystemsPreviousstudiessuggestedmotor-controlledmaypossessHoweverimplicationsgaitgenerationperformanceremainuncoveredUsingmodelsstudieddevelopedmodularvariousdesignscentralpatterngeneratorsCPGstorqueactingreinforcementlearnCPGmaximizingshowedfrequencyconvergedfasteremergentgaitsdisruptionsappliedfrequenciestestedproportionalmeanundulationvelocitycaudal-fincombinedyieldingundulation-basedrevealedtwofundamentalclassesbiologicalroboticfishesactuatorsdemonstratedfunctionmotorsspringsmassesprovidenovelinsightsunderstandingRobotshowsemergencefrequency-modulatedbioinspiredroboticsfish

Similar Articles

Cited By