Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future.

Wei Li, Ping Shi, Hongliu Yu
Author Information
  1. Wei Li: Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China.
  2. Ping Shi: Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China.
  3. Hongliu Yu: Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China.

Abstract

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

Keywords

References

  1. J Neuroeng Rehabil. 2014 May 30;11:91 [PMID: 24886664]
  2. Sensors (Basel). 2020 Jul 17;20(14): [PMID: 32709164]
  3. IEEE Trans Biomed Eng. 2011 Jun;58(6):1698-705 [PMID: 21317073]
  4. J Neural Eng. 2020 Jul 13;17(4):046004 [PMID: 32521522]
  5. IEEE Trans Biomed Eng. 2013 Jun;60(6):1563-70 [PMID: 23322756]
  6. J Neuroeng Rehabil. 2018 Mar 13;15(1):21 [PMID: 29534764]
  7. Annu Int Conf IEEE Eng Med Biol Soc. 2013;2013:3622-5 [PMID: 24110514]
  8. Sensors (Basel). 2020 Jun 29;20(13): [PMID: 32610658]
  9. IEEE Trans Biomed Eng. 2020 Jun;67(6):1707-1717 [PMID: 31545709]
  10. Front Neurorobot. 2016 Sep 07;10:9 [PMID: 27656140]
  11. Front Neurorobot. 2017 Feb 14;11:7 [PMID: 28261085]
  12. IEEE Trans Neural Syst Rehabil Eng. 2014 Mar;22(2):269-79 [PMID: 24608685]
  13. Biosensors (Basel). 2020 Jul 26;10(8): [PMID: 32722542]
  14. IEEE Trans Biomed Circuits Syst. 2020 Apr;14(2):232-243 [PMID: 31765319]
  15. Annu Int Conf IEEE Eng Med Biol Soc. 2018 Jul;2018:5636-5639 [PMID: 30441614]
  16. Front Neurorobot. 2016 Apr 11;10:3 [PMID: 27148039]
  17. IEEE Trans Biomed Eng. 2011 Mar;58(3):681-8 [PMID: 20729161]
  18. IEEE Trans Biomed Eng. 2008 Aug;55(8):1956-65 [PMID: 18632358]
  19. IEEE Trans Biomed Eng. 2011 Sep;58(9):2537-44 [PMID: 21659017]
  20. IEEE Trans Biomed Eng. 2009 Jan;56(1):65-73 [PMID: 19224720]
  21. IEEE Trans Neural Syst Rehabil Eng. 2011 Dec;19(6):644-51 [PMID: 21846608]
  22. J Electromyogr Kinesiol. 2014 Oct;24(5):770-7 [PMID: 25048642]
  23. IEEE Trans Neural Syst Rehabil Eng. 2016 Sep;24(9):961-970 [PMID: 26513794]
  24. Front Neurosci. 2017 Feb 28;11:90 [PMID: 28293163]
  25. IEEE Trans Neural Syst Rehabil Eng. 2020 Jul;28(7):1678-1688 [PMID: 32634104]
  26. Nature. 2015 May 28;521(7553):436-44 [PMID: 26017442]
  27. J Neural Eng. 2019 Jun;16(3):036015 [PMID: 30849774]
  28. IEEE Trans Neural Syst Rehabil Eng. 2018 Jan;26(1):244-251 [PMID: 29324410]
  29. J Neural Eng. 2019 Apr;16(2):026003 [PMID: 30524028]
  30. IEEE Trans Neural Syst Rehabil Eng. 2014 Jul;22(4):797-809 [PMID: 24760934]
  31. Annu Int Conf IEEE Eng Med Biol Soc. 2018 Jul;2018:5978-5981 [PMID: 30441698]
  32. PLoS One. 2013 Jun 24;8(6):e67101 [PMID: 23826203]
  33. IEEE Trans Biomed Eng. 2009 Apr;56(4):1070-80 [PMID: 19272889]
  34. Annu Int Conf IEEE Eng Med Biol Soc. 2009;2009:3004-7 [PMID: 19963788]
  35. J Appl Physiol (1985). 2014 Dec 1;117(11):1215-30 [PMID: 25277737]
  36. Sci Rep. 2020 May 20;10(1):8377 [PMID: 32433481]
  37. Front Bioeng Biotechnol. 2019 Nov 15;7:316 [PMID: 31799243]
  38. IEEE Trans Biomed Eng. 2016 May;63(5):879-893 [PMID: 27046865]
  39. J Electromyogr Kinesiol. 2016 Aug;29:21-7 [PMID: 26190031]
  40. IEEE J Transl Eng Health Med. 2017 Nov 29;5:2100310 [PMID: 29255654]
  41. J Neuroeng Rehabil. 2017 Jan 7;14(1):2 [PMID: 28061779]
  42. Am J Phys Med Rehabil. 2007 Dec;86(12):977-87 [PMID: 18090439]
  43. IEEE Trans Biomed Eng. 2001 Mar;48(3):302-11 [PMID: 11327498]
  44. J Electromyogr Kinesiol. 2002 Feb;12(1):1-16 [PMID: 11804807]
  45. Front Neurosci. 2018 Feb 12;12:26 [PMID: 29483859]
  46. Comput Methods Programs Biomed. 2020 Oct;195:105643 [PMID: 32650088]
  47. Clin Neurophysiol. 2010 Oct;121(10):1616-23 [PMID: 20444646]
  48. J Neuroeng Rehabil. 2020 May 13;17(1):63 [PMID: 32404174]
  49. IEEE Trans Biomed Circuits Syst. 2020 Apr;14(2):244-256 [PMID: 31831433]
  50. IEEE Trans Inf Technol Biomed. 2011 Jul;15(4):522-30 [PMID: 21558060]
  51. J Rehabil Res Dev. 2015;52(5):605-18 [PMID: 26437448]
  52. Artif Organs. 2018 May;42(5):E67-E77 [PMID: 29068076]
  53. Front Bioeng Biotechnol. 2020 Apr 30;8:361 [PMID: 32426344]
  54. IEEE Trans Biomed Eng. 2006 Nov;53(11):2232-9 [PMID: 17073328]
  55. Med Eng Phys. 2015 May;37(5):518-24 [PMID: 25862333]
  56. Sci Transl Med. 2014 Oct 8;6(257):257ps12 [PMID: 25298319]
  57. PLoS One. 2018 Sep 13;13(9):e0203835 [PMID: 30212573]
  58. Sensors (Basel). 2020 Feb 21;20(4): [PMID: 32098264]
  59. IEEE Trans Neural Syst Rehabil Eng. 2014 May;22(3):623-33 [PMID: 24132017]
  60. Front Neurosci. 2016 Jan 06;9:496 [PMID: 26778953]
  61. Annu Int Conf IEEE Eng Med Biol Soc. 2012;2012:4324-7 [PMID: 23366884]
  62. Annu Int Conf IEEE Eng Med Biol Soc. 2010;2010:5058-61 [PMID: 21096026]
  63. Front Neurorobot. 2018 Sep 21;12:58 [PMID: 30297994]
  64. Front Neurorobot. 2016 Aug 22;10:7 [PMID: 27597823]
  65. J Prosthet Orthot. 2013 Jan 1;25(1):30-41 [PMID: 23459166]
  66. J Neuroeng Rehabil. 2011 May 22;8:29 [PMID: 21600048]
  67. Sci Rep. 2016 Nov 15;6:36571 [PMID: 27845347]
  68. Front Neurorobot. 2018 Feb 02;12:1 [PMID: 29456499]
  69. IEEE Trans Biomed Eng. 2009 May;56(5):1427-34 [PMID: 19473933]
  70. IEEE Trans Neural Syst Rehabil Eng. 2012 May;20(3):371-8 [PMID: 22180516]
  71. IEEE Trans Neural Syst Rehabil Eng. 2014 May;22(3):501-10 [PMID: 23996582]
  72. Med Biol Eng Comput. 2004 Jul;42(4):446-54 [PMID: 15320453]
  73. IEEE Trans Neural Syst Rehabil Eng. 2014 May;22(3):549-58 [PMID: 24235278]
  74. Sensors (Basel). 2018 Aug 01;18(8): [PMID: 30071617]
  75. J Rehabil Res Dev. 2013;50(8):1123-8 [PMID: 24458898]
  76. IEEE Trans Neural Syst Rehabil Eng. 2017 Jan;25(1):68-77 [PMID: 27164596]
  77. Annu Int Conf IEEE Eng Med Biol Soc. 2011;2011:1620-3 [PMID: 22254633]
  78. J Rehabil Res Dev. 2013;50(5):599-618 [PMID: 24013909]
  79. J Appl Physiol (1985). 2004 Apr;96(4):1486-95 [PMID: 15016793]
  80. Exp Brain Res. 2019 Feb;237(2):291-311 [PMID: 30506366]
  81. IEEE Trans Neural Syst Rehabil Eng. 2019 Apr;27(4):760-771 [PMID: 30714928]
  82. IEEE Trans Neural Syst Rehabil Eng. 2016 Jul;24(7):744-53 [PMID: 26173217]
  83. J Neuroeng Rehabil. 2009 Nov 17;6:41 [PMID: 19919710]
  84. J Exp Psychol. 1954 Jun;47(6):381-91 [PMID: 13174710]
  85. IEEE Trans Biomed Eng. 2003 Jul;50(7):848-54 [PMID: 12848352]
  86. Sensors (Basel). 2020 Jan 26;20(3): [PMID: 31991849]
  87. Front Comput Neurosci. 2014 Sep 17;8:100 [PMID: 25278868]
  88. IEEE Trans Neural Syst Rehabil Eng. 2005 Sep;13(3):280-91 [PMID: 16200752]
  89. IEEE Trans Neural Syst Rehabil Eng. 2016 Sep;24(9):928-939 [PMID: 26415203]
  90. IEEE Trans Biomed Eng. 2011 Aug;58(8): [PMID: 21592916]
  91. J Neural Eng. 2020 Jul 24;17(4):046016 [PMID: 32554885]
  92. Annu Int Conf IEEE Eng Med Biol Soc. 2015 Aug;2015:1140-3 [PMID: 26736467]
  93. Sensors (Basel). 2017 Feb 24;17(3): [PMID: 28245586]
  94. Front Neurorobot. 2017 Sep 27;11:51 [PMID: 29021753]
  95. J Neural Eng. 2015 Aug;12(4):046005 [PMID: 26028132]
  96. IEEE Trans Neural Syst Rehabil Eng. 2010 Apr;18(2):185-92 [PMID: 20071269]
  97. Front Comput Neurosci. 2013 Apr 29;7:51 [PMID: 23641212]
  98. Sensors (Basel). 2017 Jun 13;17(6): [PMID: 28608824]
  99. Sensors (Basel). 2019 Oct 22;19(20): [PMID: 31652616]
  100. J Neural Eng. 2016 Apr;13(2):026027 [PMID: 26924829]
  101. J Hand Ther. 2014 Jul-Sep;27(3):225-33; quiz 234 [PMID: 24878351]
  102. IEEE Trans Neural Syst Rehabil Eng. 2012 Sep;20(5):663-77 [PMID: 22665514]
  103. IEEE Trans Neural Syst Rehabil Eng. 2014 Jul;22(4):879-85 [PMID: 24710835]
  104. IEEE Int Conf Rehabil Robot. 2017 Jul;2017:1154-1159 [PMID: 28813977]
  105. Sensors (Basel). 2016 Aug 17;16(8): [PMID: 27548165]
  106. Front Bioeng Biotechnol. 2020 Mar 03;8:158 [PMID: 32195238]
  107. Prosthet Orthot Int. 2017 Jun;41(3):286-293 [PMID: 27473642]
  108. IEEE Trans Neural Syst Rehabil Eng. 2017 Oct;25(10):1821-1831 [PMID: 28358690]
  109. IEEE Trans Neural Syst Rehabil Eng. 2013 Nov;21(6):949-58 [PMID: 23475379]
  110. J Neuroeng Rehabil. 2012 Oct 05;9:74 [PMID: 23036049]
  111. J Neurosci. 2017 Nov 15;37(46):11285-11292 [PMID: 29054880]
  112. Front Comput Neurosci. 2015 Jan 09;8:169 [PMID: 25620928]
  113. PLoS One. 2018 Oct 30;13(10):e0206049 [PMID: 30376567]
  114. Front Neurosci. 2017 Jul 11;11:379 [PMID: 28744189]
  115. IEEE Trans Haptics. 2013 Jul-Sep;6(3):296-308 [PMID: 24808326]
  116. Sci Data. 2014 Dec 23;1:140053 [PMID: 25977804]
  117. IEEE Trans Inf Technol Biomed. 2009 Mar;13(2):274-80 [PMID: 19174357]
  118. Sci Rep. 2017 Oct 23;7(1):13840 [PMID: 29062019]
  119. Sensors (Basel). 2019 Jul 18;19(14): [PMID: 31323888]
  120. IEEE Trans Biomed Eng. 1993 Jan;40(1):82-94 [PMID: 8468080]
  121. Sensors (Basel). 2019 Jan 17;19(2): [PMID: 30658480]
  122. Sensors (Basel). 2020 Mar 15;20(6): [PMID: 32183473]

Word Cloud

Created with Highcharts 10.0.0sEMGrecognitioncontrolprostheticsignalDLlifehandssignalspatternhandupperlimbamputeesqualityperformancenaturalSurfaceelectromyographysystemscanDeeplearninggesturetechniquesneuralnetworkAmputationbringsheavyburdenreduceslimitsactivitiesdailyrealizationcrucialimprovingonewidelyusedbiologicalpredictionmotorintentionessentialelementconversioneffectiveoftenrequireslotcomputationalpowercomplexprocessExistingcommercialprovideactivedegreesfreedomperformedsurprisinglywelldevelopmentintelligentrecentyearssignificantimprovementhardwareequipmentcontinuousemergencelargedatasetsalsoboostedresearchprocessingeffectivelyimproveaccuracyreduceinfluenceinterferencefactorspaperanalyzesapplicabilityefficiencysEMG-basedreviewskeyDL-basedincludingacquisitionpreprocessingfeatureextractionclassificationpatternspost-processingevaluationFinallycurrentchallengesfutureprospectsclinicalapplicationoutlineddiscussedGestureRecognitionUsingElectromyographyLearningProsthesesHand:State-of-the-ArtChallengesFutureconvolutionaldeepprosthesisrecurrentsurface

Similar Articles

Cited By