OpenLB
Open Library of Bioscience
Advanced Search
Medical & biological engineering & computing
May, 2024;62 (5): 1333-1346.

Prediction of medial knee contact force using multisource fusion recurrent neural network and transfer learning.

Jianjun Zou, Xiaogang Zhang, Yali Zhang, Zhongmin Jin
Author Information
  1. Jianjun Zou: School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China.
  2. Xiaogang Zhang: School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China. xg@swjtu.edu.cn.
  3. Yali Zhang: School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China.
  4. Zhongmin Jin: School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China.
PMID: 38182944 DOI: 10.1007/s11517-023-03011-w

Abstract

Estimation of knee contact force (KCF) during gait provides essential information to evaluate knee joint function. Machine learning has been employed to estimate KCF because of the advantages of low computational cost and real-time. However, the existing machine learning models do not adequately consider gait-related data's temporal-dependent, multidimensional, and highly heterogeneous nature. This study is aimed at developing a multisource fusion recurrent neural network to predict the medial condyle KCF. First, a multisource fusion long short-term memory (MF-LSTM) model was established. Then, we developed a transfer learning strategy based on the MF-LSTM model for subject-specific medial KCF prediction. Four subjects with instrumented tibial prostheses were obtained from the literature. The results showed that the MF-LSTM model could predict medial KCF to a certain high level of accuracy (the mean of ρ = 0.970). The transfer learning model improved the prediction accuracy (the mean of ρ = 0.987). This study shows that the MF-LSTM model is a powerful and accurate computational tool for medial KCF prediction. Introducing transfer learning techniques could further improve the prediction performance for the target subject. This coupling strategy can help clinicians accurately estimate and track joint contact forces in real time.

Keywords

Knee contact force Long short-term memory network Multisource fusion Transfer learning

References

  1. Bennett HJ, Shen G, Cates HE et al (2017) Effects of toe-in and toe-in with wider step width on level walking knee biomechanics in varus, valgus, and neutral knee alignments. Knee 24(6):1326–1334. https://doi.org/10.1016/j.knee.2017.08.058 [DOI: 10.1016/j.knee.2017.08.058]
  2. Dell’isola A, Smith SL, Andersen MS et al (2017) Knee internal contact force in a varus malaligned phenotype in knee osteoarthritis (KOA). Osteoarthr Cartil 25(12):2007–2013. https://doi.org/10.1016/j.joca.2017.08.010 [DOI: 10.1016/j.joca.2017.08.010]
  3. Konrath JM, Karatsidis A, Schepers HM et al (2019) Estimation of the knee adduction moment and joint contact force during daily living activities using inertial motion capture. Sensors 19(7):1681–1692. https://doi.org/10.3390/s19071681 [DOI: 10.3390/s19071681]
  4. Richards RE, Andersen MS, Harlaar J et al (2018) Relationship between knee joint contact forces and external knee joint moments in patients with medial knee osteoarthritis: effects of gait modifications. Osteoarthr Cartil 26(9):1203–1214. https://doi.org/10.1016/j.joca.2018.04.011 [DOI: 10.1016/j.joca.2018.04.011]
  5. Liu F, Wang M, Wang J et al (2019) The influence of frontally flat bearing design on contact mechanics and kinematics in total knee joint replacements. Tribol Int 08(136):23–30. https://doi.org/10.1016/j.triboint.2019.03.006 [DOI: 10.1016/j.triboint.2019.03.006]
  6. Phanphet S, Dechjarern S, Jomjanyong S (2017) Above-knee prosthesis design based on fatigue life using finite element method and design of experiment. Med Eng Phys 43:86–91. https://doi.org/10.1016/j.medengphy.2017.01.001 [DOI: 10.1016/j.medengphy.2017.01.001]
  7. Walker PS, Lowry MT, Kumar A (2014) The effect of geometric variations in posterior-stabilized knee designs on motion characteristics measured in a knee loading machine. Clin Orthop Relat Res 472(1):238–247. https://doi.org/10.1007/s11999-013-3088-2 [DOI: 10.1007/s11999-013-3088-2]
  8. Houserman DJ, Berend KR, Lombardi AV et al (2022) The viability of an artificial intelligence/machine learning prediction model to determine candidates for knee arthroplasty. J Arthroplasty. https://doi.org/10.1016/j.arth.2022.04.003 [DOI: 10.1016/j.arth.2022.04.003]
  9. Lambrechts A, Wirix-Speetjens R, Maes F et al (2022) Artificial intelligence based patient-specific preoperative planning algorithm for total knee arthroplasty. Front Robot AI 9:840282. https://doi.org/10.3389/frobt.2022.840282 [DOI: 10.3389/frobt.2022.840282]
  10. Kluge F, Hannink J, Pasluosta C et al (2018) Pre-operative sensor-based gait parameters predict functional outcome after total knee arthroplasty. Gait Posture 66:194–200. https://doi.org/10.1016/j.gaitpost.2018.08.026 [DOI: 10.1016/j.gaitpost.2018.08.026]
  11. Young-Shand KL, Roy PC, Dunbar MJ et al (2023) Gait biomechanics phenotypes among total knee arthroplasty candidates by machine learning cluster analysis. J Orthop Res 41(2):335–344. https://doi.org/10.1002/jor.25363 [DOI: 10.1002/jor.25363]
  12. Simic M, Hinman RS, Wrigley TV et al (2011) Gait modification strategies for altering medial knee joint load: a systematic review. Arthritis Care Res 63(3):405–426. https://doi.org/10.1002/acr.20380 [DOI: 10.1002/acr.20380]
  13. Barrios JA, Crossley KM, Davis IS (2010) Gait retraining to reduce the knee adduction moment through real-time visual feedback of dynamic knee alignment. J Biomech 43(11):2208–2213. https://doi.org/10.1016/j.jbiomech.2010.03.040 [DOI: 10.1016/j.jbiomech.2010.03.040]
  14. Felson DT, Nevitt MC, Zhang Y et al (2002) High prevalence of lateral knee osteoarthritis in Beijing Chinese compared with Framingham Caucasian subjects. Arthritis Rheum 46(5):1217–1222. https://doi.org/10.1002/art.10293 [DOI: 10.1002/art.10293]
  15. Jones RK, Chapman GJ, Findlow AH et al (2013) A new approach to prevention of knee osteoarthritis: reducing medial load in the contralateral knee. J Rheumatol 40(3):309–315. https://doi.org/10.3899/jrheum.120589 [DOI: 10.3899/jrheum.120589]
  16. Heinlein B, Graichen F, Bender A et al (2007) Design, calibration and pre-clinical testing of an instrumented tibial tray. J Biomech 40(Supp-S1):S4–S10. https://doi.org/10.1016/j.jbiomech.2007.02.014 [DOI: 10.1016/j.jbiomech.2007.02.014]
  17. Arami A, Simoncini M, Atasoy O et al (2013) Instrumented knee prosthesis for force and kinematics measurements. IEEE Trans Autom Sci Eng 10(3):615–624. https://doi.org/10.1109/tase.2012.2226030 [DOI: 10.1109/tase.2012.2226030]
  18. Kutzner I, Heinlein B, Graichen F et al (2010) Loading of the knee joint during activities of daily living measured in vivo in five subjects. J Biomech 43(11):2164–2173. https://doi.org/10.1016/j.jbiomech.2010.03.046 [DOI: 10.1016/j.jbiomech.2010.03.046]
  19. Peng Y, Zhang Z, Gao Y et al (2018) Concurrent prediction of ground reaction forces and moments and tibiofemoral contact forces during walking using musculoskeletal modelling. Med Eng Phys 52:31–40. https://doi.org/10.1016/j.medengphy.2017.11.008 [DOI: 10.1016/j.medengphy.2017.11.008]
  20. Lerner ZF, Demer MS, Delp SL et al (2015) How tibiofemoral alignment and contact locations affect predictions of medial and lateral tibiofemoral contact forces. J Biomech 48(4):644–650. https://doi.org/10.1016/j.jbiomech.2014.12.049 [DOI: 10.1016/j.jbiomech.2014.12.049]
  21. Jung Y, Phan CB, Koo S (2016) Intra-articular knee contact force estimation during walking using force-reaction elements and subject-specific joint model. J Biomech Eng 138(2):021016–021021. https://doi.org/10.1115/1.4032414 [DOI: 10.1115/1.4032414]
  22. Durandau G, Farina D, Sartori M (2018) Robust real-time musculoskeletal modeling driven by electromyograms. IEEE Trans Biomed Eng 65(3):556–564. https://doi.org/10.1109/TBME.2017.2704085 [DOI: 10.1109/TBME.2017.2704085]
  23. Kang KT, Son J, Baek C et al (2018) Femoral component alignment in unicompartmental knee arthroplasty leads to biomechanical change in contact stress and collateral ligament force in knee joint. Arch Orthop Trauma Surg 138(4):563–572. https://doi.org/10.1007/s00402-018-2884-2 [DOI: 10.1007/s00402-018-2884-2]
  24. Damsgaard M, Rasmussen J, Christensen ST et al (2006) Analysis of musculoskeletal systems in the AnyBody Modeling System. Simul Model Pract Theory 14(8):1100–1111. https://doi.org/10.1016/j.simpat.2006.09.001 [DOI: 10.1016/j.simpat.2006.09.001]
  25. Fregly BJ, Besier TF, Lloyd DG et al (2011) Grand challenge competition to predict in vivo knee loads. J Orthop Res 30(4):503–513. https://doi.org/10.1002/jor.22023 [DOI: 10.1002/jor.22023]
  26. Zaroug A, Lei DTH, Mudie K et al (2020) Lower limb kinematics trajectory prediction using long short-term memory neural networks. Front Bioeng Biotechnol 8:362. https://doi.org/10.3389/fbioe.2020.00362 [DOI: 10.3389/fbioe.2020.00362]
  27. Xu L, Chen X, Cao S et al (2018) Feasibility study of advanced neural networks applied to sEMG-based force estimation. Sensors 18(10):3226. https://doi.org/10.3390/s18103226 [DOI: 10.3390/s18103226]
  28. Kim SH, Kwon Y, Kim K et al (2020) Estimation of hand motion from piezoelectric soft sensor using deep recurrent network. Appl Sci 10(6):2194. https://doi.org/10.3390/app10062194 [DOI: 10.3390/app10062194]
  29. Choi A, Jung H, Lee KY et al (2019) Machine learning approach to predict center of pressure trajectories in a complete gait cycle: a feedforward neural network vs. LSTM network. Med Biol Eng Comput 57:2693–2703. https://doi.org/10.1007/s11517-019-02056-0 [DOI: 10.1007/s11517-019-02056-0]
  30. Steven Eyobu O, Han DS (2018) Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network. Sensors 18(9):2892. https://doi.org/10.3390/s18092892 [DOI: 10.3390/s18092892]
  31. Sadeghzadehyazdi N, Batabyal T, Acton ST (2021) Modeling spatiotemporal patterns of gait anomaly with a CNN-LSTM deep neural network. Expert Syst Appl 185:115582. https://doi.org/10.1016/j.eswa.2021.115582 [DOI: 10.1016/j.eswa.2021.115582]
  32. Gautam A, Panwar M, Biswas D et al (2020) MyoNet: a transfer-learning-based LRCN for lower limb movement recognition and knee joint angle prediction for remote monitoring of rehabilitation progress from sEMG. IEEE J Transl Eng Health Med 8:1–10. https://doi.org/10.1109/jtehm.2020.2972523 [DOI: 10.1109/jtehm.2020.2972523]
  33. Stetter BJ, Ringhof S, Krafft FC et al (2019) Estimation of knee joint forces in sport movements using wearable sensors and machine learning. Sensors 19(17):3690. https://doi.org/10.3390/s19173690 [DOI: 10.3390/s19173690]
  34. Ardestani MM, Chen Z, Wang L et al (2014) Feed forward artificial neural network to predict contact force at medial knee joint: application to gait modification. Neurocomputing 139:114–129. https://doi.org/10.1016/j.neucom.2014.02.054 [DOI: 10.1016/j.neucom.2014.02.054]
  35. Zhu Y, Xu W, Luo G et al (2020) Random forest enhancement using improved artificial fish swarm for the medial knee contact force prediction. Artif Intell Med 103:101811. https://doi.org/10.1016/j.artmed.2020.101811 [DOI: 10.1016/j.artmed.2020.101811]
  36. Dao TT (2019) From deep learning to transfer learning for the prediction of skeletal muscle forces. Med Biol Eng Comput 57(5):1049–1058. https://doi.org/10.1007/s11517-018-1940-y [DOI: 10.1007/s11517-018-1940-y]
  37. Zhang L, Soselia D, Wang R et al (2022) Lower-limb joint torque prediction using LSTM neural networks and transfer learning. IEEE Trans Neural Syst Rehabil Eng 30:600–609. https://doi.org/10.1109/TNSRE.2022.3156786 [DOI: 10.1109/TNSRE.2022.3156786]
  38. Burton WS, Myers CA, Rullkoetter PJ (2021) Machine learning for rapid estimation of lower extremity muscle and joint loading during activities of daily living. J Biomech 123:110439. https://doi.org/10.1016/j.jbiomech.2021.110439 [DOI: 10.1016/j.jbiomech.2021.110439]
  39. Windsor E, Cao W (2022) Improving exchange rate forecasting via a new deep multimodal fusion model. Appl Intell 52(14):16701–16717. https://doi.org/10.1007/s10489-022-03342-5 [DOI: 10.1007/s10489-022-03342-5]
  40. Wang Y, Liu M, Bao Z et al (2018) Short-term load forecasting with multi-source data using gated recurrent unit neural networks. Energies 11(5):1138. https://doi.org/10.3390/en11051138 [DOI: 10.3390/en11051138]
  41. Kutzner I, Bender A, Dymke J et al (2017) Mediolateral force distribution at the knee joint shifts across activities and is driven by tibiofemoral alignment. Bone Joint J 99-B(6):779–787. https://doi.org/10.1302/0301-620x.99b6.Bjj-20160713.R1 [DOI: 10.1302/0301-620x.99b6.Bjj-20160713.R1]
  42. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359. https://doi.org/10.1109/TKDE.2009.191 [DOI: 10.1109/TKDE.2009.191]
  43. Patrini I, Ruperti M, Moccia S et al (2020) Transfer learning for informative-frame selection in laryngoscopic videos through learned features. Med Biol Eng Comput 58:1225–1238. https://doi.org/10.1007/s11517-020-02127-7 [DOI: 10.1007/s11517-020-02127-7]
  44. Zhang J, Zhou W, Chen X et al (2019) Multisource selective transfer framework in multiobjective optimization problems. IEEE Trans Evol Comput 24(3):424–438. https://doi.org/10.1109/tevc.2019.2926107 [DOI: 10.1109/tevc.2019.2926107]
  45. Sepp H, Schmidhuber J (1997) Long-short term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 [DOI: 10.1162/neco.1997.9.8.1735]
  46. Zou J, Zhang X, Zhang Y et al (2022) Prediction on the medial knee contact force in patients with knee valgus using transfer learning approaches: application to rehabilitation gaits. Comput Biol Med 150:106099. https://doi.org/10.1016/j.compbiomed.2022.106099 [DOI: 10.1016/j.compbiomed.2022.106099]
  47. Howard J, Ruder S. Universal language model fine-tuning for text classification. 56th Annual Meeting of the Association-for-Computational-Linguistics (ACL), 2018: 328–339
  48. Yosinski J, Clune J, Bengio Y et al (2014) How transferable are features in deep neural networks ?. 28th Conference on Neural Information Processing Systems (NIPS)
  49. Razu SS (2017) EMG-driven forward dynamics simulation to estimate in vivo joint contact forces during normal, smooth, and bouncy gait. J Biomech Eng 140:071012. https://doi.org/10.1115/1.4038507 [DOI: 10.1115/1.4038507]
  50. Shu L, Yamamoto K, Yao J et al (2018) A subject-specific finite element musculoskeletal framework for mechanics analysis of a total knee replacement. J Biomech 77:146–154. https://doi.org/10.1016/j.jbiomech.2018.07.008 [DOI: 10.1016/j.jbiomech.2018.07.008]

Grants

  1. 52035012/National Natural Science Foundation of China
  2. 52005418/National Natural Science Foundation of China
  3. 52275215/National Natural Science Foundation of China
  4. 2022NSFSC1940/Natural Science Foundation of Sichuan Province

MeSH Term

Humans
Walking
Biomechanical Phenomena
Knee Joint
Gait
Neural Networks, Computer
Machine Learning
Journal Article

Word Cloud

Created with Highcharts 10.0.0learningKCFmedialmodelcontactfusionMF-LSTMtransferpredictionkneeforcemultisourcenetworkjointestimatecomputationalstudyrecurrentneuralpredictshort-termmemorystrategyaccuracymeanρ = 0EstimationgaitprovidesessentialinformationevaluatefunctionMachineemployedadvantageslowcostreal-timeHoweverexistingmachinemodelsadequatelyconsidergait-relateddata'stemporal-dependentmultidimensionalhighlyheterogeneousnatureaimeddevelopingcondyleFirstlongestablisheddevelopedbasedsubject-specificFoursubjectsinstrumentedtibialprosthesesobtainedliteratureresultsshowedcertainhighlevel970improved987showspowerfulaccuratetoolIntroducingtechniquesimproveperformancetargetsubjectcouplingcanhelpcliniciansaccuratelytrackforcesrealtimePredictionusingKneeLongMultisourceTransfer

Similar Articles

Cited By