Skip to main content
SpringerLink
Log in

Robust Machine Learning Mapping of sEMG Signals to Future Actuator Commands in Biomechatronic Devices

  • Research Article
  • Published:
Journal of Bionic Engineering Aims and scope Submit manuscript

Abstract

A machine learning model for regression of interrupted Surface Electromyography (sEMG) signals to future control-oriented signals (e.g., robot’s joint angle and assistive torque) of an active biomechatronic device for high-level myoelectric-based hierarchical control is proposed. A Recurrent Neural Network (RNN) was trained using output data, initially obtained from offline optimization of the biomechatronic (human–robot) device and shifted by the prediction horizon. The input of the RNN consisted of interrupted sEMG signals (to mimic signal disconnections) and previous kinematic signals of the assistive system. The RNN with a 0.1-s prediction horizon could predict the control-oriented joint angle and assistive torque with 92% and 86.5% regression accuracy, respectively, for the test dataset. This proposed approach permits a fast, predictive, and direct estimation of control-oriented signals instead of an iterative process that optimizes assistive torque in the inverse dynamic simulation of a multibody human–robot system. Training with these interrupted input signals significantly improves the regression accuracy in the case of sEMG signal disconnection. This Robust Predictive Control-oriented Machine Learning (Robust-MuscleNET) model can support volitional high-level myoelectric-based control of biomechatronic devices, such as exoskeletons, prostheses, and assistive/resistive robots. Future work should study the application to prosthesis control as well as the repeatability of the high-level controller with electrode shift. The low-level hierarchical controller that manages the human–robot interaction, the assistance/resistance strategy, and the actuator coordination should also be studied.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability statement

The data generated and/or analyzed during the current study are not publicly available for legal/ethical reasons but are available from the corresponding author on reasonable request.

References

  1. Yang, Z. Y., Guo, S. X., Suzuki, K., Liu, Y., & Kawanishi, M. (2023). An EMG-based biomimetic variable stiffness modulation strategy for bilateral motor skills relearning of upper limb elbow joint rehabilitation. Journal of Bionic Engineering. https://doi.org/10.1007/s42235-023-00339-9

    Article  PubMed  PubMed Central  Google Scholar 

  2. Copaci, D., Serrano, D., Moreno, L., & Blanco, D. (2018). A high-level control algorithm based on sEMG signalling for an elbow joint SMA exoskeleton. Sensors, 18(8), 2522. https://doi.org/10.3390/s18082522

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  3. Young, A. J., & Ferris, D. P. (2017). State of the art and future directions for lower limb robotic exoskeletons. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(2), 171–182. https://doi.org/10.1109/TNSRE.2016.2521160

    Article  PubMed  Google Scholar 

  4. Nasr, A., Bell, S., & McPhee, J. (2023). Optimal design of active-passive shoulder exoskeletons: A computational modeling of human-robot interaction. Multibody System Dynamics, 57, 73–106. https://doi.org/10.1007/s11044-022-09855-8

    Article  MathSciNet  Google Scholar 

  5. Li, N., Yang, T., Yang, Y., Chen, W. Y., Yu, P., Zhang, C., Xi, N., Zhao, Y., & Wang, W. (2022). Designing unpowered shoulder complex exoskeleton via contralateral drive for self-rehabilitation of post-stroke hemiparesis. Journal of Bionic Engineering. https://doi.org/10.1007/s42235-022-00299-6

    Article  Google Scholar 

  6. Nguiadem, C., Raison, M., & Achiche, S. (2020). Motion planning of upper-limb exoskeleton robots: A review. Applied Sciences, 10(21), 1–21. https://doi.org/10.3390/app10217626

    Article  CAS  Google Scholar 

  7. Nasr, A., Laschowski, B., & McPhee, J. (2021). Myoelectric control of robotic leg prostheses and exoskeletons: A review. In Proceedings of the ASME international design engineering technical conferences & computers and information in engineering conference (vol. 85444, pp. 2021–69203). ASME, Online, Virtual. https://doi.org/10.1115/DETC2021-69203.

  8. Nasr, A., Bell, S., He, J., Whittaker, R. L., Jiang, N., Dickerson, C. R., & McPhee, J. (2021). MuscleNET: Mapping electromyography to kinematic and dynamic biomechanical variables. Journal of Neural Engineering, 18(4), 0460d3. https://doi.org/10.1088/1741-2552/ac1adc

    Article  Google Scholar 

  9. Ghannadi, B., Razavian, R. S., & McPhee, J. (2018). Upper extremity rehabilitation robots: A survey. In Handbook of Biomechatronics, chap 9 (pp. 319–353). Elsevier. https://doi.org/10.1016/B978-0-12-812539-7.00012-X.

  10. Gaudet, G., Raison, M., & Achiche, S. (2021). Current trends and challenges in pediatric access to sensorless and sensor-based upper limb exoskeletons. Sensors. https://doi.org/10.3390/s21103561

    Article  PubMed  PubMed Central  Google Scholar 

  11. Yang, C., Xi, X. G., Chen, S. J., Miran, S. M., Hua, X., & Luo, Z. Z. (2019). SEMG-based multifeatures and predictive model for knee-joint-angle estimation. AIP Advances, 9(9), 095042. https://doi.org/10.1063/1.5120470

    Article  ADS  Google Scholar 

  12. Zhu, K., Xue, T., Zhang, T., & Zhang, M. (2019). SEMG-based joint moment estimation for hip exoskeleton general assistive strategy. In Proceedings of the Chinese automation congress (pp. 3826–3830). IEEE. https://doi.org/10.1109/CAC48633.2019.8997479.

  13. Yao, T., Lv, J., Yang, L., Xu, A., & Qu, S. (2023). Design of the pneumatic pressure smart shoes for an ankle-assisted exoskeleton. Journal of Bionic Engineering. https://doi.org/10.1007/s42235-023-00335-z

    Article  Google Scholar 

  14. Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020). Human motion trajectory prediction: A survey. International Journal of Robotics Research, 39(8), 895–935. https://doi.org/10.1177/0278364920917446

    Article  Google Scholar 

  15. Abdel-Malek, K., & Arora, J. (2013). Human motion simulation: Predictive dynamics. Elsevier. https://doi.org/10.1016/C2012-0-00620-5

  16. Asghari Oskoei, M., & Hu, H. (2007). Myoelectric control systems—A survey. Biomedical Signal Processing and Control, 2(4), 275–294. https://doi.org/10.1016/j.bspc.2007.07.009

    Article  Google Scholar 

  17. Akmal, M., Zubair, S., Jochumsen, M., Kamavuako, E. N., & Niazi, I. K. (2019). A tensor-based method for completion of missing electromyography data. IEEE Access, 7, 104710–104720. https://doi.org/10.1109/ACCESS.2019.2931371

    Article  Google Scholar 

  18. Naber, A., Mastinu, E., & Ortiz-Catalan, M. (2019). Stationary wavelet processing and data imputing in myoelectric pattern recognition on a low-cost embedded system. IEEE Transactions on Medical Robotics and Bionics, 1(4), 256–266. https://doi.org/10.1109/tmrb.2019.2949853

    Article  Google Scholar 

  19. Singh, R. M., Chatterji, S., & Kumar, A. (2014). A review on surface EMG based control schemes of exoskeleton robot in stroke rehabilitation. In Proceedings of the international conference on machine intelligence research and advancement, IEEE (pp. 310–315). IEEE. https://doi.org/10.1109/ICMIRA.2013.65.

  20. Durandau, G., Farina, D., & Sartori, M. (2018). Robust real-time musculoskeletal modeling driven by electromyograms. IEEE Transactions on Biomedical Engineering, 65(3), 556–564. https://doi.org/10.1109/TBME.2017.2704085

    Article  PubMed  Google Scholar 

  21. Liu, Y. X., Xin, D. X., Hua, J., & Liu, M. Z. (2020). SEMG motion intention recognition based on wavelet time-frequency spectrum and ConvLSTM. Journal of Physics: Conference Series. https://doi.org/10.1088/1742-6596/1631/1/012150

    Article  Google Scholar 

  22. Conforto, S., Mathieu, P. A., Schmid, M., Bibbo, D., Florestal, J. R., & D’Alessio, T. (2006). How much can we trust the electromechanical delay estimated by using electromyography? In Proceedings of the annual international conference of the IEEE engineering in medicine and biology (pp. 1256–1259). IEEE. https://doi.org/10.1109/IEMBS.2006.259335.

  23. Sözen, H., Cè, E., Bisconti, A. V., Rampichini, S., Longo, S., Coratella, G., Shokohyar, S., Doria, C., Borrelli, M., Limonta, E., & Esposito, F. (2019). Differences in electromechanical delay components induced by sex, age and physical activity level: New insights from a combined electromyographic, mechanomyographic and force approach. Sport Sciences for Health, 15(3), 623–633. https://doi.org/10.1007/s11332-019-00563-z

    Article  Google Scholar 

  24. Durandau, G., Farina, D., Asín-Prieto, G., Dimbwadyo-Terrer, I., Lerma-Lara, S., Pons, J. L., Moreno, J. C., & Sartori, M. (2019). Voluntary control of wearable robotic exoskeletons by patients with paresis via neuromechanical modeling. Journal of NeuroEngineering and Rehabilitation, 16(1), 1–18. https://doi.org/10.1186/s12984-019-0559-z

    Article  Google Scholar 

  25. Nasr, A., Ferguson, S., & McPhee, J. (2021). Model-based design and optimization of passive shoulder exoskeletons. In Proceedings of the ASME international design engineering technical conferences & computers and information in engineering conference (pp. 2021–69437). ASME, Online, Virtual. https://doi.org/10.1115/DETC2021-69437.

  26. Gopura, R., Kiguchi, K., & Yi, Y. (2009). SUEFUL-7: A 7DOF upper-limb exoskeleton robot with muscle-model-oriented EMG-based control. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 1126–1131). IEEE. https://doi.org/10.1109/IROS.2009.5353935.

  27. Malcolm, P., Galle, S., & De Clercq, D. (2017). Fast exoskeleton optimization. Science, 356(6344), 1230–1231. https://doi.org/10.1126/science.aan5367

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  28. Nasr, A., Hunter, J., Dickerson, C. R., & McPhee, J. (2023). Evaluation of a machine learning-driven active-passive upper limb exoskeleton robot: Experimental human-in-the-loop study. Wearable Technologies, 4, e13. https://doi.org/10.1017/wtc.2023.9

    Article  Google Scholar 

  29. Hayashi, Y., Dubey, R., & Kiguchi, K. (2011). Torque optimization for a 7DOF upper-limb power-assist exoskeleton robot. In Proceedings of the IEEE workshop on robotic intelligence in informationally structured space (pp. 49–54). IEEE. https://doi.org/10.1109/RIISS.2011.5945786.

  30. Sarac, M., Solazzi, M., Sotgiu, E., Bergamasco, M., & Frisoli, A. (2017). Design and kinematic optimization of a novel underactuated robotic hand exoskeleton. Meccanica, 52(3), 749–761. https://doi.org/10.1007/s11012-016-0530-z

    Article  MathSciNet  Google Scholar 

  31. Zhou, L. L., Li, Y.B., & Bai, S. P. (2017). A human-centered design optimization approach for robotic exoskeletons through biomechanical simulation. Robotics and Autonomous Systems, 91, 337–347. https://doi.org/10.1016/j.robot.2016.12.012

    Article  Google Scholar 

  32. Zhang, J., Fiers, P., Witte, K. A., Jackson, R. W., Poggensee, K. L., Atkeson, C. G., & Collins, S. H. (2017). Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 356(6344), 1280–1283. https://doi.org/10.1126/science.aal5054

    Article  ADS  CAS  PubMed  Google Scholar 

  33. Hashemi, A., & McPhee, J. (2021). Assistive sliding mode control of a rehabilitation robot with automatic weight adjustment. In Proceedings of the 43rd annual international conference of the IEEE engineering in medicine & biology society (pp. 4891–4896). IEEE. https://doi.org/10.1109/embc46164.2021.9631110.

  34. Mehrabi, N., Razavian, R. S., Ghannadi, B., & McPhee, J. (2017). Predictive simulation of reaching moving targets using nonlinear model predictive control. Frontiers in Computational Neuroscience, 10, 143. https://doi.org/10.3389/fncom.2016.00143

    Article  PubMed  PubMed Central  Google Scholar 

  35. Chen, C. J., Huang, K., Li, D. N., Zhao, Z. X, & Hong, J. (2020). Multi-segmentation parallel CNN model for estimating assembly torque using surface electromyography signals. Sensors, 20(15), 1–22. https://doi.org/10.3390/s20154213

    Article  CAS  Google Scholar 

  36. Li, C., Li, G., Jiang, G., Chen, D., & Liu, H. (2020). Surface EMG data aggregation processing for intelligent prosthetic action recognition. Neural Computing and Applications, 32(22), 16795–16806. https://doi.org/10.1007/s00521-018-3909-z

    Article  Google Scholar 

  37. Liu, J., Kang, S. H., Xu, D., Ren, Y., Lee, S. J., & Zhang, L. Q. (2017). EMG-Based continuous and simultaneous estimation of arm kinematics in able-bodied individuals and stroke survivors. Frontiers in Neuroscience, 11, 480. https://doi.org/10.3389/fnins.2017.00480

    Article  PubMed  PubMed Central  Google Scholar 

  38. Zhang, Y., Zhang, X. D., Lu, Z. F., Jiang, Z. M., & Zhang, T. (2020). A novel wrist joint torque prediction method based on EMG and LSTM. In Proceedings of the 10th IEEE international conference on cyber technology in automation, control and intelligent systems (pp. 242–245). IEEE. https://doi.org/10.1109/CYBER50695.2020.9279119.

  39. Fleming, A., Stafford, N., Huang, S., Hu, X., Ferris, D. P., & Huang, H. H. (2021). Myoelectric control of robotic lower limb prostheses: A review of electromyography interfaces, control paradigms, challenges and future directions. Journal of Neural Engineering, 18, 041004. https://doi.org/10.1088/1741-2552/ac1176

    Article  ADS  Google Scholar 

  40. Fougner, A., Stavdahl, O., Kyberd, P. J., Losier, Y. G., & Parker, P. A. (2012). Control of upper limb prostheses: Terminology and proportional myoelectric control a review. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(5), 663–677. https://doi.org/10.1109/TNSRE.2012.2196711

    Article  PubMed  Google Scholar 

  41. Nsugbe, E., Samuel, O. W., Asogbon, M. G., & Li, G. (2021). Phantom motion intent decoding for transhumeral prosthesis control with fused neuromuscular and brain wave signals. IET Cyber-Systems and Robotics, 3(1), 77–88. https://doi.org/10.1049/csy2.12009

    Article  Google Scholar 

  42. Vaca Benitez, L. M., Tabie, M., Will, N., Schmidt, S., Jordan, M., & Kirchner, E. A. (2013). Exoskeleton technology in rehabilitation: Towards an EMG-based orthosis system for upper limb neuromotor rehabilitation. Journal of Robotics, 2013, 610589. https://doi.org/10.1155/2013/610589

    Article  Google Scholar 

  43. Whittaker, R. L., Park, W., & Dickerson, C. R. (2018). Application of a symbolic motion structure representation algorithm to identify upper extremity kinematic changes during a repetitive task. Journal of Biomechanics, 72, 235–240. https://doi.org/10.1016/j.jbiomech.2018.02.027

    Article  PubMed  Google Scholar 

  44. Kelly, B. T., Kadrmas, W. R., & Speer, K. P. (1996). The manual muscle examination for rotator cuff strength: An electromyographic investigation. American Journal of Sports Medicine, 24(5), 581–588. https://doi.org/10.1177/036354659602400504

    Article  CAS  PubMed  Google Scholar 

  45. Pope, G. D. (1998). Introduction to surface electromyography. Physiotherapy, 84(8), 405. https://doi.org/10.1016/s0031-9406(05)61482-4

    Article  Google Scholar 

  46. Whittaker, R. L., La Delfa, N. J., & Dickerson, C. R. (2019). Algorithmically detectable directional changes in upper extremity motion indicate substantial myoelectric shoulder muscle fatigue during a repetitive manual task. Ergonomics, 62(3), 431–443. https://doi.org/10.1080/00140139.2018.1536808

    Article  PubMed  Google Scholar 

  47. Galle, S., Malcolm, P., Derave, W., & De Clercq, D. (2013). Adaptation to walking with an exoskeleton that assists ankle extension. Gait and Posture, 38(3), 495–499. https://doi.org/10.1016/j.gaitpost.2013.01.029

    Article  CAS  PubMed  Google Scholar 

  48. Nasr, A., Hashemi, A., & McPhee, J. (2022). Model-based mid-level regulation for assist-as-needed hierarchical control of wearable robots: A computational study of human-robot adaptation. Robotics, 11(1), 20. https://doi.org/10.3390/robotics11010020

    Article  PubMed  Google Scholar 

  49. Mehrabi, N., Shourijeh, M. S., & McPhee, J. (2012). Study of human steering tasks using neuromuscular driver model. In International symposium on advanced vehicle control (pp. 1–6), Seoul, Korea.

  50. Nasr, A., & McPhee, J. (2022). Multibody constrained dynamic modelling of human-exoskeleton: Toward optimal design and control of an active-passive wearable robot. In Proceedings of the 6th joint international conference on multibody system dynamics and the 10th Asian conference on multibody system dynamics (p. 189). Springer.

  51. Nasr, A., & McPhee, J. (2022). Biarticular MuscleNET: A machine learning model of biarticular muscles. In Proceedings of the North American congress on biomechanics, Ottawa.

  52. McPhee, J., & Nasr, A. (2023). Multibody system dynamics: A fundamental tool for biomechatronic system design. In ECCOMAS thematic conference on multibody dynamics, Lisbon.

  53. Febrer-Nafría, M., Nasr, A., Ezati, M., Brown, P., Font-Llagunes, J. M., & McPhee, J. (2022). Predictive multibody dynamic simulation of human neuromusculoskeletal systems: A review. Multibody System Dynamics. https://doi.org/10.1007/s11044-022-09852-x

    Article  Google Scholar 

  54. Nasr, A., Hashemi, A., & McPhee, J. (2023). Scalable musculoskeletal model for dynamic simulations of upper body movement. Computer Methods in Biomechanics and Biomedical Engineering. https://doi.org/10.1080/10255842.2023.2184747

    Article  PubMed  Google Scholar 

  55. Wu, G., Van Der Helm, F. C., Veeger, H. E., Makhsous, M., Van Roy, P., Anglin, C., Nagels, J., Karduna, A. R., McQuade, K., Wang, X., Werner, F. W., & Buchholz, B. (2005). ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion - Part II: Shoulder, elbow, wrist and hand. Journal of Biomechanics, 38(5), 981–992. https://doi.org/10.1016/j.jbiomech.2004.05.042

    Article  CAS  PubMed  Google Scholar 

  56. Nasr, A., Ferguson, S., & McPhee, J. (2022). Model-based design and optimization of passive shoulder exoskeletons. Journal of Computational and Nonlinear Dynamics, 17(5), 051004. https://doi.org/10.1115/1.4053405

    Article  Google Scholar 

  57. De Luca, C. J., Donald Gilmore, L., Kuznetsov, M., & Roy, S. H. (2010). Filtering the surface EMG signal: Movement artifact and baseline noise contamination. Journal of Biomechanics, 43(8), 1573–1579. https://doi.org/10.1016/j.jbiomech.2010.01.027

    Article  PubMed  Google Scholar 

  58. Reaz, M. B. I., Hussain, S., & Mohd-Yasin, F. (2006). Techniques of EMG signal analysis: Detection, processing, classification and applications. Biological Procedures Online, 8(1), 11–35. https://doi.org/10.1251/bpo115

    Article  Google Scholar 

  59. Kannape, O. A., & Herr, H. M. (2014). Volitional control of ankle plantar flexion in a powered transtibial prosthesis during stair-ambulation. In Proceedings of the 36th annual international conference of the ieee engineering in medicine and biology society (pp. 1662–1665). IEEE. https://doi.org/10.1109/EMBC.2014.6943925.

  60. Shourijeh, M. S., Razavian, R. S., & McPhee, J. (2017). Estimation of maximum finger tapping frequency using musculoskeletal dynamic simulations. Journal of Computational and Nonlinear Dynamics, 12(5), 051009. https://doi.org/10.1115/1.4036288

    Article  Google Scholar 

  61. Nasr, A., He, J., Jiang, N., & McPhee, J. (2021). Muscle modelling using machine learning and optimal filtering of sEMG signals. In Proceedings of the 45th meeting of the American Society of Biomechanics, virtual (p. 83).

  62. Razavian, R. S., Ghannadi, B., & McPhee, J. (2019). On the relationship between muscle synergies and redundant degrees of freedom in musculoskeletal systems. Frontiers in Computational Neuroscience, 13, 23. https://doi.org/10.3389/fncom.2019.00023

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by funding from the Canada Research Chairs Program and the Natural Sciences and Engineering Research Council of Canada. The authors wish to thank Ekso Bionics Holdings Inc. for providing the Ekso EVO passive shoulder exoskeleton.

Author information

Authors and Affiliations

  1. Systems Design Engineering, University of Waterloo, Waterloo, N2L 3G1, Canada

    Ali Nasr, Sydney Bell & John McPhee

  2. Kinesiology and Health Sciences, University of Waterloo, Waterloo, N2L 3G3, Canada

    Rachel L. Whittaker & Clark R. Dickerson

Authors
  1. Ali Nasr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sydney Bell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachel L. Whittaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Clark R. Dickerson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John McPhee
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, AN and JM; methodology, software, visualization, and writing original draft preparation AN; formal analysis, AN, CD, and JM; data acquisition, AN and RW; writing review and editing, AN, SB, CD, and JM; supervision, project administration, and funding acquisition, JM. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Ali Nasr.

Ethics declarations

Conflict of interest

The authors have no competing interests that might be perceived to influence the results and/or discussion reported in this paper.

Institutional review board statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Office of Research Ethics of the University of Waterloo #21246 and #43980.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nasr, A., Bell, S., Whittaker, R.L. et al. Robust Machine Learning Mapping of sEMG Signals to Future Actuator Commands in Biomechatronic Devices. J Bionic Eng 21, 270–287 (2024). https://doi.org/10.1007/s42235-023-00453-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42235-023-00453-8

Keywords

Search

Navigation