Hostname: page-component-76fb5796d-5g6vh Total loading time: 0 Render date: 2024-04-28T01:51:21.971Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian optimization for assist-as-needed controller in robot-assisted upper limb training based on energy information

Published online by Cambridge University Press:  10 July 2023

Jianxi Zhang Open the ORCID record for Jianxi Zhang [Opens in a new window] ,
Hong Zeng ,
Xiao Li ,
Guozheng Xu ,
Yongqiang Li  and
Aiguo Song
Jianxi Zhang
Affiliation:
The State Key Laboratory of Digital Medical Engineering and Jiangsu Province Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
Hong Zeng*
Affiliation:
The State Key Laboratory of Digital Medical Engineering and Jiangsu Province Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
Xiao Li
Affiliation:
The State Key Laboratory of Digital Medical Engineering and Jiangsu Province Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
Guozheng Xu
Affiliation:
College of Automation Engineering, Nanjing University of Posts and Telecommunications, Nanjing, China
Yongqiang Li
Affiliation:
Center of Rehabilitation Medicine, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
Aiguo Song
Affiliation:
The State Key Laboratory of Digital Medical Engineering and Jiangsu Province Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, China
*
Corresponding author: Hong Zeng; Email: hzeng@seu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

The assist-as-needed (AAN) controller is effective in robot-assisted rehabilitation. However, variations of the engagement of subjects with fixed controller often lead to unsatisfying results. Therefore, adaptive AAN that adjusts control parameters based on individualized engagement is essential to enhance the training effect further. Nevertheless, current approaches mainly focus on the within-trial real-time engagement estimation, and the presence of measurement noise may cause improper evaluation of engagement. In addition, most studies on human-in-loop optimization strategies modulate the controller by greedy strategies, which are prone to fall into local optima. These shortcomings in previous studies could significantly limit the efficacy of AAN. This paper proposes an adaptive AAN to promote engagement by providing subjects with a subject-adaptive assistance level based on trial-wise engagement estimation and performance. Firstly, the engagement is estimated from energy information, which assesses the work done by the subject during a full trial to reduce the influence of measurement outliers. Secondly, the AAN controller is adapted by Bayesian optimization (BO) to maximize the subject’s performance according to historical trial-wise performance. The BO algorithm is good at dealing with noisy signals within limited steps. Experiments with ten healthy subjects resulted in a decrease of 34.59$\%$ in their average trajectory error, accompanied by a reduction of 9.71$\%$ in their energy consumption, thus verifying the superiority of the proposed method to prior work. These results suggest that the proposed method could potentially improve the effect of upper limb rehabilitation.

Keywords

adaptive AANengagement estimationBayesian optimization
Type
Research Article
Information
Robotica , Volume 41 , Issue 10 , October 2023 , pp. 3101 - 3115
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Feigin, V. L., Forouzanfar, M. H., Krishnamurthi, R., Mensah, G. A., Connor, M., Bennett, D. A., Moran, A. E., Sacco, R. L., Anderson, L., Truelsen, T., O’Donnell, M., Venketasubramanian, N., Barker-Collo, S., Lawes, C. M. M., Wang, W., Shinohara, Y., Witt, E., Ezzati, M., Naghavi, M., Murray, C. and Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010) and the GBD Stroke Experts Group, “Global and regional burden of stroke during 1990-2010: Findings from the Global Burden of Disease Study 2010,” Lancet 383(9913), 245–255 (2014).CrossRefGoogle ScholarPubMed
Pérez-Ibarra, J. C., Siqueira, A. A. G., Silva-Couto, M. A., de Russo, T. L. and Krebs, H. I., “Adaptive impedance control applied to robot-aided neuro-rehabilitation of the ankle,” IEEE Robot. Autom. Lett. 4(2), 185–192 (2018).CrossRefGoogle Scholar
Komura, H., Kubo, T., Honda, M. and Ohka, M., “Degree of muscle-and-tendon tonus effects on kinesthetic illusion in wrist joints toward advanced rehabilitation robotics,” Robotica 40(4), 1222–1232 (2022).CrossRefGoogle Scholar
Oyman, E., Korkut, M., Yilmaz, C., Bayraktaroglu, Z. and Arslan, M., “Design and control of a cable-driven rehabilitation robot for upper and lower limbs,” Robotica 40(1), 1–37 (2022).CrossRefGoogle Scholar
Talat, H., Munawar, H., Hussain, H. and Azam, U., “Design, modeling and control of an index finger exoskeleton for rehabilitation,” Robotica 40(10), 3514–3538 (2022).CrossRefGoogle Scholar
Davarzani, S., Ahmadi-Pajouh, M. and Ghafarirad, H., “Design of sensing system for experimental modeling of soft actuator applied for finger rehabilitation,” Robotica 40(7), 2091–2111 (2022).CrossRefGoogle Scholar
Hogan, N., Krebs, H. I., Rohrer, B., Palazzolo, J. J., Dipietro, L., Fasoli, S. E., Stein, J., Hughes, R., Frontera, W. R., Lynch, D., Volpe, B. T., “Motions or muscles? Some behavioral factors underlying robotic,” J. Rehabil. Res. Dev. 43(7), 601–618 (2006).CrossRefGoogle ScholarPubMed
Li, Z., Li, G., Wu, X., Kan, Z., Su, H. and Liu, Y., “Asymmetric cooperation control of dual-arm exoskeletons using human collaborative manipulation models,” IEEE Trans. Cybern. 52(11), 12126–12139 (2022).CrossRefGoogle ScholarPubMed
Warraich, Z. and Kleim, J. A., “Neural plasticity: The biological substrate for neurorehabilitation,” PM&R 2(12), S208–S219 (2010).Google ScholarPubMed
Emken, J. L., Benitez, R. and Reinkensmeyer, D. J., “Human-robot cooperative movement training: Learning a novel sensory motor transformation during walking with robotic assistance-as-needed,” J. Neuroeng. Rehabil. 4(1), 1–16 (2007).CrossRefGoogle ScholarPubMed
Chowdhury, A., Nishad, S. S., Meena, Y. K., Dutta, A. and Prasad, G., “Hand-exoskeleton assisted progressive neurorehabilitation using impedance adaptation based challenge level adjustment method,” IEEE Trans. Haptics 12(2), 128–140 (2018).CrossRefGoogle Scholar
Wang, J., Wang, W., Ren, S., Shi, W. and Hou, Z. G., “Engagement enhancement based on human-in-the-loop optimization for neural rehabilitation,” Front. Neurorobot. 14, 596019 (2020).CrossRefGoogle ScholarPubMed
Li, C., Rusak, Z., Horvath, I., Kooijman, A. and Ji, L., “Implementation and validation of engagement monitoring in an engagement enhancing rehabilitation system,” IEEE Trans. Neural Syst. Rehabil. Eng. 25(6), 726–738 (2016).CrossRefGoogle Scholar
Kiguchi, K. and Hayashi, Y., “An EMG-based control for an upper-limb power-assist exoskeleton robot,” IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(4), 1064–1071 (2012).CrossRefGoogle ScholarPubMed
Han, H., Wang, W., Zhang, F., Li, X., Chen, J., Han, J. and Zhang, J., “Selection of muscle-activity-based cost function in human-in-the-loop optimization of multi-gait ankle exoskeleton assistance,” IEEE Trans. Neural Syst. Rehabil. Eng. 29, 944–952 (2021).CrossRefGoogle ScholarPubMed
Li, Z., Xu, C., Wei, Q., Shi, C. and Su, C.-Y., “Human-inspired control of dual-arm exoskeleton robots with force and impedance adaptation,” IEEE Trans. Syst. Man Cybern.: Syst. 50(12), 5296–5305 (2020).CrossRefGoogle Scholar
Sanner, R. M. and Slotine, J. E., “Gaussian networks for direct adaptive control,” IEEE Trans. Neural Netw. 3(6), 837–863 (1992).CrossRefGoogle ScholarPubMed
Sanner, R. M. and Kosha, M., “A mathematical model of the adaptive control of human arm motions,” Biol. Cybern. 80(5), 369–382 (1999).CrossRefGoogle ScholarPubMed
Guidali, M., Schlink, P., Duschau-Wicke, A. and Riener, R.. Online Learning and Adaptation of Patient Support During ADL Training. In: 2011 IEEE International Conference on Rehabilitation Robotics, IEEE, (2011).Google Scholar
Pehlivan, A. U., Losey, D. P. and O’Malley, M. K., “Minimal assist-as-needed controller for upper limb robotic rehabilitation,” IEEE Trans. Robot. 32(1), 113–124 (2015).CrossRefGoogle Scholar
Pehlivan, A. U., Losey, D. P., Rose, C. G. and O’Malley, M. K.. Maintaining Subject Engagement During Robotic Rehabilitation with a Minimal Assist-as-Needed (mAAN) Controller. In: 2017 International Conference on Rehabilitation Robotics (ICORR), IEEE, (2017).Google Scholar
Wolbrecht, E. T., Chan, V., Reinkensmeyer, D. J. and Bobrow, J. E., “Optimizing compliant, model-based robotic assistance to promote neurorehabilitation,” IEEE Trans. Neural Sys. Rehabil. Eng. 16(3), 286–297 (2008).CrossRefGoogle ScholarPubMed
Zhang, Y., Li, S., Nolan, K. J. and Zanotto, D., “Shaping individualized impedance landscapes for gait training via reinforcement learning,” IEEE Trans. Med. Robot. Bionics 4(1), 194–205 (2021).CrossRefGoogle Scholar
Hocine, N., Gouaïch, A., Di Loreto, I. and Joab, M.. Motivation Based Difficulty Adaptation for Therapeutic Games. In: 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH), IEEE, (2011).Google Scholar
Rodrigues, L. and Gonçalves, R., “Development of a novel body weight support system for gait rehabilitation,” Robotica 41(4), 1275–1294 (2023).CrossRefGoogle Scholar
Arefeen, A. and Xiang, Y., “Subject specific optimal control of powered knee exoskeleton to assist human lifting tasks under controlled environment,” Robotica. First View, 1–20 (2023).Google Scholar
Agarwal, P. and Deshpande, A. D., “A framework for adaptation of training task, assistance and feedback for optimizing motor (re)-learning with a robotic exoskeleton,” IEEE Robot. Autom. Lett. 4(2), 808–815 (2019).CrossRefGoogle Scholar
Grimm, F., Naros, G. and Gharabaghi, A., “Closed-loop task difficulty adaptation during virtual reality reach-to-grasp training assisted with an exoskeleton for stroke rehabilitation,” Front. Neurosci. 10, 518 (2016).CrossRefGoogle ScholarPubMed
Cen, Y., Yuan, J., Ma, S., Luo, J. and Wang, H.. “Trajectory Optimization Algorithm of Trajectory Rehabilitation Training Mode for Rehabilitation Robot,” 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), Jinhong, China (2022) pp. 2153–2158.Google Scholar
Stroppa, F., Marcheschi, S., Mastronicola, N., Loconsole, C. and Frisoli, A.. Online Adaptive Assistance Control in Robot-Based Neurorehabilitation Therapy. In: 2017 International Conference on Rehabilitation Robotics (ICORR), IEEE, (2017).Google Scholar
Ozkul, F., Palaska, Y., Masazade, E. and Erol‐Barkana, D., “Exploring dynamic difficulty adjustment mechanism for rehabilitation tasks using physiological measures and subjective ratings,” IET Signal Process. 13(3), 378–386 (2019).CrossRefGoogle Scholar
Shirzad, N. and Van der Loos, H. F. M.. Adaptation of Task Difficulty in Rehabilitation Exercises Based on the User’s Motor Performance and Physiological Responses. In: 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR), IEEE, (2013).Google Scholar
Luo, L., Peng, L., Wang, C. and Hou, Z. G., “A greedy assist-as-needed controller for upper limb rehabilitation,” IEEE Trans. Neural Netw. Learn. Syst. 30(11), 3433–3443 (2019).CrossRefGoogle ScholarPubMed
Daniel, C. R., Yazbek, P., Santos, A. C. A. and Battistella, L. R., “Validity study of a triaxial accelerometer for measuring energy expenditure in stroke inpatients of a physical medicine and rehabilitation center,” Top. Stroke Rehabil. 30, 402–409 (2023).CrossRefGoogle ScholarPubMed
Brochu, E., Cora, V. M. and De Freitas, N., “A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning,” arXiv preprint arXiv. 1012.2599 (2010).Google Scholar
Kushner, H. J., “A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise,” J. Fluids Eng. 64(1), 97–106 (1964).Google Scholar
Li, G., Li, Z. and Kan, Z., “Assimilation control of a robotic exoskeleton for physical human-robot interaction,” IEEE Robot. Autom. Lett. 7(2), 2977–2984 (2022).CrossRefGoogle Scholar
Liu, X., Jiang, W., Su, H., Qi, W. and Ge, S. S., “A control strategy of robot eye-head coordinated gaze behavior achieved for minimized neural transmission noise,” IEEE/ASME Trans. Mechatron. 28(2), 956–966 (2023).CrossRefGoogle Scholar
Bull, A. D., “Convergence rates of efficient global optimization algorithms,” J. Mach. Learn. Res. 12(10), 2879–2904 (2011).Google Scholar
Abdelhameed, E. H., Sato, N. and Morita, Y.. Design of a Variable Resistance Training System Using Rotary Magneto-Rheological Brake. In: 2017 3rd International Conference on Control, Automation and Robotics (ICCAR), IEEE, (2017).Google Scholar
Agarwal, P. and Deshpande, A. D.. Impedance and Force-Field Control of the Index Finger Module of a Hand Exoskeleton for Rehabilitation. In: 2015 IEEE International Conference on Rehabilitation Robotics (ICORR), IEEE, (2015).Google Scholar
Liu, X., Maghlakelidze, G., Zhou, J., Izadi, O. H., Shen, L., Pommerenke, M., Ge, S. S. and Pommerenke, D., “Detection of ESD-induced soft failures by analyzing linux kernel function calls,” IEEE Trans. Device Mater. Reliab. 20(1), 128–135 (2020).CrossRefGoogle Scholar
Sebastian, G., Li, Z., Crocher, V., Kremers, D., Tan, Y. and Oetomo, D., “Interaction force estimation using extended state observers: An application to impedance-based assistive and rehabilitation robotics,” IEEE Robot. Autom. Lett. 4(2), 1156–1161 (2019).CrossRefGoogle Scholar
Asl, H. J., Yamashita, M., Narikiyo, T. and Kawanishi, M., “Field-based assist-as-needed control schemes for rehabilitation robots,” IEEE/ASME Trans. Mechatron. 25(4), 2100–2111 (2020).CrossRefGoogle Scholar
Verdel, D., Bastide, S., Vignais, N., Bruneau, O. and Berret, B., “An identification-based method improving the transparency of a robotic upper limb exoskeleton,” Robotica 39(9), 1711–1728 (2021).CrossRefGoogle Scholar
Olsson, H., Åström, K. J., De Wit, C. C., Gäfvert, M. and Lischinsky, P., “Friction models and friction compensation,” Eur. J. Control 4(3), 5517–5522 (1998).CrossRefGoogle Scholar
Lenze, E. J., Munin, M. C., Quear, T., Dew, M. A., Rogers, J. C., Begley, A. E. and Reynolds, C. F. III, “The Pittsburgh rehabilitation participation scale: Reliability and validity of a clinician-rated measure of participation in acute rehabilitation,” Arch. Phys. Med. Rehabil. 85(3), 380–384 (2004).CrossRefGoogle ScholarPubMed
Huang, J., Li, C., Cui, Z., Zhang, L. and Dai, W., “An improved grasshopper optimization algorithm for optimizing hybrid active power filters’ parameters,” IEEE Access 99, 1 (2020).Google Scholar
Luong, P., Nguyen, D., Gupta, S., Rana, S. and Venkatesh, S., “Adaptive cost-aware Bayesian optimization,” Knowl.-Based Syst. 232, 107481 (2021).CrossRefGoogle Scholar
Toscano-Palmerin, S. and Frazier, P. I., “Bayesian optimization with expensive integrands,” arXiv preprint arXiv. 08661, (2018).Google Scholar
Ding, Y., Kim, M., Kuindersma, S. and Walsh, C. J., “Human-in-the-loop optimization of hip assistance with a soft exosuit during walking,” Sci. Robot. 3(15), eaar5438 (2018).CrossRefGoogle ScholarPubMed
Snoek, J., Larochelle, H. and Adams, R. P., “Practical Bayesian optimization of machine learning algorithms,” Adv. Neural Inf. Process. Syst. 4, 2951–2959 (2012).Google Scholar
Stegeman, D. and Hermens, H., “Standards for surface electromyography: The European project Surface EMG for non-invasive assessment of muscles (SENIAM),” Enschede: Roessingh Res. Dev. 10, 8–12 (2007).Google Scholar
Qi, W., Liu, X., Zhang, L., Wu, L., Zang, W. and Su, H., “Adaptive sensor fusion labeling framework for hand pose recognition in robot teleoperation,” Assem. Autom. 41(3), 393–400 (2021).CrossRefGoogle Scholar
Todorov, E., “Optimality principles in sensorimotor control,” Nat. Neurosci. 7(9), 907–915 (2004).CrossRefGoogle ScholarPubMed