In a recent article published in the Journal of NeuroEngineering and Rehabilitation, researchers introduced an innovative deep reinforcement learning (RL)-based controller for lower limb rehabilitation exoskeletons (LLREs). This approach aims to provide autonomous and robust walking assistance to patients with various neuromuscular disorders
RL-based motion imitation control of the LLRE (the LLRE control loop in Fig. 2). The inputs of the motion imitation network consist of the joint state history, the action history and the future target motions. This learning network outputs joint target positions, which are processed by a low-pass filter and then translated into torque-level commands by PD control. Image Credit: https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-023-01147-2
Background
LLREs are wearable robots designed to provide gait assistance and functional substitution for individuals suffering from motor disorders or loss of motor control due to spinal cord injury, stroke, or other neuromuscular diseases. Designing robust controllers for LLREs that can manage the uncertain human-exoskeleton interaction forces from patients is a challenging task.
Conventional controllers often rely on trajectory tracking, model-based predictive control, or impedance control, which may be ineffective, unstable, or require laborious tuning of control parameters. In contrast, data-driven, RL-based controllers are emerging as a promising alternative, capable of learning optimal control policies from data without relying on accurate models or predefined rules.
About the Research
In this paper, the authors proposed a novel deep neural network and reinforcement learning (RL)-based robust controller for a lower limb rehabilitation exoskeleton (LLRE). They employed a decoupled human-exoskeleton simulation training technique utilizing three independent networks: a motion imitation network, an interaction network, and muscle coordination networks.
The motion imitation network learns a control policy for the LLRE to mimic a reference walking motion while maintaining balance and stability under the influence of human-exoskeleton interaction forces. The interaction network learns a control policy for the human to minimize interaction forces with the LLRE, considering the patient's desire to follow the exoskeleton's movements and reduce strap pressure.
The muscle coordination network learns a control policy for humans to coordinate muscle activations to produce joint accelerations close to the desired ones from the interaction network. These three networks are trained together through simulations using the state-of-the-art RL algorithm known as proximal policy optimization (PPO).
To increase robustness across different human conditions, the researchers employed domain randomization during training. They randomized human muscle strength, exoskeleton mechanical properties, motor command tracking accuracy, control delay, and observation latency. This approach aimed to produce a universal controller that can adapt to varying degrees of human disability and exoskeleton uncertainty without requiring any control parameter tuning.
Furthermore, the trained controller was tested in various simulated scenarios with human subjects exhibiting different neuromuscular disorders, such as muscle weakness, passive muscles (quadriplegic), or hemiplegic conditions. It was evaluated based on joint tracking accuracy, center of pressure (CoP)-based stability, gait symmetry, and reward statistics. Additionally, an ablation study was conducted to demonstrate the effects of different components of the controller, including the interaction network, the muscle coordination network, and domain randomization.
Research Findings
The outcomes showed that the new controller provided robust walking assistance under varying human-exoskeleton interactions and exoskeleton dynamics. It effectively handled different patients without needing any control parameter modifications. The controller demonstrated strong robustness against a wide range of exoskeleton dynamic properties and interaction forces.
The interaction and muscle coordination networks improved performance by reducing interaction forces and enhancing muscle coordination. Additionally, domain randomization enhanced the controller's generalization ability by exposing it to diverse environments during training.
Applications
The developed controller has significant potential for both home and clinical use of LLREs. It offers reliable walking assistance to patients with various neuromuscular disorders without requiring prior knowledge of the patient's condition. The controller adapts to different patient needs and conditions by handling uncertainties and variations in exoskeleton dynamics and human-exoskeleton interactions.
This controller can improve mobility and quality of life by allowing patients to perform daily activities without the use of crutches or other balance aids. It makes the rehabilitation process more efficient and less labor-intensive for both patients and therapists. In clinical settings, it can be integrated into rehabilitation programs to provide consistent and reliable assistance, leading to more personalized rehabilitation plans as the exoskeleton adjusts in real-time to the patient’s progress.
The technology also reduces the burden on caregivers and therapists by minimizing the need for constant supervision. It can be extended to other wearable robots, such as upper limb exoskeletons and prosthetic devices, enhancing their functionality and usability.
Furthermore, the robust and adaptable nature of this controller can lead to innovations in other fields, like industrial robotics, where precise and adaptive control is essential. The principles developed for this LLRE controller could inspire new approaches to designing controllers for various robotic systems that interact closely with humans.
Conclusion
In summary, the novel controller effectively leveraged the potential of deep learning for LLRE. It could be adapted for other types of wearable robots, such as upper limb exoskeletons or prosthetic devices.
The researchers suggested future directions for their work, including testing the controller on physical exoskeleton hardware, incorporating more realistic human models and exoskeleton designs, and extending the controller to other tasks and skills beyond walking.
Journal Reference
Luo, S., Androwis, G., Adamovich, S. et al. Robust walking control of a lower limb rehabilitation exoskeleton coupled with a musculoskeletal model via deep reinforcement learning. J NeuroEngineering Rehabil 20, 34 (2023). https://doi.org/10.1186/s12984-023-01147-2, https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-023-01147-2.
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.
Article Revisions
- Jun 21 2024 - Title changed from "Universal Neural Network Controller for Rehabilitation Exoskeletons " to "AI Improves Gait Assistance for Neuromuscular Disorders"