Application of Robotics and Machine Learning to Assist Infants with Cerebral Palsy

Andrew H. Fagg, Professor of Computer Science and Bioengineering at the University of Oklahoma talks to Kal Kaur, Editor of AZoRobotics about the application of robotics and machine learning to assist infants with Cerebral Palsy.

Your team of researchers have been awarded with a grant to combine robotics, machine learning and brain imaging solutions to help children with cerebral palsy (CP) perform movements required for crawling. Can you discuss the aim of your research and how you plan on exploring this research area?

Cerebral Palsy is an affliction that is due to damage to motor areas of the brain that occurs sometime around the time of birth. This damage affects the transmission of motor signals to the muscles, resulting in poor muscle strength and motor coordination.  Although this damage does not continue to get worse with time, it has a dramatic effect on an infant's ability to learn key skills, including crawling and walking.

Development of crawling behavior is substantially delayed, and many infants with CP never learn to walk. These abilities to move around one's environment are critical to cognitive development: infants equipped with these skills must begin to reason about and remember objects and people who are out of reach or around the corner.  When development of these locomotory abilities is delayed, cognitive development is also delayed.  The effects of these delays can sometimes be seen into adulthood.

Our aim in the previous and current work is to develop robotic systems that aid the infant in locomotion and that encourage the infant to practice locomotory skills that improve muscle strength and coordination. In taking these steps, we hope to encourage the learning of unassisted locomotory skills and to bring the development of cognitive skills to a schedule that is more representative of typically developing infants.

In addition, in the current project, a key aim is to track the macro-scale changes that take place within the infant's brain as the new locomotion skills are learned.

How will the application of robotics help assist individual movements for crawling behaviour?

Infants start the process of learning how to crawl by making random exploratory movements of their limbs and trunk. Occasionally, these movements lead to an unexpected result: their body might roll, twist or even scoot a little bit. This unexpected event is a surprise to the infant - and it is rewarding. Their brains are driven to identify the cause of the rewarding event by attempting to repeat recently generated movements.

In addition to supporting the weight of the infant (which directly aids locomotion), our robots will provide an artificial form of the rewarding events.

Initially, small passive movements of the robot (caused by the infant pushing against the ground) will be amplified automatically by the robot, carrying the infant over a distance that he may not be able to achieve on his own. Secondly, we will recognize arm and leg movements that resemble movements that are exhibited during crawling. When such a limb movement is recognized, the robot will provide a corresponding movement of the body (e.g., a sweep of the right arm from left to right might trigger a small, leftward turn of the robot).

What robot platforms does the research team plan on developing to allow for a greater range of infant mobility?

The current robotic prototype resembles a motorized skateboard. The infant lays down on the robot in a prone configuration (face down), with the arms and legs hanging off of the robot's support structure. This robot is non-holonomic: it can instantaneously produce angular accelerations (about the gravity vector) and forward/backward accelerations.  The new robotic system will be holonomic, allowing acceleration in all three dimensions (forward/backward, left/right and rotational).

This new robotic platform will also differentially support the weight of the infant. Early in the developmental process, the robot will support all of the weight of the infant. However, as the motor strength and coordination of the infant increases, the robot will support less and less weight. The intent is that this will encourage the infant to bring his limbs into a configuration that will support this weight.

In a recent press release on this subject area, Professor David Miller from the OU departments of Aerospace and Mechanical Engineering and Bioengineering, stated that the research team “will also start working with the transition from crawling to walking”. How do you plan on exploring this transition in children with CP using robot technology?

The transition from crawling to walking requires a large amount of lower-body strength and overall coordination to maintain balance.  The robot will provide physical support to help with both of these. As with the crawling robot, this will support locomotion early, allowing the infant to explore a new environment and encouraging practice.  As the infant gains strength and skill, the assistance provided by the robot will be reduced gradually, with the hope that the infant will reduce his reliance on the assistance.

It is important that robots used to help with the cognitive and physiological development of a child need to be tactile and sensitive when interacting with a child and monitoring a child’s behaviour. Thinking about your research, what aspects of the robot (i.e., what materials and sensors) have been used to help assist an infant with CP and what were/are the challenges associated with this application?

The new platform will use a 6 degree of freedom force/torque sensor between it and the infant to assess the forces and torques that the infant is applying to the robot through limb motion and through contacts with the floor. In turn, the robot will respond by assisting the infant to move in the direction that he is pushing.

In addition, we are developing a kinematic suit that is strapped to the infant before he is placed on the crawler.  This suit allows us to measure limb and trunk motions in real time. As discussed already, the robot will respond to motions that resemble crawling by moving the infant along the appropriate direction. This suit also gives us a way to measure the progress of the infant from week-to-week.

What is the next step of this research with regard to the level of help provided for these children and how will this impact the type of existing technology used to explore this? Will this technology impact the type of help provided by the carers?

We are in the process of developing machine learning algorithms that will help us to identify the common limb/trunk movement patterns that are involved in crawling behavior produced by typically developing infants. We plan to use these patterns as an initial set of templates toward which we hope to guide infants with CP through interaction with the robot.  This “guiding” process is also a subject of the current research.

At each step of the development processes, our goal is to be constantly challenging the infant, while ensuring that success is frequent enough to hold his interest in the learning process. In part, the kinematic suit has become possible in the last year due to the availability of small and relatively inexpensive, all-in-one inertial measurement units (IMUs).

Continuing to reduce the size and cost of these components will make it easier for us to distribute a larger number of these IMUs across the body, giving us a better picture of infant activity. If our approach is successful, we imagine that such robotic systems will augment the typical daily activities of human caregivers by providing regular, highly structured feedback and support as the infant is learning locomotory skills. (Note that we do not imagine that the caregiver will be out of the process, even when the infant is working with the robot).  

What changes or outcomes are you looking for to determine a positive change in the child’s development through the use of robots and machine learning? What parameters will help you understand how effective robotic and machine learning technology is in assisting infants with CP?

We measure the infant's success using a variety of metrics, and comparing these metrics to typically developing infants, as well as with infants with CP who do not receive the robotic intervention (or the full extent of it). Metrics include: distances travelled by the infant in a given period of time (both on and off the robot), the degree to which the infant supports his own weight, the organization of the activity of the limb and trunk, and the frequency to which the infant uses locomotory skills to pursue higher-order problems, such as retrieving distant toys.

Do you plan to explore the application of robots in assisting adults with CP?

Although an important question, assisting adults with CP is really a different set of problems: the forces required are very different and the level of action complexity is substantially higher. In addition, such assistants would be required chronically. If our current project is successful, our hope is that many of the CP infants will only require assistance for a relatively short period of time.

Under a couple of other projects, I am working with several different researchers on the questions of brain–machine interfaces. Such interfaces could (in the future) provide a high-bandwidth conduit through which motor commands and sensory feedback could be communicated between the brain of a human user and a robotic prosthesis or assistant.

How do you see the application of robotics evolving in helping children with disabilities?  

There are many exciting directions being pursued by a range of research groups.  For me, one of the very interesting problems is that of how to provide a safe and rich experience for the child that addresses the child's immediate needs while setting him up for long-lasting improvement, and perhaps even independence. This requires us to develop models of how development and learning are affected by disease and by robotic intervention. It also requires us to identify ways of using these models in real time to decide how best a robot should be interacting with the child to help him along the learning/development path.

About Andrew Fagg

Professor Andrew H. Fagg is the Associate Professor of Computer Science and Bioengineering at the University of Oklahoma. His research focuses on the relationships between biological systems and machines. In this area of symbiotic computing, he studies the interaction of humans with machines, machines as models of biological systems, and biological systems as inspiration for new robot control and learning techniques. Specific areas of interest include:

• Motor skill learning in robots and models of primate skill learning. I specifically work in the areas of reaching, grasping, and manipulation.
• Interplay of multiple learning systems, including supervised- and reinforcement-style learning algorithms, and robot learning through human interaction.
• Learning task-oriented representations.
• Brain-machine interfaces for advanced prosthetic devices.
• Interactive art.

Andrew graduated with a B.S. in Applied Mathematics, Computer Science Track (with honors) from Carnegie Mellon University in 1989. He went on to study for an M.S. in Computer Science from the University of Southern California, and stayed at USC for his PhD on computational modelling of the cortical processes involved in primate grasping, which he completed in 1996.