Jun 3 2021
A team of scientists in the Human Brain Project is using the EBRAINS research infrastructure to learn more about how the brain coordinates complex hand movements.
The physical Shadowhand is one of the world’s most advanced robotic hands. Shadowhand is a Partnering Project of the Human Brain Project © Shadow Robot Company
It is easy to take the dexterity of the human hand for granted. Every day we benefit greatly from our ability to manipulate objects with a high degree of control – whether we’re typing on a computer, playing a musical instrument, or even just turning a key to unlock a door.
While these tasks may seem simple, in-hand object manipulation is actually very complex and engages a large-scale brain network encompassing sensory, association, and motor regions.
Now, a team of scientists in the Human Brain Project is implementing this brain network as a biologically constrained recurrent convolutional neural network (RCNN) on EBRAINS. This type of network is often used to model sequence data. The network’s ability to learn sequences of states – for example, how a sequence of limb positions evolves when a certain force is applied to a joint – is especially relevant to robotics.
The design of the RCNN was informed by the EBRAINS Multilevel Human Brain Atlas, which allowed the researchers to identify the connection profile of brain regions involved in complex hand movements. The RCNN was then trained in-silico on supercomputers at the Swiss National Supercomputing Centre, which is part of the EBRAINS high-performance computing infrastructure.
The RCNN was tasked with generating a simple digit configuration – to join fingers with the thumb – which it successfully performed after 2,000 iterations of self-generated experience.
“The RCNN requires hours of simulated experience to train", says Mario Senden, one of the lead scientists. "The EBRAINS high-performance computing infrastructure allows us to speed up the process immensely and train the network in just a few minutes."
The team of scientists is now trying to train the network to perform in-hand object manipulation. “Our aim is to better understand the brain through biologically-inspired functional models and to provide brain-based alternatives to modern engineering solutions”, says Rainer Goebel, one of the head researchers.
Work is also being done to implement the Shadowhand robot – which performed the digit configurations in a simulator – on the EBRAINS Neurorobotics Platform.
“The unique combination of interoperable services on EBRAINS will allow us to break new ground in cognitive computational neuroscience", says Paweł Świeboda, CEO of EBRAINS and Director General of the HBP.
Katrin Amunts, the Scientific Director of the HBP emphasizes: "The workflow shown here incorporates a range of complementary EBRAINS services, including the multilevel human brain atlas, high-performance computing infrastructure, and, eventually, the Neurorobotics Platform. EBRAINS is the only research infrastructure in Europe that offers this powerful combination of services.”
Video: Human Dexterity: A Proof of Concept
Human Dexterity: A Proof of Concept
Using the EBRAINS research infrastructure, scientists of the Human Brain Project have simulated a robot hand which is able to perform human-like digit configurations. The work aims to shed light on how the brain coordinates complex hand movements. Video Credit: EBRAINS