Aug 28 2020
A new study by researchers at University College London has discovered a technique to enhance the accuracy of a brain-inspired computing system. Thanks to the discovery, extremely energy-efficient artificial intelligence (AI) has moved a step closer to reality.
A wafer filled with memristors. Image Credit: University College London.
The system uses memristors to develop artificial neural networks and is nearly 1,000 times more energy efficient compared to traditional transistor-based AI hardware. Yet, so far, it has been more error-prone.
Currently available AI is highly energy-intensive—284 tonnes of carbon dioxide is produced in training one AI model, which is equivalent to the lifetime emissions of five cars. Modifying all digital devices by using memristors—an innovative electronic device first developed in 2008, as alternatives to the transistors could decrease the emission to a fraction of a tonne of carbon dioxide, which is equivalent to just emissions produced in an afternoon’s drive.
Memristors are highly energy-efficient compared to present-day computing systems, so they can potentially pack enormous amounts of computing power into hand-held devices, eliminating the need to be connected to the Internet.
This is considered specifically crucial because over-dependence on the Internet is anticipated to turn problematic in the years to come because of the ever-increasing demand for data and the challenges posed by increasing data transmission capacity beyond a specific point.
In the new research, reported in Nature Communications, engineers from UCL discovered that it is possible to considerably enhance the accuracy by making memristors work together in multiple sub-groups of neural networks and averaging their calculations, which implies that defects in each of the networks could be canceled out.
Memristors, or the so-called “resistors with memory” as they can remember the amount of electric charge that passed through them even after being switched off, were regarded a breakthrough when they were first developed about 10 years ago—a “missing link” in the field of electronics to supplement the resistor, inductor, and capacitor.
Since then, they have been commercially manufactured in memory devices. However, the study researchers note that they could be employed to create AI systems within the next three years.
Memristors provide much more enhanced efficiency as they work not only in a binary code of ones and zeros but also at several levels between zero and one simultaneously, which implies that more information can be packed into each bit.
Furthermore, memristors are usually described as a neuromorphic (brain-inspired) form of computing since, quite similar to what happens in the brain, memory and processing are executed in the same adaptive building blocks, as opposed to existing computer systems that waste a lot of energy during data movement.
As part of the study, Dr Adnan Mehonic, PhD student Dovydas Joksas (both UCL Electronic & Electrical Engineering), and collaborators from the United Kingdom and the United States tested the new technique in memristors of various types and discovered that it enhanced the accuracy of all of them, irrespective of specific memristor technology or material. It also worked for many different problems that may hamper the accuracy of the memristors.
The team identified that their method improved the accuracy of the neural networks for standard AI tasks to a comparable level to software tools that work on traditional digital hardware.
We hoped that there might be more generic approaches that improve not the device-level, but the system-level behaviour, and we believe we found one. Our approach shows that, when it comes to memristors, several heads are better than one. Arranging the neural network into several smaller networks rather than one big network led to greater accuracy overall.
Dr Adnan Mehonic, Director of the Study, University College London
According to Dovydas Joksas, “We borrowed a popular technique from computer science and applied it in the context of memristors. And it worked! Using preliminary simulations, we found that even simple averaging could significantly increase the accuracy of memristive neural networks.”
We believe now is the time for memristors, on which we have been working for several years, to take a leading role in a more energy-sustainable era of IoT devices and edge computing.
Tony Kenyon, Study Co-Author and Professor, Department of Electronic and Electrical Engineering, University College London
Journal Reference:
Joksas, D., et al. (2020) Committee machines—a universal method to deal with non-idealities in memristor-based neural networks. Nature Communications. doi.org/10.1038/s41467-020-18098-0.