Chinese scientists have developed a closed-loop human-machine interface (CL-HMI) which fixes flexible electronics onto human skin, enabling haptic feedback control of robots over Bluetooth, Wi-Fi and the internet.
Image Credit: Shutterstock.com/Damir Khabirov
Existing HMIs are bulky, expensive and limited in scope. Combining visual and haptic virtual reality and flexible skin-mounted electronics with closed-loop feedback control shows immense potential for contact-free interventions in medicine, industry and other fields.
A Brief Introduction to Human-Machine Interfaces
In broad terms, user interfaces (UIs) denote the space where humans interact with machines to control operations and processes. These machines may also provide feedback to humans via these interfaces.
Most of us are already familiar with keyboards, mice, computer monitors and printers. But user interfaces also include hand-operated tools and other industrial machinery. The concept of user interfaces can also be extended to the five senses, namely sight, sound, touch, smell and taste.
Composite user interfaces (CUIs) engage two or more of these senses. A tablet’s screen is one such interface, i.e., a graphical user interface (GUI) that engages the senses of touch and sight. Adding sound to a GUI turns it into a multimedia user interface. When a CUI blocks out the physical world, it becomes a virtual reality interface. Where it adds to physical interactions, it becomes an augmented reality interface.
Strictly speaking, a user interface describes human interactions with electronic devices. Where a human interfaces with a machine, or another piece of industrial equipment, it is a human-machine interface (HMI). There are countless types of interfaces, including gesture interfaces, conversational interfaces, command-line interfaces and motion tracking interfaces.
The development of these interfaces has largely been driven by the needs of individuals with physical disabilities. Brain-computer interfaces, for example, enable paralyzed individuals to move objects by imagining the movements of various parts of their bodies.
In a broader sense, human cognitive and perceptual abilities have inspired the development of machines that can respond to these abilities, whether through speech recognition, gesture recognition or others.
Next-Generation Human-Machine Interfaces (HMIs)
Flexible electronics have garnered extensive interest for decades. This really took off in the space age when flexible silicon wafers were developed for use in satellites.
Since then, organic semiconductors, conductive polymers and other materials have been developed for applications that require bending, stretching and other properties not attainable with conventional electronics. These developments enable the progress of bioinspired robotics, where devices can conform to complex surfaces.
Existing HMI applications are limited in wearability and comfort by rigid electronics and cumbersome equipment. Furthermore, without effective feedback mechanisms for dynamic control, complex tasks are unachievable.
The team from the City University of Hong Kong, Tsinghua University, Dalian University of Technology and the University of Electronic Science and Technology in China developed a closed-loop human-machine interface (CL-HMI) that integrates auditory, visual and haptic virtual reality (VR) into a robotic VR system.
Fig. 1. Design and architecture of the epidermal CL-HMI system. Image Credit: Yiming, L., et al. (2022) Electronic skin as wireless human-machine interfaces for robotic VR. Science Advances, [online]. Available at: https://www.science.org/doi/10.1126/sciadv.abl6700
Skin-tone elastomeric silicon serves as an adhesive interface to the user’s skin. Copper wires connect the battery, sensors and other electronics. Van der Waals forces ensure tight adhesion to the user’s body (including forearms, legs and back) over extended periods of time - up to 8 hours.
Wireless transmission enables the communication of instructions by the user to the remote machine. As long as this machine is connected to the internet, it can be located anywhere in the world.
Wireless transmission occurs over Bluetooth in less than 4 μs at distances ranging from 1 m to 5 m. In Wi-Fi mode, transmission can be extended over hundreds of meters over the same network. At 350 μs, in Wi-Fi mode, response time is shorter than the typical human response time of 550 μs.
Piezoresistive thin film sensors sitting on top of interdigital electrodes function as bending sensors. As they are mounted on the joints of the user’s body, they capture motion precisely by the resistance caused by different bending angles. Their movements also mirror the motions of the remote machine.
Skin-mounted vibratory actuators driven by Lorentz force effects feed the haptic feedback system. Five actuators and seven bending sensors are connected to analog-to-digital converters and general-purpose inputs/outputs. A lithium-ion battery ensures operation for up to two hours.
In the wake of the COVID-19 pandemic caused by SARS-CoV-2, the search for non-contact medical intervention procedures has become more urgent. In the Chinese study, a medical professional was able to conduct a nasopharyngeal swab by operating a seven-degree-of-freedom robotic hand.
This presents unique opportunities for non-contact interventions in medicine, industry and other areas.
References and Further reading
Yiming, L., et al. (2022) Electronic skin as wireless human-machine interfaces for robotic VR. Science Advances, [online]. Available at: https://www.science.org/doi/10.1126/sciadv.abl6700
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.