A breakthrough of a robot sensing its environment is needed to make human-robot interactions safer and more beneficial.
The inside of ProTac’s ‘skin’ is covered with reflective markers. Cameras at the ends of the robotic link can capture changes in the relative positions of the markers, which indicates physical touch. Moreover, the skin can be made entirely transparent by applying an external voltage, allowing these cameras to detect objects in close proximity. Image Credit: Van Anh Ho from Japan Advanced Institute of Science and Technology.
In a new study, scientists from the Japan Advanced Institute of Science and Technology (JAIST) came up with a novel robotic link with tactile and proximity sensing abilities.
The researchers developed a simulation and learning framework to train a robotic link to sense its environment, setting the stage for a future where humans and robots can work together harmoniously. In recent years, robots have advanced to the point where they can execute or assist humans in all kinds of tasks, and it is now possible to imagine them working closely with people.
To ensure that robots can interact safely and pleasantly with humans, they must be developed to be able to sense their environment in multiple ways, including sound, touch, and sight. Of these, tactile sensation is particularly crucial for robots that are likely to come into physical contact with humans during operation. However, the development of large-scale tactile sensors has been challenging.
To address this, a research team led by Associate Professor Van Anh Ho from the Japan Advanced Institute of Science and Technology (JAIST) developed ProTac, a soft robotic link with proximity and tactile sensing abilities. ProTac consists of an outer "soft magic skin" that can be slightly deformed by touch without damage, and inside, it is patterned with arrays of reflective markers, with fisheye cameras installed at both ends of the robotic link looking towards the markers. The cameras capture changes in the relative positions of the markers upon physical contact and deformation of the skin, which are then processed to determine the precise location and intensity of the contact.
The team also developed SimTacLS, an open-source simulation and learning framework based on the SOFA and Gazebo physics engines, to train ProTac to make proximity and tactile measurements easily. This machine learning framework has been trained with simulated and experimental data that take into account the physics of soft contact and the realistic rendering of sensor images.
The development of ProTac and the SimTacLS framework is an important step towards enhancing human-robot interactions, as it enables robots to sense their environment in new ways, making it easier for them to work alongside humans.
SimTacLS enabled us to effectively implement tactile perception in robotic links without the high costs of complex experimental setups. Furthermore, with this framework, users can readily validate sensor designs and learning-based sensing performance before proceeding to actual fabrication and implementation.
Van Anh Ho, Associate Professor, Japan Advanced Institute of Science and Technology
In general, this work will help set the stage for a world where humans can harmoniously coexist and work besides robots.
We expect the proposed sensing device and framework to bring in ultimate solutions for the design of robots with softness, whole-body and multimodal sensing, and safety control strategies.
Van Anh Ho, Associate Professor, Japan Advanced Institute of Science and Technology
It is worth remarking that proposed methods could be extended to other kinds of robotic systems beyond the robotic manipulator illustrated in the study, like flying and mobile robots. Furthermore, ProTac or similar robotic links can be utilized to allow robotic manipulation in cluttered surroundings or when operating in the vicinity of humans.
Journal Reference
Luu, Q. K., et al. (2023) Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale. IEEE Transactions on Robotics. https://doi.org/10.1109/TRO.2023.3245983.