A research team at the Institute of Cognitive Research (ICS) at the TU Munchen along with a Japanese team have developed a Mask-Bot, which is a robotic face that communicates with humans.
Dr. Takaaki Kuratate interacting with Mask-bot
The Mask-Bot is capable of fluttering its eyelids, moving its head and raising its eyebrows in response to human speech or reactions. With the Mask-Bot, one can project any number of three-dimensional faces that look very realistic, similar to a human face and can also change the projected face on request. Dr. Takaaki Kuratate explained that the creators of Mask-Bot have equipped it with capabilities of projecting a face using a rear-board to provide realistic face-to-face interaction.
The Mask-Bot is able to provide seamless interaction even within short distances and projects images that are bright enough even during daytime. These bright images are produced with the help of a small projector and a coat of luminous paint on the inner side of the plastic mask. Kuratate explained that the mask-bot can provide a new face to video conferencing sessions by projecting realistic images of the person, which seem like they are sitting in front of you and talking. Custom-made masks can be made to represent male and female participants in the conference. A special program enables the conversion of a two-dimensional photograph into a three-dimensional mask and algorithms are in place for producing facial expressions and voices with the help of a head animation engine. This animation engine is capable of filtering a series of facial motion data and choosing from them the one that best suits the sounds emitted during speech. The mask-bot is also capable of reproducing keyboard input in English and Japanese and will soon be able to read German as well. There is a text-to-speech system, which is responsible for producing male or female voices in response to audio signals along with emotions with a simple touch of a button.