Researchers Develop a Unique Robotic System that Feeds Desired Food Items to People

According to the 2010 census data from, around 1 million adults in the U.S. requires a caregiver to help them eat food.

Such a task is usually awkward, involves a great deal of time, and is mostly performed out of necessity instead of choice.

Now, at the University of Washington, scientists are working on a unique robotic system that can aid in making this task easier. The novel robot first identifies the various types of foods placed on a plate and then strategizes how to utilize a fork to pick up the food and deliver the required bite to an individual’s mouth.

The researchers published their findings in a series of papers—one has been recently reported in Automation Letters and IEEE Robotics, whereas the other will be presented at the ACM/IEEE International Conference on Human-Robot Interaction, which will be held in South Korea on March 13^th, 2019.

Being dependent on a caregiver to feed every bite every day takes away a person’s sense of independence. Our goal with this project is to give people a bit more control over their lives.

Siddhartha Srinivasa, Study Corresponding Author and Boeing Endowed Professor, Paul G. Allen School of Computer Science & Engineering, University of Washington.

The goal was to create a unique, autonomous feeding system that can possibly be fixed to the wheelchairs of people and feed them whatever they wished to eat.

When we started the project we realized: There are so many ways that people can eat a piece of food depending on its size, shape or consistency. How do we start? So we set up an experiment to see how humans eat common foods like grapes and carrots.

Tapomayukh Bhattacharjee, Study Co-Author and Postdoctoral Research Associate, Paul G. Allen School of Computer Science & Engineering, University of Washington.

About 12 different types of food, ranging in consistency from soft bananas to hard carrots, were arranged in plates by the researchers. Foods like grapes and tomatoes, which have soft insides and a hard skin, were also included in the plates. Volunteers were then given a fork and asked to pick up varied pieces of foods to feed them to a mannequin. A sensor included in the fork measured the amount of force used by people when they pick up the food.

Various strategies were used by the volunteers to pick up food with varying consistencies. For instance, the volunteers skewered soft foods like bananas at a specific angle so that they do not slip off the fork, and for items like grapes and carrots, they tended to utilize wiggling motions to boost the force and spear every bite.

“People seemed to use different strategies not just based on the size and shape of the food but also how hard or soft it is. But do we actually need to do that?” said Bhattacharjee. “We decided to do an experiment with the robot where we had it skewer food until the fork reached a certain depth inside, regardless of the type of food.”

The same force-and-skewering strategy was used by the new robot in an attempt to pick up all the items of food, irrespective of their consistency. While the robot found it easy to pick up hard foods, it struggled with foods that had soft insides and tough skins and also with soft foods. Therefore, just like humans, robots have to adjust the amount of angle and force they apply to pick up different types of food.

The researchers also observed that the task of picking up a specific kind of food and feeding it to someone are dependent on each other. Usually, volunteers would particularly place a piece of food on the fork so that it can be easily eaten.

You can pick up a carrot stick by skewering it in the center of the stick, but it will be difficult for a person to eat. On the other hand, if you pick it up on one of the ends and then tilt the carrot toward someone's mouth, it's easier to take a bite.

Tapomayukh Bhattacharjee, Study Co-Author and Postdoctoral Research Associate, Paul G. Allen School of Computer Science & Engineering, University of Washington.

Therefore, the investigators combined two varied algorithms to develop a skewering and feeding strategy that alters based on the kinds of food items. Initially, they utilized RetinaNet—an object-detection algorithm that is capable of scamming the plate, identifying the kinds of food arranged on it, and placing a frame around each food item. The researchers subsequently developed an algorithm called SPNet that analyzes the kind of food in a particular frame and instructs the robot the most optimized way to pick up the food item. For instance, the SPNet algorithm informs the robot to skewer a slice of banana or strawberry in the center, and spear carrots at one of the either end.

The researchers made the robot to pick up food items and feed them to volunteers using the SPNet algorithm or a more uniform approach—a method that skewered the middle of each food item irrespective of what it was. The different strategies of SPNet outpaced or executed the same as the uniform method for all the food items.

Many engineering challenges are not picky about their solutions, but this research is very intimately connected with people. If we don't take into account how easy it is for a person to take a bite, then people might not be able to use our system. There's a universe of types of food out there, so our biggest challenge is to develop strategies that can deal with all of them.

Siddhartha Srinivasa, Study Corresponding Author and Boeing Endowed Professor, Paul G. Allen School of Computer Science & Engineering, University of Washington.

At present, the researchers are working with the Taskar Center for Accessible Technology to receive feedback from caregivers as well as patients residing in assisted living facilities on how to further develop the system to match the requirements of people.

“Ultimately our goal is for our robot to help people have their lunch or dinner on their own,” stated Srinivasa. “But the point is not to replace caregivers: We want to empower them. With a robot to help, the caregiver can set up the plate, and then do something else while the person eats.”

The first paper’s co-authors include research scientist Hanjun Song and doctoral student Gilwoo Lee, both at the Allen School. The second paper’s co-authors include Daniel Gallenberger, a master’s student at Technische Universität München in Germany who executed this study while at the UW, and Youngsun Kim, a research scientist at the Allen School.

The study was initially unveiled at the recent NeurIPS conference, where it received an award for best demonstration.

The National Institutes of Health, the National Science Foundation, the Office of Naval Research, the Robotics Collaborative Technology Alliance, Amazon, and Honda funded the research.

(Video credit: University of Washington)